Add credentials using a generic EncryptedConfiguration class (#30067)

* WIP: Add credentials using a generic EncryptedConfiguration class

This is sketch code so far.

* Flesh out EncryptedConfiguration and test it

* Better name

* Add command and generator for credentials

* Use the Pathnames

* Extract EncryptedFile from EncryptedConfiguration and add serializers

* Test EncryptedFile

* Extract serializer validation

* Stress the point about losing comments

* Allow encrypted configuration to be read without parsing for display

* Use credentials by default and base them on the master key

* Derive secret_key_base in test/dev, source it from credentials in other envs

And document the usage.

* Document the new credentials setup

* Stop generating the secrets.yml file now that we have credentials

* Document what we should have instead

Still need to make it happen, tho.

* [ci skip] Keep wording to `key base`; prefer defaults.

Usually we say we change defaults, not "spec" out a release.

Can't use backticks in our sdoc generated documentation either.

* Abstract away OpenSSL; prefer MessageEncryptor.

* Spare needless new when raising.

* Encrypted file test shouldn't depend on subclass.

* [ci skip] Some woordings.

* Ditch serializer future coding.

* I said flip it. Flip it good.

* [ci skip] Move require_master_key to the real production.rb.

* Add require_master_key to abort the boot process.

In case the master key is required in a certain environment
we should inspect that the key is there and abort if it isn't.

* Print missing key message and exit immediately.

Spares us a lengthy backtrace and prevents further execution.

I've verified the behavior in a test app, but couldn't figure the
test out as loading the app just exits immediately with:

```
/Users/kasperhansen/Documents/code/rails/activesupport/lib/active_support/testing/isolation.rb:23:in `load': marshal data too short (ArgumentError)
	from /Users/kasperhansen/Documents/code/rails/activesupport/lib/active_support/testing/isolation.rb:23:in `run'
	from /Users/kasperhansen/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/minitest-5.10.2/lib/minitest.rb:830:in `run_one_method'
	from /Users/kasperhansen/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/minitest-5.10.2/lib/minitest/parallel.rb:32:in `block (2 levels) in start'
```

It's likely we need to capture and prevent the exit somehow.
Kernel.stub(:exit) didn't work. Leaving it for tomorrow.

* Fix require_master_key config test.

Loading the app would trigger the `exit 1` per require_master_key's
semantics, which then aborted the test.

Fork and wait for the child process to finish, then inspect the
exit status.

Also check we aborted because of a missing master key, so something
else didn't just abort the boot.

Much <3 to @tenderlove for the tip.

* Support reading/writing configs via methods.

* Skip needless deep symbolizing.

* Remove save; test config reader elsewhere.

* Move secret_key_base check to when we're reading it.

Otherwise we'll abort too soon since we don't assign the secret_key_base
to secrets anymore.

* Add missing string literal comments; require unneeded yaml require.

* ya ya ya, rubocop.

* Add master_key/credentials after bundle.

Then we can reuse the existing message on `rails new bc4`.

It'll look like:

```
Using web-console 3.5.1 from https://github.com/rails/web-console.git (at master@ce985eb)
Using rails 5.2.0.alpha from source at `/Users/kasperhansen/Documents/code/rails`
Using sass-rails 5.0.6
Bundle complete! 16 Gemfile dependencies, 72 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Adding config/master.key to store the master encryption key: 97070158c44b4675b876373a6bc9d5a0

Save this in a password manager your team can access.

If you lose the key, no one, including you, can access anything encrypted with it.

      create  config/master.key
```

And that'll be executed even if `--skip-bundle` was passed.

* Ensure test app has secret_key_base.

* Assign secret_key_base to app or omit.

* Merge noise

* Split options for dynamic delegation into its own method and use deep symbols to make it work

* Update error to point to credentials instead

* Appease Rubocop

* Validate secret_key_base when reading it.

Instead of relying on the validation in key_generator move that into
secret_key_base itself.

* Fix generator and secrets test.

Manually add config.read_encrypted_secrets since it's not there by default
anymore.

Move mentions of config/secrets.yml to config/credentials.yml.enc.

* Remove files I have no idea how they got here.

* [ci skip] swap secrets for credentials.

* [ci skip] And now, changelogs are coming.

4 files changed, 72 insertions, 83 deletions

diff --git a/railties/test/application/configuration_test.rb b/railties/test/application/configuration_test.rb
index 64939e4ab4..c1a80eaeaf 100644
--- a/railties/test/application/configuration_test.rb
+++ b/railties/test/application/configuration_test.rb
@@ -476,45 +476,35 @@ module ApplicationTests
 
     test "application message verifier can be used when the key_generator is ActiveSupport::LegacyKeyGenerator" do
       app_file "config/initializers/secret_token.rb", <<-RUBY
+        Rails.application.credentials.secret_key_base = nil
         Rails.application.config.secret_token = "b3c631c314c0bbca50c1b2843150fe33"
       RUBY
-      app_file "config/secrets.yml", <<-YAML
-        development:
-          secret_key_base:
-      YAML
 
-      app "development"
+      app "production"
 
-      assert_equal app.env_config["action_dispatch.key_generator"], Rails.application.key_generator
-      assert_equal app.env_config["action_dispatch.key_generator"].class, ActiveSupport::LegacyKeyGenerator
+      assert_kind_of ActiveSupport::LegacyKeyGenerator, Rails.application.key_generator
       message = app.message_verifier(:sensitive_value).generate("some_value")
       assert_equal "some_value", Rails.application.message_verifier(:sensitive_value).verify(message)
     end
 
-    test "warns when secrets.secret_key_base is blank and config.secret_token is set" do
+    test "raises when secret_key_base is blank" do
       app_file "config/initializers/secret_token.rb", <<-RUBY
-        Rails.application.config.secret_token = "b3c631c314c0bbca50c1b2843150fe33"
+        Rails.application.credentials.secret_key_base = nil
       RUBY
-      app_file "config/secrets.yml", <<-YAML
-        development:
-          secret_key_base:
-      YAML
-
-      app "development"
 
-      assert_deprecated(/You didn't set `secret_key_base`./) do
-        app.env_config
+      error = assert_raise(ArgumentError) do
+        app "production"
       end
+      assert_match(/Missing `secret_key_base`./, error.message)
     end
 
-    test "raise when secrets.secret_key_base is not a type of string" do
-      app_file "config/secrets.yml", <<-YAML
-        development:
-          secret_key_base: 123
-      YAML
+    test "raise when secret_key_base is not a type of string" do
+      add_to_config <<-RUBY
+        Rails.application.credentials.secret_key_base = 123
+      RUBY
 
       assert_raise(ArgumentError) do
-        app "development"
+        app "production"
       end
     end
 
@@ -534,7 +524,7 @@ module ApplicationTests
 
     test "application verifier can build different verifiers" do
       make_basic_app do |application|
-        application.secrets.secret_key_base = "b3c631c314c0bbca50c1b2843150fe33"
+        application.credentials.secret_key_base = "b3c631c314c0bbca50c1b2843150fe33"
         application.config.session_store :disabled
       end
 
@@ -652,37 +642,15 @@ module ApplicationTests
 
     test "uses ActiveSupport::LegacyKeyGenerator as app.key_generator when secrets.secret_key_base is blank" do
       app_file "config/initializers/secret_token.rb", <<-RUBY
+        Rails.application.credentials.secret_key_base = nil
         Rails.application.config.secret_token = "b3c631c314c0bbca50c1b2843150fe33"
       RUBY
-      app_file "config/secrets.yml", <<-YAML
-        development:
-          secret_key_base:
-      YAML
 
-      app "development"
+      app "production"
 
       assert_equal "b3c631c314c0bbca50c1b2843150fe33", app.config.secret_token
-      assert_nil app.secrets.secret_key_base
-      assert_equal app.key_generator.class, ActiveSupport::LegacyKeyGenerator
-    end
-
-    test "uses ActiveSupport::LegacyKeyGenerator with config.secret_token as app.key_generator when secrets.secret_key_base is blank" do
-      app_file "config/initializers/secret_token.rb", <<-RUBY
-        Rails.application.config.secret_token = ""
-      RUBY
-      app_file "config/secrets.yml", <<-YAML
-        development:
-          secret_key_base:
-      YAML
-
-      app "development"
-
-      assert_equal "", app.config.secret_token
-      assert_nil app.secrets.secret_key_base
-      e = assert_raise ArgumentError do
-        app.key_generator
-      end
-      assert_match(/\AA secret is required/, e.message)
+      assert_nil app.credentials.secret_key_base
+      assert_kind_of ActiveSupport::LegacyKeyGenerator, app.key_generator
     end
 
     test "that nested keys are symbolized the same as parents for hashes more than one level deep" do
@@ -699,6 +667,20 @@ module ApplicationTests
       assert_equal "697361616320736c6f616e2028656c6f7265737429", app.secrets.smtp_settings[:password]
     end
 
+    test "require_master_key aborts app boot when missing key" do
+      skip "can't run without fork" unless Process.respond_to?(:fork)
+
+      remove_file "config/master.key"
+      add_to_config "config.require_master_key = true"
+
+      error = capture(:stderr) do
+        Process.wait(Process.fork { app "development" })
+      end
+
+      assert_equal 1, $?.exitstatus
+      assert_match(/Missing.*RAILS_MASTER_KEY/, error)
+    end
+
     test "protect from forgery is the default in a new app" do
       make_basic_app
 
diff --git a/railties/test/application/middleware/sendfile_test.rb b/railties/test/application/middleware/sendfile_test.rb
index 4731396029..9def3a0ce7 100644
--- a/railties/test/application/middleware/sendfile_test.rb
+++ b/railties/test/application/middleware/sendfile_test.rb
@@ -15,10 +15,6 @@ module ApplicationTests
       teardown_app
     end
 
-    def app
-      @app ||= Rails.application
-    end
-
     define_method :simple_controller do
       class ::OmgController < ActionController::Base
         def index
diff --git a/railties/test/application/middleware/session_test.rb b/railties/test/application/middleware/session_test.rb
index 15acfe93e9..a17988235a 100644
--- a/railties/test/application/middleware/session_test.rb
+++ b/railties/test/application/middleware/session_test.rb
@@ -337,31 +337,37 @@ module ApplicationTests
 
       add_to_config <<-RUBY
         # Use a static key
-        secrets.secret_key_base = "known key base"
+        Rails.application.credentials.secret_key_base = "known key base"
 
         # Enable AEAD cookies
         config.action_dispatch.use_authenticated_cookie_encryption = true
       RUBY
 
-      require "#{app_path}/config/environment"
+      begin
+        old_rails_env, ENV["RAILS_ENV"] = ENV["RAILS_ENV"], "production"
 
-      get "/foo/write_raw_session"
-      get "/foo/read_session"
-      assert_equal "1", last_response.body
+        require "#{app_path}/config/environment"
 
-      get "/foo/write_session"
-      get "/foo/read_session"
-      assert_equal "2", last_response.body
+        get "/foo/write_raw_session"
+        get "/foo/read_session"
+        assert_equal "1", last_response.body
 
-      get "/foo/read_encrypted_cookie"
-      assert_equal "2", last_response.body
+        get "/foo/write_session"
+        get "/foo/read_session"
+        assert_equal "2", last_response.body
 
-      cipher = "aes-256-gcm"
-      secret = app.key_generator.generate_key("authenticated encrypted cookie")
-      encryptor = ActiveSupport::MessageEncryptor.new(secret[0, ActiveSupport::MessageEncryptor.key_len(cipher)], cipher: cipher)
+        get "/foo/read_encrypted_cookie"
+        assert_equal "2", last_response.body
 
-      get "/foo/read_raw_cookie"
-      assert_equal 2, encryptor.decrypt_and_verify(last_response.body)["foo"]
+        cipher = "aes-256-gcm"
+        secret = app.key_generator.generate_key("authenticated encrypted cookie")
+        encryptor = ActiveSupport::MessageEncryptor.new(secret[0, ActiveSupport::MessageEncryptor.key_len(cipher)], cipher: cipher)
+
+        get "/foo/read_raw_cookie"
+        assert_equal 2, encryptor.decrypt_and_verify(last_response.body)["foo"]
+      ensure
+        ENV["RAILS_ENV"] = old_rails_env
+      end
     end
 
     test "session upgrading legacy signed cookies to new signed cookies" do
@@ -400,26 +406,32 @@ module ApplicationTests
 
       add_to_config <<-RUBY
         secrets.secret_token = "3b7cd727ee24e8444053437c36cc66c4"
-        secrets.secret_key_base = nil
+        Rails.application.credentials.secret_key_base = nil
       RUBY
 
-      require "#{app_path}/config/environment"
+      begin
+        old_rails_env, ENV["RAILS_ENV"] = ENV["RAILS_ENV"], "production"
 
-      get "/foo/write_raw_session"
-      get "/foo/read_session"
-      assert_equal "1", last_response.body
+        require "#{app_path}/config/environment"
 
-      get "/foo/write_session"
-      get "/foo/read_session"
-      assert_equal "2", last_response.body
+        get "/foo/write_raw_session"
+        get "/foo/read_session"
+        assert_equal "1", last_response.body
 
-      get "/foo/read_signed_cookie"
-      assert_equal "2", last_response.body
+        get "/foo/write_session"
+        get "/foo/read_session"
+        assert_equal "2", last_response.body
 
-      verifier = ActiveSupport::MessageVerifier.new(app.secrets.secret_token)
+        get "/foo/read_signed_cookie"
+        assert_equal "2", last_response.body
 
-      get "/foo/read_raw_cookie"
-      assert_equal 2, verifier.verify(last_response.body)["foo"]
+        verifier = ActiveSupport::MessageVerifier.new(app.secrets.secret_token)
+
+        get "/foo/read_raw_cookie"
+        assert_equal 2, verifier.verify(last_response.body)["foo"]
+      ensure
+        ENV["RAILS_ENV"] = old_rails_env
+      end
     end
 
     test "calling reset_session on request does not trigger an error for API apps" do
diff --git a/railties/test/application/url_generation_test.rb b/railties/test/application/url_generation_test.rb
index 4f962db6c4..f22b9fda3d 100644
--- a/railties/test/application/url_generation_test.rb
+++ b/railties/test/application/url_generation_test.rb
@@ -16,7 +16,6 @@ module ApplicationTests
       require "action_view/railtie"
 
       class MyApp < Rails::Application
-        secrets.secret_key_base = "3b7cd727ee24e8444053437c36cc66c4"
         config.session_store :cookie_store, key: "_myapp_session"
         config.active_support.deprecation = :log
         config.eager_load = false