Add credentials using a generic EncryptedConfiguration class (#30067)

* WIP: Add credentials using a generic EncryptedConfiguration class

This is sketch code so far.

* Flesh out EncryptedConfiguration and test it

* Better name

* Add command and generator for credentials

* Use the Pathnames

* Extract EncryptedFile from EncryptedConfiguration and add serializers

* Test EncryptedFile

* Extract serializer validation

* Stress the point about losing comments

* Allow encrypted configuration to be read without parsing for display

* Use credentials by default and base them on the master key

* Derive secret_key_base in test/dev, source it from credentials in other envs

And document the usage.

* Document the new credentials setup

* Stop generating the secrets.yml file now that we have credentials

* Document what we should have instead

Still need to make it happen, tho.

* [ci skip] Keep wording to `key base`; prefer defaults.

Usually we say we change defaults, not "spec" out a release.

Can't use backticks in our sdoc generated documentation either.

* Abstract away OpenSSL; prefer MessageEncryptor.

* Spare needless new when raising.

* Encrypted file test shouldn't depend on subclass.

* [ci skip] Some woordings.

* Ditch serializer future coding.

* I said flip it. Flip it good.

* [ci skip] Move require_master_key to the real production.rb.

* Add require_master_key to abort the boot process.

In case the master key is required in a certain environment
we should inspect that the key is there and abort if it isn't.

* Print missing key message and exit immediately.

Spares us a lengthy backtrace and prevents further execution.

I've verified the behavior in a test app, but couldn't figure the
test out as loading the app just exits immediately with:

```
/Users/kasperhansen/Documents/code/rails/activesupport/lib/active_support/testing/isolation.rb:23:in `load': marshal data too short (ArgumentError)
	from /Users/kasperhansen/Documents/code/rails/activesupport/lib/active_support/testing/isolation.rb:23:in `run'
	from /Users/kasperhansen/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/minitest-5.10.2/lib/minitest.rb:830:in `run_one_method'
	from /Users/kasperhansen/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/minitest-5.10.2/lib/minitest/parallel.rb:32:in `block (2 levels) in start'
```

It's likely we need to capture and prevent the exit somehow.
Kernel.stub(:exit) didn't work. Leaving it for tomorrow.

* Fix require_master_key config test.

Loading the app would trigger the `exit 1` per require_master_key's
semantics, which then aborted the test.

Fork and wait for the child process to finish, then inspect the
exit status.

Also check we aborted because of a missing master key, so something
else didn't just abort the boot.

Much <3 to @tenderlove for the tip.

* Support reading/writing configs via methods.

* Skip needless deep symbolizing.

* Remove save; test config reader elsewhere.

* Move secret_key_base check to when we're reading it.

Otherwise we'll abort too soon since we don't assign the secret_key_base
to secrets anymore.

* Add missing string literal comments; require unneeded yaml require.

* ya ya ya, rubocop.

* Add master_key/credentials after bundle.

Then we can reuse the existing message on `rails new bc4`.

It'll look like:

```
Using web-console 3.5.1 from https://github.com/rails/web-console.git (at master@ce985eb)
Using rails 5.2.0.alpha from source at `/Users/kasperhansen/Documents/code/rails`
Using sass-rails 5.0.6
Bundle complete! 16 Gemfile dependencies, 72 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Adding config/master.key to store the master encryption key: 97070158c44b4675b876373a6bc9d5a0

Save this in a password manager your team can access.

If you lose the key, no one, including you, can access anything encrypted with it.

      create  config/master.key
```

And that'll be executed even if `--skip-bundle` was passed.

* Ensure test app has secret_key_base.

* Assign secret_key_base to app or omit.

* Merge noise

* Split options for dynamic delegation into its own method and use deep symbols to make it work

* Update error to point to credentials instead

* Appease Rubocop

* Validate secret_key_base when reading it.

Instead of relying on the validation in key_generator move that into
secret_key_base itself.

* Fix generator and secrets test.

Manually add config.read_encrypted_secrets since it's not there by default
anymore.

Move mentions of config/secrets.yml to config/credentials.yml.enc.

* Remove files I have no idea how they got here.

* [ci skip] swap secrets for credentials.

* [ci skip] And now, changelogs are coming.

3 files changed, 154 insertions, 0 deletions

diff --git a/activesupport/lib/active_support/encrypted_configuration.rb b/activesupport/lib/active_support/encrypted_configuration.rb
new file mode 100644
index 0000000000..b403048627
--- /dev/null
+++ b/activesupport/lib/active_support/encrypted_configuration.rb
@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+require "yaml"
+require "active_support/encrypted_file"
+require "active_support/ordered_options"
+require "active_support/core_ext/object/inclusion"
+require "active_support/core_ext/module/delegation"
+
+module ActiveSupport
+  class EncryptedConfiguration < EncryptedFile
+    delegate :[], :fetch, to: :config
+    delegate_missing_to :options
+
+    def initialize(config_path:, key_path:, env_key:)
+      super content_path: config_path, key_path: key_path, env_key: env_key
+    end
+
+    # Allow a config to be started without a file present
+    def read
+      super
+    rescue ActiveSupport::EncryptedFile::MissingContentError
+      ""
+    end
+
+    def config
+      @config ||= deserialize(read).deep_symbolize_keys
+    end
+
+    private
+      def options
+        @options ||= ActiveSupport::InheritableOptions.new(config)
+      end
+
+      def serialize(config)
+        config.present? ? YAML.dump(config) : ""
+      end
+
+      def deserialize(config)
+        config.present? ? YAML.load(config) : {}
+      end
+  end
+end
diff --git a/activesupport/lib/active_support/encrypted_file.rb b/activesupport/lib/active_support/encrypted_file.rb
new file mode 100644
index 0000000000..1c4c1cb457
--- /dev/null
+++ b/activesupport/lib/active_support/encrypted_file.rb
@@ -0,0 +1,101 @@
+# frozen_string_literal: true
+
+require "pathname"
+require "active_support/message_encryptor"
+require "active_support/core_ext/string/strip"
+require "active_support/core_ext/module/delegation"
+
+module ActiveSupport
+  class EncryptedFile
+    class MissingContentError < RuntimeError
+      def initialize(content_path)
+        super "Missing encrypted content file in #{content_path}."
+      end
+    end
+
+    class MissingKeyError < RuntimeError
+      def initialize(key_path:, env_key:)
+        super \
+          "Missing encryption key to decrypt file with. " +
+          "Ask your team for your master key and write it to #{key_path} or put it in the ENV['#{env_key}']."
+      end
+    end
+
+    CIPHER = "aes-128-gcm"
+
+    def self.generate_key
+      SecureRandom.hex(ActiveSupport::MessageEncryptor.key_len(CIPHER))
+    end
+
+
+    attr_reader :content_path, :key_path, :env_key
+
+    def initialize(content_path:, key_path:, env_key:)
+      @content_path, @key_path = Pathname.new(content_path), Pathname.new(key_path)
+      @env_key = env_key
+    end
+
+    def key
+      read_env_key || read_key_file || handle_missing_key
+    end
+
+    def read
+      if content_path.exist?
+        decrypt content_path.binread
+      else
+        raise MissingContentError, content_path
+      end
+    end
+
+    def write(contents)
+      IO.binwrite "#{content_path}.tmp", encrypt(contents)
+      FileUtils.mv "#{content_path}.tmp", content_path
+    end
+
+    def change(&block)
+      writing read, &block
+    end
+
+
+    private
+      def writing(contents)
+        tmp_file = "#{content_path.basename}.#{Process.pid}"
+        tmp_path = Pathname.new File.join(Dir.tmpdir, tmp_file)
+        tmp_path.binwrite contents
+
+        yield tmp_path
+
+        updated_contents = tmp_path.binread
+
+        write(updated_contents) if updated_contents != contents
+      ensure
+        FileUtils.rm(tmp_path) if tmp_path.exist?
+      end
+
+
+      def encrypt(contents)
+        encryptor.encrypt_and_sign contents
+      end
+
+      def decrypt(contents)
+        encryptor.decrypt_and_verify contents
+      end
+
+      def encryptor
+        @encryptor ||= ActiveSupport::MessageEncryptor.new([ key ].pack("H*"), cipher: CIPHER)
+      end
+
+
+      def read_env_key
+        ENV[env_key]
+      end
+
+      def read_key_file
+        key_path.binread.strip if key_path.exist?
+      end
+
+      def handle_missing_key
+        raise MissingKeyError, key_path: key_path, env_key: env_key
+      end
+  end
+end
diff --git a/activesupport/lib/active_support/railtie.rb b/activesupport/lib/active_support/railtie.rb
index 2efe391d01..fe132ca7c6 100644
--- a/activesupport/lib/active_support/railtie.rb
+++ b/activesupport/lib/active_support/railtie.rb
@@ -49,6 +49,17 @@ module ActiveSupport
       Date.beginning_of_week_default = beginning_of_week_default
     end
 
+    initializer "active_support.require_master_key" do |app|
+      if app.config.respond_to?(:require_master_key) && app.config.require_master_key
+        begin
+          app.credentials.key
+        rescue ActiveSupport::EncryptedFile::MissingKeyError => error
+          $stderr.puts error.message
+          exit 1
+        end
+      end
+    end
+
     initializer "active_support.set_configs" do |app|
       app.config.active_support.each do |k, v|
         k = "#{k}="