Default to sorting user's test cases for now

Goals:

1. Default to :random for newly generated applications
2. Default to :sorted for existing applications with a warning
3. Only show the warning once
4. Only show the warning if the app actually uses AS::TestCase

Fixes #16769

1 files changed, 18 insertions, 0 deletions

diff --git a/guides/source/upgrading_ruby_on_rails.md b/guides/source/upgrading_ruby_on_rails.md
index 989d8400e5..71358ad3f5 100644
--- a/guides/source/upgrading_ruby_on_rails.md
+++ b/guides/source/upgrading_ruby_on_rails.md
@@ -65,6 +65,24 @@ TODO: mention https://github.com/rails/rails/pull/16526
 
 TODO: mention https://github.com/rails/rails/pull/16537
 
+### Ordering of test cases
+
+In Rails 5.0, test cases will be executed in random order by default. In
+anticipation of this change, Rails 4.2 introduced a new configuration option
+`active_support.test_order` for explicitly specifying the test ordering. This
+allows you to either locking down the current behavior by setting the option to
+`:sorted`, or opt into the future behavior by setting the option to `:random`.
+
+If you do not specify a value for this option, a deprecation warning will be
+emitted. To avoid this, add the following line to your test environment:
+
+```ruby
+# config/environments/test.rb
+Rails.application.configure do
+  config.active_support.test_order = :sorted # or `:random` if you prefer
+end
+```
+
 ### Serialized attributes
 
 When using a custom coder (e.g. `serialize :metadata, JSON`),