diff options
Diffstat (limited to 'guides')
-rw-r--r-- | guides/bug_report_templates/action_controller_master.rb | 1 | ||||
-rw-r--r-- | guides/bug_report_templates/active_job_master.rb | 1 | ||||
-rw-r--r-- | guides/bug_report_templates/active_record_master.rb | 1 | ||||
-rw-r--r-- | guides/bug_report_templates/active_record_migrations_master.rb | 1 | ||||
-rw-r--r-- | guides/bug_report_templates/benchmark.rb | 1 | ||||
-rw-r--r-- | guides/bug_report_templates/generic_master.rb | 1 | ||||
-rw-r--r-- | guides/source/active_record_callbacks.md | 2 | ||||
-rw-r--r-- | guides/source/api_documentation_guidelines.md | 6 | ||||
-rw-r--r-- | guides/source/caching_with_rails.md | 51 | ||||
-rw-r--r-- | guides/source/command_line.md | 2 | ||||
-rw-r--r-- | guides/source/configuring.md | 4 | ||||
-rw-r--r-- | guides/source/documents.yaml | 5 | ||||
-rw-r--r-- | guides/source/layouts_and_rendering.md | 2 | ||||
-rw-r--r-- | guides/source/security.md | 4 | ||||
-rw-r--r-- | guides/source/testing.md | 10 | ||||
-rw-r--r-- | guides/source/threading_and_code_execution.md | 324 |
16 files changed, 398 insertions, 18 deletions
diff --git a/guides/bug_report_templates/action_controller_master.rb b/guides/bug_report_templates/action_controller_master.rb index 932d329943..732cdad259 100644 --- a/guides/bug_report_templates/action_controller_master.rb +++ b/guides/bug_report_templates/action_controller_master.rb @@ -15,7 +15,6 @@ gemfile(true) do git_source(:github) { |repo| "https://github.com/#{repo}.git" } gem "rails", github: "rails/rails" - gem "arel", github: "rails/arel" end require "action_controller/railtie" diff --git a/guides/bug_report_templates/active_job_master.rb b/guides/bug_report_templates/active_job_master.rb index 36d9137b71..c0c67879f3 100644 --- a/guides/bug_report_templates/active_job_master.rb +++ b/guides/bug_report_templates/active_job_master.rb @@ -15,7 +15,6 @@ gemfile(true) do git_source(:github) { |repo| "https://github.com/#{repo}.git" } gem "rails", github: "rails/rails" - gem "arel", github: "rails/arel" end require "active_job" diff --git a/guides/bug_report_templates/active_record_master.rb b/guides/bug_report_templates/active_record_master.rb index b66deb36f3..b1c83a51f6 100644 --- a/guides/bug_report_templates/active_record_master.rb +++ b/guides/bug_report_templates/active_record_master.rb @@ -15,7 +15,6 @@ gemfile(true) do git_source(:github) { |repo| "https://github.com/#{repo}.git" } gem "rails", github: "rails/rails" - gem "arel", github: "rails/arel" gem "sqlite3" end diff --git a/guides/bug_report_templates/active_record_migrations_master.rb b/guides/bug_report_templates/active_record_migrations_master.rb index 737ce66d7b..0979a42a41 100644 --- a/guides/bug_report_templates/active_record_migrations_master.rb +++ b/guides/bug_report_templates/active_record_migrations_master.rb @@ -15,7 +15,6 @@ gemfile(true) do git_source(:github) { |repo| "https://github.com/#{repo}.git" } gem "rails", github: "rails/rails" - gem "arel", github: "rails/arel" gem "sqlite3" end diff --git a/guides/bug_report_templates/benchmark.rb b/guides/bug_report_templates/benchmark.rb index f5c88086a9..520c5e8bab 100644 --- a/guides/bug_report_templates/benchmark.rb +++ b/guides/bug_report_templates/benchmark.rb @@ -15,7 +15,6 @@ gemfile(true) do git_source(:github) { |repo| "https://github.com/#{repo}.git" } gem "rails", github: "rails/rails" - gem "arel", github: "rails/arel" gem "benchmark-ips" end diff --git a/guides/bug_report_templates/generic_master.rb b/guides/bug_report_templates/generic_master.rb index 240571ba9a..f7c9fedf02 100644 --- a/guides/bug_report_templates/generic_master.rb +++ b/guides/bug_report_templates/generic_master.rb @@ -15,7 +15,6 @@ gemfile(true) do git_source(:github) { |repo| "https://github.com/#{repo}.git" } gem "rails", github: "rails/rails" - gem "arel", github: "rails/arel" end require "active_support" diff --git a/guides/source/active_record_callbacks.md b/guides/source/active_record_callbacks.md index 53417f012e..630dafe632 100644 --- a/guides/source/active_record_callbacks.md +++ b/guides/source/active_record_callbacks.md @@ -213,6 +213,7 @@ The following methods trigger callbacks: * `save!` * `save(validate: false)` * `toggle!` +* `touch` * `update_attribute` * `update` * `update!` @@ -245,7 +246,6 @@ Just as with validations, it is also possible to skip callbacks by using the fol * `increment` * `increment_counter` * `toggle` -* `touch` * `update_column` * `update_columns` * `update_all` diff --git a/guides/source/api_documentation_guidelines.md b/guides/source/api_documentation_guidelines.md index 2c153d3783..10b89433e7 100644 --- a/guides/source/api_documentation_guidelines.md +++ b/guides/source/api_documentation_guidelines.md @@ -16,7 +16,7 @@ RDoc ---- The [Rails API documentation](http://api.rubyonrails.org) is generated with -[RDoc](http://docs.seattlerb.org/rdoc/). To generate it, make sure you are +[RDoc](https://ruby.github.io/rdoc/). To generate it, make sure you are in the rails root directory, run `bundle install` and execute: ```bash @@ -26,9 +26,9 @@ in the rails root directory, run `bundle install` and execute: Resulting HTML files can be found in the ./doc/rdoc directory. Please consult the RDoc documentation for help with the -[markup](http://docs.seattlerb.org/rdoc/RDoc/Markup.html), +[markup](https://ruby.github.io/rdoc/RDoc/Markup.html), and also take into account these [additional -directives](http://docs.seattlerb.org/rdoc/RDoc/Parser/Ruby.html). +directives](https://ruby.github.io/rdoc/RDoc/Parser/Ruby.html). Wording ------- diff --git a/guides/source/caching_with_rails.md b/guides/source/caching_with_rails.md index 96650b5be9..31bc478015 100644 --- a/guides/source/caching_with_rails.md +++ b/guides/source/caching_with_rails.md @@ -366,9 +366,9 @@ There are some common options used by all cache implementations. These can be pa * `:namespace` - This option can be used to create a namespace within the cache store. It is especially useful if your application shares a cache with other applications. -* `:compress` - This option can be used to indicate that compression should be used in the cache. This can be useful for transferring large cache entries over a slow network. +* `:compress` - Enabled by default. Compresses cache entries so more data can be stored in the same memory footprint, leading to fewer cache evictions and higher hit rates. -* `:compress_threshold` - This option is used in conjunction with the `:compress` option to indicate a threshold under which cache entries should not be compressed. This defaults to 16 kilobytes. +* `:compress_threshold` - Defaults to 1kB. Cache entries larger than this threshold, specified in bytes, are compressed. * `:expires_in` - This option sets an expiration time in seconds for the cache entry when it will be automatically removed from the cache. @@ -444,6 +444,53 @@ The `write` and `fetch` methods on this cache accept two additional options that config.cache_store = :mem_cache_store, "cache-1.example.com", "cache-2.example.com" ``` +### ActiveSupport::Cache::RedisCacheStore + +The Redis cache store takes advantage of Redis support for least-recently-used +and least-frequently-used key eviction when it reaches max memory, allowing it +to behave much like a Memcached cache server. + +Deployment note: Redis doesn't expire keys by default, so take care to use a +dedicated Redis cache server. Don't fill up your persistent-Redis server with +volatile cache data! Read the +[Redis cache server setup guide](https://redis.io/topics/lru-cache) in detail. + +For an all-cache Redis server, set `maxmemory-policy` to an `allkeys` policy. +Redis 4+ support least-frequently-used (`allkeys-lfu`) eviction, an excellent +default choice. Redis 3 and earlier should use `allkeys-lru` for +least-recently-used eviction. + +Set cache read and write timeouts relatively low. Regenerating a cached value +is often faster than waiting more than a second to retrieve it. Both read and +write timeouts default to 1 second, but may be set lower if your network is +consistently low latency. + +Cache reads and writes never raise exceptions. They just return `nil` instead, +behaving as if there was nothing in the cache. To gauge whether your cache is +hitting exceptions, you may provide an `error_handler` to report to an +exception gathering service. It must accept three keyword arguments: `method`, +the cache store method that was originally called; `returning`, the value that +was returned to the user, typically `nil`; and `exception`, the exception that +was rescued. + +Putting it all together, a production Redis cache store may look something +like this: + +```ruby +cache_servers = %w[ "redis://cache-01:6379/0", "redis://cache-02:6379/0", … ], +config.cache_store = :redis_cache_store, url: cache_servers, + + connect_timeout: 30, # Defaults to 20 seconds + read_timeout: 0.2, # Defaults to 1 second + write_timeout: 0.2, # Defaults to 1 second + + error_handler: -> (method:, returning:, exception:) { + # Report errors to Sentry as warnings + Raven.capture_exception exception, level: 'warning", + tags: { method: method, returning: returning } + } +``` + ### ActiveSupport::Cache::NullStore This cache store implementation is meant to be used only in development or test environments and it never stores anything. This can be very useful in development when you have code that interacts directly with `Rails.cache` but caching may interfere with being able to see the results of code changes. With this cache store, all `fetch` and `read` operations will result in a miss. diff --git a/guides/source/command_line.md b/guides/source/command_line.md index 88c559921c..648645af7c 100644 --- a/guides/source/command_line.md +++ b/guides/source/command_line.md @@ -290,7 +290,7 @@ INFO: You can also use the alias "c" to invoke the console: `rails c`. You can specify the environment in which the `console` command should operate. ```bash -$ bin/rails console staging +$ bin/rails console -e staging ``` If you wish to test out some code without changing any data, you can do that by invoking `rails console --sandbox`. diff --git a/guides/source/configuring.md b/guides/source/configuring.md index 7a32607eb7..6e129a5680 100644 --- a/guides/source/configuring.md +++ b/guides/source/configuring.md @@ -973,7 +973,7 @@ By default Rails ships with three environments: "development", "test", and "prod Imagine you have a server which mirrors the production environment but is only used for testing. Such a server is commonly called a "staging server". To define an environment called "staging" for this server, just create a file called `config/environments/staging.rb`. Please use the contents of any existing file in `config/environments` as a starting point and make the necessary changes from there. -That environment is no different than the default ones, start a server with `rails server -e staging`, a console with `rails console staging`, `Rails.env.staging?` works, etc. +That environment is no different than the default ones, start a server with `rails server -e staging`, a console with `rails console -e staging`, `Rails.env.staging?` works, etc. ### Deploy to a subdirectory (relative url root) @@ -1057,7 +1057,7 @@ After loading the framework and any gems in your application, Rails turns to loa NOTE: You can use subfolders to organize your initializers if you like, because Rails will look into the whole file hierarchy from the initializers folder on down. -TIP: If you have any ordering dependency in your initializers, you can control the load order through naming. Initializer files are loaded in alphabetical order by their path. For example, `01_critical.rb` will be loaded before `02_normal.rb`. +TIP: While Rails supports numbering of initializer file names for load ordering purposes, a better technique is to place any code that need to load in a specific order within the same file. This reduces file name churn, makes dependencies more explicit, and can help surface new concepts within your application. Initialization events --------------------- diff --git a/guides/source/documents.yaml b/guides/source/documents.yaml index 59205ee465..126d2e4845 100644 --- a/guides/source/documents.yaml +++ b/guides/source/documents.yaml @@ -159,6 +159,11 @@ url: engines.html description: This guide explains how to write a mountable engine. work_in_progress: true + - + name: Threading and Code Execution in Rails + url: threading_and_code_execution.html + description: This guide describes the considerations needed and tools available when working directly with concurrency in a Rails application. + work_in_progress: true - name: Contributing to Ruby on Rails documents: diff --git a/guides/source/layouts_and_rendering.md b/guides/source/layouts_and_rendering.md index b9b327252f..f4597b0e60 100644 --- a/guides/source/layouts_and_rendering.md +++ b/guides/source/layouts_and_rendering.md @@ -1266,7 +1266,7 @@ You can also pass in arbitrary local variables to any partial you are rendering In this case, the partial will have access to a local variable `title` with the value "Products Page". -TIP: Rails also makes a counter variable available within a partial called by the collection, named after the member of the collection followed by `_counter`. For example, if you're rendering `@products`, within the partial you can refer to `product_counter` to tell you how many times the partial has been rendered. This does not work in conjunction with the `as: :value` option. +TIP: Rails also makes a counter variable available within a partial called by the collection, named after the title of the partial followed by `_counter`. For example, when rendering a collection `@products` the partial `_product.html.erb` can access the variable `product_counter` which indexes the number of times it has been rendered within the enclosing view. You can also specify a second partial to be rendered between instances of the main partial by using the `:spacer_template` option: diff --git a/guides/source/security.md b/guides/source/security.md index cfa777d433..fa90cadcd2 100644 --- a/guides/source/security.md +++ b/guides/source/security.md @@ -192,9 +192,9 @@ rotations going at any one time. For more details on key rotation with encrypted and signed messages as well as the various options the `rotate` method accepts, please refer to the -[MessageEncryptor API](api.rubyonrails.org/classes/ActiveSupport/MessageEncryptor.html) +[MessageEncryptor API](http://api.rubyonrails.org/classes/ActiveSupport/MessageEncryptor.html) and -[MessageVerifier API](api.rubyonrails.org/classes/ActiveSupport/MessageVerifier.html) +[MessageVerifier API](http://api.rubyonrails.org/classes/ActiveSupport/MessageVerifier.html) documentation. ### Replay Attacks for CookieStore Sessions diff --git a/guides/source/testing.md b/guides/source/testing.md index c5b2a694e7..8416fd163d 100644 --- a/guides/source/testing.md +++ b/guides/source/testing.md @@ -671,6 +671,16 @@ class ApplicationSystemTestCase < ActionDispatch::SystemTestCase end ``` +If you want to use a headless browser, you could use Headless Chrome by adding `headless_chrome` in the `:using` argument. + +```ruby +require "test_helper" + +class ApplicationSystemTestCase < ActionDispatch::SystemTestCase + driven_by :selenium, using: :headless_chrome +end +``` + If your Capybara configuration requires more setup than provided by Rails, this additional configuration could be added into the `application_system_test_case.rb` file. diff --git a/guides/source/threading_and_code_execution.md b/guides/source/threading_and_code_execution.md new file mode 100644 index 0000000000..3d3d31b97e --- /dev/null +++ b/guides/source/threading_and_code_execution.md @@ -0,0 +1,324 @@ +**DO NOT READ THIS FILE ON GITHUB, GUIDES ARE PUBLISHED ON http://guides.rubyonrails.org.** + +Threading and Code Execution in Rails +===================================== + +After reading this guide, you will know: + +* What code Rails will automatically execute concurrently +* How to integrate manual concurrency with Rails internals +* How to wrap all application code +* How to affect application reloading + +-------------------------------------------------------------------------------- + +Automatic Concurrency +--------------------- + +Rails automatically allows various operations to be performed at the same time. + +When using a threaded web server, such as the default Puma, multiple HTTP +requests will be served simultaneously, with each request provided its own +controller instance. + +Threaded Active Job adapters, including the built-in Async, will likewise +execute several jobs at the same time. Action Cable channels are managed this +way too. + +These mechanisms all involve multiple threads, each managing work for a unique +instance of some object (controller, job, channel), while sharing the global +process space (such as classes and their configurations, and global variables). +As long as your code doesn't modify any of those shared things, it can mostly +ignore that other threads exist. + +The rest of this guide describes the mechanisms Rails uses to make it "mostly +ignorable", and how extensions and applications with special needs can use them. + +Executor +-------- + +The Rails Executor separates application code from framework code: any time the +framework invokes code you've written in your application, it will be wrapped by +the Executor. + +The Executor consists of two callbacks: `to_run` and `to_complete`. The Run +callback is called before the application code, and the Complete callback is +called after. + +### Default callbacks + +In a default Rails application, the Executor callbacks are used to: + +* track which threads are in safe positions for autoloading and reloading +* enable and disable the Active Record query cache +* return acquired Active Record connections to the pool +* constrain internal cache lifetimes + +Prior to Rails 5.0, some of these were handled by separate Rack middleware +classes (such as `ActiveRecord::ConnectionAdapters::ConnectionManagement`), or +directly wrapping code with methods like +`ActiveRecord::Base.connection_pool.with_connection`. The Executor replaces +these with a single more abstract interface. + +### Wrapping application code + +If you're writing a library or component that will invoke application code, you +should wrap it with a call to the executor: + +```ruby +Rails.application.executor.wrap do + # call application code here +end +``` + +TIP: If you repeatedly invoke application code from a long-running process, you +may want to wrap using the Reloader instead. + +Each thread should be wrapped before it runs application code, so if your +application manually delegates work to other threads, such as via `Thread.new` +or Concurrent Ruby features that use thread pools, you should immediately wrap +the block: + +```ruby +Thread.new do + Rails.application.executor.wrap do + # your code here + end +end +``` + +NOTE: Concurrent Ruby uses a `ThreadPoolExecutor`, which it sometimes configures +with an `executor` option. Despite the name, it is unrelated. + +The Executor is safely re-entrant; if it is already active on the current +thread, `wrap` is a no-op. + +If it's impractical to wrap the application code in a block (for +example, the Rack API makes this problematic), you can also use the `run!` / +`complete!` pair: + +```ruby +Thread.new do + execution_context = Rails.application.executor.run! + # your code here +ensure + execution_context.complete! if execution_context +end +``` + +### Concurrency + +The Executor will put the current thread into `running` mode in the Load +Interlock. This operation will block temporarily if another thread is currently +either autoloading a constant or unloading/reloading the application. + +Reloader +-------- + +Like the Executor, the Reloader also wraps application code. If the Executor is +not already active on the current thread, the Reloader will invoke it for you, +so you only need to call one. This also guarantees that everything the Reloader +does, including all its callback invocations, occurs wrapped inside the +Executor. + +```ruby +Rails.application.reloader.wrap do + # call application code here +end +``` + +The Reloader is only suitable where a long-running framework-level process +repeatedly calls into application code, such as for a web server or job queue. +Rails automatically wraps web requests and Active Job workers, so you'll rarely +need to invoke the Reloader for yourself. Always consider whether the Executor +is a better fit for your use case. + +### Callbacks + +Before entering the wrapped block, the Reloader will check whether the running +application needs to be reloaded -- for example, because a model's source file has +been modified. If it determines a reload is required, it will wait until it's +safe, and then do so, before continuing. When the application is configured to +always reload regardless of whether any changes are detected, the reload is +instead performed at the end of the block. + +The Reloader also provides `to_run` and `to_complete` callbacks; they are +invoked at the same points as those of the Executor, but only when the current +execution has initiated an application reload. When no reload is deemed +necessary, the Reloader will invoke the wrapped block with no other callbacks. + +### Class Unload + +The most significant part of the reloading process is the Class Unload, where +all autoloaded classes are removed, ready to be loaded again. This will occur +immediately before either the Run or Complete callback, depending on the +`reload_classes_only_on_change` setting. + +Often, additional reloading actions need to be performed either just before or +just after the Class Unload, so the Reloader also provides `before_class_unload` +and `after_class_unload` callbacks. + +### Concurrency + +Only long-running "top level" processes should invoke the Reloader, because if +it determines a reload is needed, it will block until all other threads have +completed any Executor invocations. + +If this were to occur in a "child" thread, with a waiting parent inside the +Executor, it would cause an unavoidable deadlock: the reload must occur before +the child thread is executed, but it cannot be safely performed while the parent +thread is mid-execution. Child threads should use the Executor instead. + +Framework Behavior +------------------ + +The Rails framework components use these tools to manage their own concurrency +needs too. + +`ActionDispatch::Executor` and `ActionDispatch::Reloader` are Rack middlewares +that wraps the request with a supplied Executor or Reloader, respectively. They +are automatically included in the default application stack. The Reloader will +ensure any arriving HTTP request is served with a freshly-loaded copy of the +application if any code changes have occurred. + +Active Job also wraps its job executions with the Reloader, loading the latest +code to execute each job as it comes off the queue. + +Action Cable uses the Executor instead: because a Cable connection is linked to +a specific instance of a class, it's not possible to reload for every arriving +websocket message. Only the message handler is wrapped, though; a long-running +Cable connection does not prevent a reload that's triggered by a new incoming +request or job. Instead, Action Cable uses the Reloader's `before_class_unload` +callback to disconnect all its connections. When the client automatically +reconnects, it will be speaking to the new version of the code. + +The above are the entry points to the framework, so they are responsible for +ensuring their respective threads are protected, and deciding whether a reload +is necessary. Other components only need to use the Executor when they spawn +additional threads. + +### Configuration + +The Reloader only checks for file changes when `cache_classes` is false and +`reload_classes_only_on_change` is true (which is the default in the +`development` environment). + +When `cache_classes` is true (in `production`, by default), the Reloader is only +a pass-through to the Executor. + +The Executor always has important work to do, like database connection +management. When `cache_classes` and `eager_load` are both true (`production`), +no autoloading or class reloading will occur, so it does not need the Load +Interlock. If either of those are false (`development`), then the Executor will +use the Load Interlock to ensure constants are only loaded when it is safe. + +Load Interlock +-------------- + +The Load Interlock allows autoloading and reloading to be enabled in a +multi-threaded runtime environment. + +When one thread is performing an autoload by evaluating the class definition +from the appropriate file, it is important no other thread encounters a +reference to the partially-defined constant. + +Similarly, it is only safe to perform an unload/reload when no application code +is in mid-execution: after the reload, the `User` constant, for example, may +point to a different class. Without this rule, a poorly-timed reload would mean +`User.new.class == User`, or even `User == User`, could be false. + +Both of these constraints are addressed by the Load Interlock. It keeps track of +which threads are currently running application code, loading a class, or +unloading autoloaded constants. + +Only one thread may load or unload at a time, and to do either, it must wait +until no other threads are running application code. If a thread is waiting to +perform a load, it doesn't prevent other threads from loading (in fact, they'll +cooperate, and each perform their queued load in turn, before all resuming +running together). + +### `permit_concurrent_loads` + +The Executor automatically acquires a `running` lock for the duration of its +block, and autoload knows when to upgrade to a `load` lock, and switch back to +`running` again afterwards. + +Other blocking operations performed inside the Executor block (which includes +all application code), however, can needlessly retain the `running` lock. If +another thread encounters a constant it must autoload, this can cause a +deadlock. + +For example, assuming `User` is not yet loaded, the following will deadlock: + +```ruby +Rails.application.executor.wrap do + th = Thread.new do + Rails.application.executor.wrap do + User # inner thread waits here; it cannot load + # User while another thread is running + end + end + + th.join # outer thread waits here, holding 'running' lock +end +``` + +To prevent this deadlock, the outer thread can `permit_concurrent_loads`. By +calling this method, the thread guarantees it will not dereference any +possibly-autoloaded constant inside the supplied block. The safest way to meet +that promise is to put it as close as possible to the blocking call: + +```ruby +Rails.application.executor.wrap do + th = Thread.new do + Rails.application.executor.wrap do + User # inner thread can acquire the load lock, + # load User, and continue + end + end + + ActiveSupport::Dependencies.interlock.permit_concurrent_loads do + th.join # outer thread waits here, but has no lock + end +end +``` + +Another example, using Concurrent Ruby: + +```ruby +Rails.application.executor.wrap do + futures = 3.times.collect do |i| + Concurrent::Future.execute do + Rails.application.executor.wrap do + # do work here + end + end + end + + values = ActiveSupport::Dependencies.interlock.permit_concurrent_loads do + futures.collect(&:value) + end +end +``` + + +### ActionDispatch::DebugLocks + +If your application is deadlocking and you think the Load Interlock may be +involved, you can temporarily add the ActionDispatch::DebugLocks middleware to +`config/application.rb`: + +```ruby +config.middleware.insert_before Rack::Sendfile, + ActionDispatch::DebugLocks +``` + +If you then restart the application and re-trigger the deadlock condition, +`/rails/locks` will show a summary of all threads currently known to the +interlock, which lock level they are holding or awaiting, and their current +backtrace. + +Generally a deadlock will be caused by the interlock conflicting with some other +external lock or blocking I/O call. Once you find it, you can wrap it with +`permit_concurrent_loads`. + |