| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
| |
Previously if an app attempts to do a write inside a read request it will be
impossilbe to switch back to writing to the primary. This PR adds an
argument to the `while_preventing_writes` so that we can make sure to
turn it off if we're doing a write on a primary.
Fixes #36830
Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
|
|
|
|
|
| |
Enables the query cache on the correct connection when
shared connections across threads are enabled
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If we put the `while_preventing_writes` on the connection then the
middleware that sends reads to the primary and ensures they can't write
will not work. The `while_preventing_writes` will only be applied to the
connection which it's called on - which in the case of the middleware is
Ar::Base.
This worked fine if you called it directly like
`OtherDbConn.connection.while_preventing_writes` but Rails didn't have a
way of knowing you wanted to call it on all the connections.
The change here moves the `while_preventing_writes` method from the
connection to the handler so that it can block writes to all queries for
that handler. This will apply to all the connections associated with
that handler.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Make ActiveRecord `ConnectionPool.connections` thread-safe.
ConnectionPool documentation is clear on the need to synchronize
access to @connections but also states that public methods do not
require synchronization. Existing code exposed @connections
directly via attr_reader. The fix uses synchronize() to lock
@connections then returns a copy to the caller using Array.dup().
Includes comments on the connections method that thread-safe access
to the connections array does not imply thread-safety of accessing
methods on the actual connections.
Adds a test-case that modifies the pool using a supported method
in one thread while a second thread accesses pool.connections.
The test fails without this patch.
Fixes #36465.
* Update activerecord/test/cases/connection_pool_test.rb
[jeffdoering + Rafael Mendonça França]
|
|
|
|
|
|
|
|
|
|
|
| |
We sometimes say "✂️ newline after `private`" in a code review (e.g.
https://github.com/rails/rails/pull/18546#discussion_r23188776,
https://github.com/rails/rails/pull/34832#discussion_r244847195).
Now `Layout/EmptyLinesAroundAccessModifier` cop have new enforced style
`EnforcedStyle: only_before` (https://github.com/rubocop-hq/rubocop/pull/7059).
That cop and enforced style will reduce the our code review cost.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR proposes moving the schema cache from the connection to the pool
so the connection can ask the pool for the cache. In a future PR our
goal is to be able to read the yaml file from the pool so we can get
rid of the `active_record.check_schema_cache_dump` initializer. This
will fix the issues surrounding dumping the schema cache and mulitple
databases.
Why do we want to get rid of the initializer you ask?
Well I was looking at #34449 and trying to make it work for our usecase
and it revealed A LOT of problems. There are a few issues that I will
fix in remaining PRs with SchemaMigration, but there's a big glaring
issue with this initializer.
When you have an application with multiple databases we'll need to loop
through all the configurations and set the schema cache on those
connections. The problem is on initialization we only have one
connection - the one for Ar::Base. This is fine in a single db
application but not fine in multi-db. If we follow the pattern in #34449
and establish a connection to those other dbs we will end up setting the
cache on the _connection object_ rather than on all connections that
connect for that config.
So even though we looped through the configs and assigned the cache the
cache will not be set (or will be set wrong) once the app is booted
because the connection objects after boot are _different_ than the
connection objects we assigned the cache to.
After trying many different ways to set the schema cache `@tenderlove`
and I came to the conclusion that the initializer is problematic, as is
setting the schema cache twice.
This is part 1 to move the cache to the pool so the cache can read from
the schema cache yaml file instead of setting it when initializing the
app.
To do this we have created a `NullPool` that initializes an empty cache. I
put the `get_schema_cache` and `set_schema_cache` in an `AbstractPool`
so we can share code between `ConnectionPool` and `NullPool` instead of
duplicating code.
Now we only need to set the schema_cache on the pool rather than the
connection. In `discard!` we need to unset the connection from the
schema_cache - we still want the cache just not the connection.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Prior to 3e2e8eeb9ea552bd4782538cf9348455f3d0e14a the Reaper thread
would hold a reference to connection pools indefinitely, preventing the
connection pool from being garbage collected, and also leaking the
Thread.
Since 3e2e8eeb9ea552bd4782538cf9348455f3d0e14a, there is only one Reaper
Thread for all pools, however all pools are still stored in a class
variable, preventing them from being garbage collected.
This commit instead holds reference to the pools through a WeakRef. This
way, connection pools referenced elsewhere will be reaped, any others
will be able to be garbage collected.
I don't love resorting to WeakRef to solve this, but I believe it's the
simplest way to accomplish the the desired behaviour.
|
|
|
|
|
| |
Previously we would spawn one thread per connection pool, which ends up
being wasteful for apps with several connection pools.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* The database version will get cached in the schema cache file during the
schema cache dump. When the database version check happens, the version will
be pulled from the schema cache and thus avoid querying the database for
the version.
* If the schema cache file doesn't exist, we'll query the database for the
version and cache it on the schema cache object.
* To facilitate this change, all connection adapters now implement
#get_database_version and #database_version. #database_version returns the
value from the schema cache.
* To take advantage of the cached database version, the database version check
will now happen after the schema cache is set on the connection in the
connection pool.
|
|\
| |
| |
| | |
Fix possible memory leak of ConnectionHandler
|
|/
|
|
| |
refs #35296
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I've found a few places in Rails code base where I think it makes sense
to calculate elapsed time more precisely by using
`Concurrent.monotonic_time`:
- Fix calculation of elapsed time in `ActiveSupport::Cache::MemoryStore#prune`
- Fix calculation of elapsed time in
`ActiveRecord::ConnectionAdapters::ConnectionPool::Queue#wait_poll`
- Fix calculation of elapsed time in
`ActiveRecord::ConnectionAdapters::ConnectionPool#attempt_to_checkout_all_existing_connections`
- Fix calculation of elapsed time in `ActiveRecord::ConnectionAdapters::Mysql2Adapter#explain`
See
https://docs.ruby-lang.org/en/2.5.0/Process.html#method-c-clock_gettime
https://blog.dnsimple.com/2018/03/elapsed-time-with-ruby-the-right-way
Related to 7c4542146f0dde962205e5a90839349631ae60fb
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While working on another feature for multiple databases (auto-switching)
I observed that in development the first request won't autoload the
application record connection for the primary database and may not yet
know about the replica connection.
In my test application this caused the application to thrown an error if
I tried to send the first request to the replica before the replica was
connected. This wouldn't be an issue in production because the
application is preloaded.
In order to fix this I decided to leave the original error message and
delete the new error message. I updated the original error message to
include the `role` to make it a bit clearer that the connection isn't
established for that particular role.
The error now reads:
```
No connection pool with 'primary' found for the 'reading' role.
```
A single database application will continue uisng the original error
message:
```
No connection pool with 'primary' found.
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In Ruby 2.3 or later, `String#+@` is available and `+@` is faster than `dup`.
```ruby
# frozen_string_literal: true
require "bundler/inline"
gemfile(true) do
source "https://rubygems.org"
gem "benchmark-ips"
end
Benchmark.ips do |x|
x.report('+@') { +"" }
x.report('dup') { "".dup }
x.compare!
end
```
```
$ ruby -v benchmark.rb
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
Warming up --------------------------------------
+@ 282.289k i/100ms
dup 187.638k i/100ms
Calculating -------------------------------------
+@ 6.775M (± 3.6%) i/s - 33.875M in 5.006253s
dup 3.320M (± 2.2%) i/s - 16.700M in 5.032125s
Comparison:
+@: 6775299.3 i/s
dup: 3320400.7 i/s - 2.04x slower
```
|
|\
| |
| | |
Prevent deadlocks when waiting for connection from pool.
|
| |
| |
| |
| |
| | |
When a thread that had the load interlock but was blocked waiting to check a connection out of the connection pool but all of the threads using the available connections were blocked waiting to obtain the load interlock an `ActiveRecord::ConnectionTimeoutError` exception was be thrown by the thread waiting for the connection.
When waiting for the connection to check out we should allow loading to proceed to avoid this deadlock.
|
|/ |
|
|\
| |
| |
| | |
Remove dormant check
|
| |
| |
| |
| |
| | |
This check was introduced in 6edaa26 and moved through multiple refactorings.
No test are broken after removal and AFAICS there is no way to trigger it.
|
|\ \
| | |
| | | |
Flush idle database connections
|
| | | |
|
|\ \ \
| |/ /
|/| | |
Improve AR connection fork safety
|
| | |
| | |
| | |
| | |
| | |
| | | |
Use whatever adapter-provided means we have available to ensure forked
children don't send quit/shutdown/goodbye messages to the server on
connections that belonged to their parent.
|
|\ \ \
| |/ /
|/| | |
Switch to LIFO for the connection pool
|
| | | |
|
| | | |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Using a FIFO for the connection pool can lead to issues when there are
upstream components (pgbouncer, haproxy, etc.) that terminate
connections that are idle after a period of time. Switching to a LIFO
reduces the probability that a thread will checkout a connection that is
about to be closed by an idle timeout in an upstream component.
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
refs: https://github.com/rails/rails/pull/30161
```
$ echo "+@size+" | rdoc --pipe
<p>+@size+</p>
$ echo "<tt>@size</tt>" | rdoc --pipe
<p><code>@size</code></p>
```
[ci skip]
|
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Otherwise `ConnectionPool#reap` may run before `@connections` has
initialized.
https://travis-ci.org/rails/rails/jobs/263037427#L888-L890
|
| | | |
|
| | |
| | |
| | |
| | |
| | | |
This reverts commit 3420a14590c0e6915d8b6c242887f74adb4120f9, reversing
changes made to afb66a5a598ce4ac74ad84b125a5abf046dcf5aa.
|
|\ \ \
| | | |
| | | |
| | | | |
Enforce frozen string in Rubocop
|
| | | | |
|
|/ / / |
|
| | | |
|
|/ /
| |
| |
| |
| |
| |
| |
| |
| | |
'lookup' is the noun. 'to look up' is the verb. Looked it up just to
be sure.
cf.
https://en.wiktionary.org/wiki/lookup
https://en.wiktionary.org/wiki/look_up
|
| |
| |
| | |
`ActiveRecord::Base.establish_connection` accepts a single symbol argument to specify a named connection; a single string argument appears to be interpreted as a connection URI
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This ensures multiple threads inside a transactional test to see consistent
database state.
When a system test starts Puma spins up one thread and Capybara spins up
another thread. Because of this when tests are run the database cannot
see what was inserted into the database on teardown. This is because
there are two threads using two different connections.
This change uses the statement cache to lock the threads to using a
single connection ID instead of each not being able to see each other.
This code only runs in the fixture setup and teardown so it does not
affect real production databases.
When a transaction is opened we set `lock_thread` to `Thread.current` so
we can keep track of which connection the thread is using. When we
rollback the transaction we unlock the thread and then there will be no
left-over data in the database because the transaction will roll back
the correct connections.
[ Eileen M. Uchitelle, Matthew Draper ]
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If a process is forked more than once, the pool was grabbing the oldest
spec, not the most recent spec. This wasn't noticed before because most
folks are lilely forking the process only once.
If you're forking the process multiple times however the wrong spec name
will be returned and an incorrect connection will be used for the
process.
This fixes the issue by reversing the list of spec names so we can grab
the most recent spec rather than the oldest spec.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
It'll be re-cleared when it's rebuilt in with_new_connections_blocked's
ensure, but we still need to clear it inside this synchronize -- we've
disconnected connections that may be available in the queue, and while
other threads are not allowed to make *new* connections, they are still
allowed to take existing ones from there.
This was incorrectly removed in d314646c965b045724e6bdb9d61dcecfabc0ba8f.
|
| |
| |
| |
| |
| |
| | |
Two methods block new connections; we were already doing the right thing
for clear_reloadable_connections, but it's better placed in
with_new_connections_blocked, where it can work for disconnect too.
|
|\ \
| | |
| | |
| | | |
Fix the race condition caused by `with_new_connections_blocked`
|
|/ /
| |
| |
| |
| |
| | |
`with_new_connections_blocked` was introduced at #14938.
But the method sometimes causes `@new_cons_enabled = false` then never
toggled to true.
|
|\ \
| | |
| | | |
Add missing `+` around a some literals.
|
| | |
| | |
| | |
| | |
| | |
| | | |
Mainly around `nil`
[ci skip]
|
|\ \ \
| | | |
| | | | |
Configure query caching (per thread) on the connection pool
|
| | | | |
|
|/ / / |
|
|/ / |
|
| |
| |
| |
| |
| |
| | |
All indentation was normalized by rubocop auto-correct at 80e66cc4d90bf8c15d1a5f6e3152e90147f00772.
But comments was still kept absolute position. This commit aligns
comments with method definitions for consistency.
|