aboutsummaryrefslogtreecommitdiffstats
path: root/activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb
Commit message (Collapse)AuthorAgeFilesLines
* Add ability to unset preventing writeseileencodes2019-08-021-2/+2
| | | | | | | | | | | Previously if an app attempts to do a write inside a read request it will be impossilbe to switch back to writing to the primary. This PR adds an argument to the `while_preventing_writes` so that we can make sure to turn it off if we're doing a write on a primary. Fixes #36830 Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
* Fix query cache when using shared connectionsHeinrich Lee Yu2019-07-081-2/+6
| | | | | Enables the query cache on the correct connection when shared connections across threads are enabled
* Move while_preventing_writes from conn to handlereileencodes2019-06-141-0/+15
| | | | | | | | | | | | | | | | | If we put the `while_preventing_writes` on the connection then the middleware that sends reads to the primary and ensures they can't write will not work. The `while_preventing_writes` will only be applied to the connection which it's called on - which in the case of the middleware is Ar::Base. This worked fine if you called it directly like `OtherDbConn.connection.while_preventing_writes` but Rails didn't have a way of knowing you wanted to call it on all the connections. The change here moves the `while_preventing_writes` method from the connection to the handler so that it can block writes to all queries for that handler. This will apply to all the connections associated with that handler.
* Make ActiveRecord `ConnectionPool.connections` thread-safe. (#36473)jeffdoering2019-06-131-1/+16
| | | | | | | | | | | | | | | | | | | | | | | * Make ActiveRecord `ConnectionPool.connections` thread-safe. ConnectionPool documentation is clear on the need to synchronize access to @connections but also states that public methods do not require synchronization. Existing code exposed @connections directly via attr_reader. The fix uses synchronize() to lock @connections then returns a copy to the caller using Array.dup(). Includes comments on the connections method that thread-safe access to the connections array does not imply thread-safety of accessing methods on the actual connections. Adds a test-case that modifies the pool using a supported method in one thread while a second thread accesses pool.connections. The test fails without this patch. Fixes #36465. * Update activerecord/test/cases/connection_pool_test.rb [jeffdoering + Rafael Mendonça França]
* Enable `Layout/EmptyLinesAroundAccessModifier` copRyuta Kamizono2019-06-131-3/+0
| | | | | | | | | | | We sometimes say "✂️ newline after `private`" in a code review (e.g. https://github.com/rails/rails/pull/18546#discussion_r23188776, https://github.com/rails/rails/pull/34832#discussion_r244847195). Now `Layout/EmptyLinesAroundAccessModifier` cop have new enforced style `EnforcedStyle: only_before` (https://github.com/rubocop-hq/rubocop/pull/7059). That cop and enforced style will reduce the our code review cost.
* Move schema cache from connection to pooleileencodes2019-06-051-1/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR proposes moving the schema cache from the connection to the pool so the connection can ask the pool for the cache. In a future PR our goal is to be able to read the yaml file from the pool so we can get rid of the `active_record.check_schema_cache_dump` initializer. This will fix the issues surrounding dumping the schema cache and mulitple databases. Why do we want to get rid of the initializer you ask? Well I was looking at #34449 and trying to make it work for our usecase and it revealed A LOT of problems. There are a few issues that I will fix in remaining PRs with SchemaMigration, but there's a big glaring issue with this initializer. When you have an application with multiple databases we'll need to loop through all the configurations and set the schema cache on those connections. The problem is on initialization we only have one connection - the one for Ar::Base. This is fine in a single db application but not fine in multi-db. If we follow the pattern in #34449 and establish a connection to those other dbs we will end up setting the cache on the _connection object_ rather than on all connections that connect for that config. So even though we looped through the configs and assigned the cache the cache will not be set (or will be set wrong) once the app is booted because the connection objects after boot are _different_ than the connection objects we assigned the cache to. After trying many different ways to set the schema cache `@tenderlove` and I came to the conclusion that the initializer is problematic, as is setting the schema cache twice. This is part 1 to move the cache to the pool so the cache can read from the schema cache yaml file instead of setting it when initializing the app. To do this we have created a `NullPool` that initializes an empty cache. I put the `get_schema_cache` and `set_schema_cache` in an `AbstractPool` so we can share code between `ConnectionPool` and `NullPool` instead of duplicating code. Now we only need to set the schema_cache on the pool rather than the connection. In `discard!` we need to unset the connection from the schema_cache - we still want the cache just not the connection.
* Use WeakRef to avoid leaking connection poolsJohn Hawthorn2019-05-271-11/+21
| | | | | | | | | | | | | | | | | | Prior to 3e2e8eeb9ea552bd4782538cf9348455f3d0e14a the Reaper thread would hold a reference to connection pools indefinitely, preventing the connection pool from being garbage collected, and also leaking the Thread. Since 3e2e8eeb9ea552bd4782538cf9348455f3d0e14a, there is only one Reaper Thread for all pools, however all pools are still stored in a class variable, preventing them from being garbage collected. This commit instead holds reference to the pools through a WeakRef. This way, connection pools referenced elsewhere will be reaped, any others will be able to be garbage collected. I don't love resorting to WeakRef to solve this, but I believe it's the simplest way to accomplish the the desired behaviour.
* Use a single thread for all ConnectionPool ReapersJohn Hawthorn2019-05-171-7/+25
| | | | | Previously we would spawn one thread per connection pool, which ends up being wasteful for apps with several connection pools.
* Cache database version in schema cacheAli Ibrahim2019-04-031-0/+1
| | | | | | | | | | | | | | | * The database version will get cached in the schema cache file during the schema cache dump. When the database version check happens, the version will be pulled from the schema cache and thus avoid querying the database for the version. * If the schema cache file doesn't exist, we'll query the database for the version and cache it on the schema cache object. * To facilitate this change, all connection adapters now implement #get_database_version and #database_version. #database_version returns the value from the schema cache. * To take advantage of the cached database version, the database version check will now happen after the schema cache is set on the connection in the connection pool.
* Merge pull request #35297 from yhara/fix-ar-connection-handler-leakRyuta Kamizono2019-02-161-7/+11
|\ | | | | | | Fix possible memory leak of ConnectionHandler
| * Fix possible memory leak of ConnectionHandlerYutaka HARA2019-02-161-7/+11
|/ | | | refs #35296
* Fix elapsed time calculationsbogdanvlviv2019-02-081-4/+4
| | | | | | | | | | | | | | | | | | | I've found a few places in Rails code base where I think it makes sense to calculate elapsed time more precisely by using `Concurrent.monotonic_time`: - Fix calculation of elapsed time in `ActiveSupport::Cache::MemoryStore#prune` - Fix calculation of elapsed time in `ActiveRecord::ConnectionAdapters::ConnectionPool::Queue#wait_poll` - Fix calculation of elapsed time in `ActiveRecord::ConnectionAdapters::ConnectionPool#attempt_to_checkout_all_existing_connections` - Fix calculation of elapsed time in `ActiveRecord::ConnectionAdapters::Mysql2Adapter#explain` See https://docs.ruby-lang.org/en/2.5.0/Process.html#method-c-clock_gettime https://blog.dnsimple.com/2018/03/elapsed-time-with-ruby-the-right-way Related to 7c4542146f0dde962205e5a90839349631ae60fb
* Fix error raised when handler doesn't existEileen Uchitelle2019-01-251-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While working on another feature for multiple databases (auto-switching) I observed that in development the first request won't autoload the application record connection for the primary database and may not yet know about the replica connection. In my test application this caused the application to thrown an error if I tried to send the first request to the replica before the replica was connected. This wouldn't be an issue in production because the application is preloaded. In order to fix this I decided to leave the original error message and delete the new error message. I updated the original error message to include the `role` to make it a bit clearer that the connection isn't established for that particular role. The error now reads: ``` No connection pool with 'primary' found for the 'reading' role. ``` A single database application will continue uisng the original error message: ``` No connection pool with 'primary' found. ```
* Enable `Performance/UnfreezeString` copyuuji.yaginuma2018-09-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In Ruby 2.3 or later, `String#+@` is available and `+@` is faster than `dup`. ```ruby # frozen_string_literal: true require "bundler/inline" gemfile(true) do source "https://rubygems.org" gem "benchmark-ips" end Benchmark.ips do |x| x.report('+@') { +"" } x.report('dup') { "".dup } x.compare! end ``` ``` $ ruby -v benchmark.rb ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux] Warming up -------------------------------------- +@ 282.289k i/100ms dup 187.638k i/100ms Calculating ------------------------------------- +@ 6.775M (± 3.6%) i/s - 33.875M in 5.006253s dup 3.320M (± 2.2%) i/s - 16.700M in 5.032125s Comparison: +@: 6775299.3 i/s dup: 3320400.7 i/s - 2.04x slower ```
* Merge pull request #31696 from BrentWheeldon/bmw-connection-pool-load-deadlockMatthew Draper2018-08-241-1/+3
|\ | | | | Prevent deadlocks when waiting for connection from pool.
| * Prevent deadlocks when waiting for connection from pool.Brent Wheeldon2018-03-231-1/+3
| | | | | | | | | | When a thread that had the load interlock but was blocked waiting to check a connection out of the connection pool but all of the threads using the available connections were blocked waiting to obtain the load interlock an `ActiveRecord::ConnectionTimeoutError` exception was be thrown by the thread waiting for the connection. When waiting for the connection to check out we should allow loading to proceed to avoid this deadlock.
* | `retrieve_connection_pool` return a pool, not a connectionRyuta Kamizono2018-08-031-2/+2
|/
* Merge pull request #25456 from ojab/masterRyuta Kamizono2018-01-031-3/+1
|\ | | | | | | Remove dormant check
| * Remove dormant checkojab2016-06-211-3/+1
| | | | | | | | | | This check was introduced in 6edaa26 and moved through multiple refactorings. No test are broken after removal and AFAICS there is no way to trigger it.
* | Merge pull request #31221 from matthewd/flush-idle-connectionsMatthew Draper2017-11-261-16/+58
|\ \ | | | | | | Flush idle database connections
| * | Flush idle database connectionsMatthew Draper2017-11-261-16/+58
| | |
* | | Merge pull request #31173 from matthewd/connection-fork-safetyMatthew Draper2017-11-251-0/+35
|\ \ \ | |/ / |/| | Improve AR connection fork safety
| * | Improve AR connection fork safetyMatthew Draper2017-11-181-0/+35
| | | | | | | | | | | | | | | | | | Use whatever adapter-provided means we have available to ensure forked children don't send quit/shutdown/goodbye messages to the server on connections that belonged to their parent.
* | | Merge pull request #28742 from quixoten/stack_conn_poolMatthew Draper2017-11-171-6/+3
|\ \ \ | |/ / |/| | Switch to LIFO for the connection pool
| * | Fix typosDevin Christensen2017-04-131-1/+1
| | |
| * | Improve documentation and add testDevin Christensen2017-04-131-7/+4
| | |
| * | Switch to LIFO for the connection poolDevin Christensen2017-04-121-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | Using a FIFO for the connection pool can lead to issues when there are upstream components (pgbouncer, haproxy, etc.) that terminate connections that are idle after a period of time. Switching to a LIFO reduces the probability that a thread will checkout a connection that is about to be closed by an idle timeout in an upstream component.
* | | Fix RDoc formatting: `+` doesn't work with `@`ohbarye2017-08-111-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | refs: https://github.com/rails/rails/pull/30161 ``` $ echo "+@size+" | rdoc --pipe <p>+@size+</p> $ echo "<tt>@size</tt>" | rdoc --pipe <p><code>@size</code></p> ``` [ci skip]
* | | Start `@reaper.run` after connection pool initializedRyuta Kamizono2017-08-111-2/+3
| | | | | | | | | | | | | | | | | | | | | Otherwise `ConnectionPool#reap` may run before `@connections` has initialized. https://travis-ci.org/rails/rails/jobs/263037427#L888-L890
* | | Use frozen-string-literal in ActiveRecordKir Shatrov2017-07-191-0/+2
| | |
* | | Revert "Merge pull request #29540 from kirs/rubocop-frozen-string"Matthew Draper2017-07-021-1/+0
| | | | | | | | | | | | | | | This reverts commit 3420a14590c0e6915d8b6c242887f74adb4120f9, reversing changes made to afb66a5a598ce4ac74ad84b125a5abf046dcf5aa.
* | | Merge pull request #29540 from kirs/rubocop-frozen-stringMatthew Draper2017-07-021-0/+1
|\ \ \ | | | | | | | | | | | | Enforce frozen string in Rubocop
| * | | Enforce frozen string in RubocopKir Shatrov2017-07-011-0/+1
| | | |
* | | | Make ActiveRecord frozen string literal friendly.Pat Allan2017-06-201-1/+1
|/ / /
* | | Lock connection before checking it inMatthew Draper2017-04-261-6/+8
| | |
* | | Fix typos [ci skip]Tobias Fankhänel2017-04-251-2/+2
|/ / | | | | | | | | | | | | | | | | 'lookup' is the noun. 'to look up' is the verb. Looked it up just to be sure. cf. https://en.wiktionary.org/wiki/lookup https://en.wiktionary.org/wiki/look_up
* | Fix example usage of ActiveRecord::Base.establish_connectionSimon Dawson2017-03-301-1/+1
| | | | | | `ActiveRecord::Base.establish_connection` accepts a single symbol argument to specify a named connection; a single string argument appears to be interpreted as a connection URI
* | Ensure test threads share a DB connectioneileencodes2017-02-201-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This ensures multiple threads inside a transactional test to see consistent database state. When a system test starts Puma spins up one thread and Capybara spins up another thread. Because of this when tests are run the database cannot see what was inserted into the database on teardown. This is because there are two threads using two different connections. This change uses the statement cache to lock the threads to using a single connection ID instead of each not being able to see each other. This code only runs in the fixture setup and teardown so it does not affect real production databases. When a transaction is opened we set `lock_thread` to `Thread.current` so we can keep track of which connection the thread is using. When we rollback the transaction we unlock the thread and then there will be no left-over data in the database because the transaction will roll back the correct connections. [ Eileen M. Uchitelle, Matthew Draper ]
* | Fix pool_from_any_process to use most recent speceileencodes2017-01-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | If a process is forked more than once, the pool was grabbing the oldest spec, not the most recent spec. This wasn't noticed before because most folks are lilely forking the process only once. If you're forking the process multiple times however the wrong spec name will be returned and an incorrect connection will be used for the process. This fixes the issue by reversing the list of spec names so we can grab the most recent spec rather than the oldest spec.
* | Clear available connections immediatelyMatthew Draper2016-11-271-0/+2
| | | | | | | | | | | | | | | | | | | | It'll be re-cleared when it's rebuilt in with_new_connections_blocked's ensure, but we still need to clear it inside this synchronize -- we've disconnected connections that may be available in the queue, and while other threads are not allowed to make *new* connections, they are still allowed to take existing ones from there. This was incorrectly removed in d314646c965b045724e6bdb9d61dcecfabc0ba8f.
* | Distribute connections to previously blocked threads when we're doneMatthew Draper2016-11-251-19/+17
| | | | | | | | | | | | Two methods block new connections; we were already doing the right thing for clear_reloadable_connections, but it's better placed in with_new_connections_blocked, where it can work for disconnect too.
* | Merge pull request #27057 from kamipo/fix_race_conditionMatthew Draper2016-11-191-6/+7
|\ \ | | | | | | | | | Fix the race condition caused by `with_new_connections_blocked`
| * | Fix the race condition caused by `with_new_connections_blocked`Ryuta Kamizono2016-11-181-8/+4
|/ / | | | | | | | | | | `with_new_connections_blocked` was introduced at #14938. But the method sometimes causes `@new_cons_enabled = false` then never toggled to true.
* | Merge pull request #26905 from bogdanvlviv/docsAndrew White2016-11-131-6/+6
|\ \ | | | | | | Add missing `+` around a some literals.
| * | Add missing `+` around a some literals.bogdanvlviv2016-10-271-6/+6
| | | | | | | | | | | | | | | | | | Mainly around `nil` [ci skip]
* | | Merge pull request #26978 from matthewd/query-cache-poolMatthew Draper2016-11-101-0/+1
|\ \ \ | | | | | | | | Configure query caching (per thread) on the connection pool
| * | | Configure query caching (per thread) on the connection poolMatthew Draper2016-11-061-0/+1
| | | |
* | | | Add ActiveRecord::Base.connection_pool.statPavel2016-11-091-0/+18
|/ / /
* / / Add more rubocop rules about whitespacesRafael Mendonça França2016-10-291-1/+1
|/ /
* | Fix broken comments indentation caused by rubocop auto-correct [ci skip]Ryuta Kamizono2016-09-141-42/+42
| | | | | | | | | | | | All indentation was normalized by rubocop auto-correct at 80e66cc4d90bf8c15d1a5f6e3152e90147f00772. But comments was still kept absolute position. This commit aligns comments with method definitions for consistency.
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-20 12:00:15 +0000