| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Renamed `@reserved_connections` -> `@thread_cached_conns`. New name
clearly conveys the purpose of the cache, which is to speed-up
`#connection` method.
The new `@thread_cached_conns` now also uses `Thread` objects as keys
(instead of previously `Thread.current.object_id`).
Since there is no longer any synchronization around
`@thread_cached_conns`, `disconnect!` and `clear_reloadable_connections!`
methods now pre-emptively obtain ownership (via `checkout`) of all
existing connections, before modifying internal data structures.
A private method `release` has been renamed `thread_conn_uncache` to
clear-up its purpose.
Fixed some brittle `thread.status == "sleep"` tests (threads can go
into sleep even without locks).
|
| | |
|
| |
|
|
| |
Move post checkout connection verification out of mutex.synchronize.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `db:schema:cache:dump` rake task dumps the database schema structure
to `db/schema_cache.dump`. If this file is present, the schema details
are loaded into the currently checked out connection by a railtie while
Rails is booting, to avoid having to query the database for its schema.
The schema cache dump is only applied to the initial connection used to
boot the application though; other connections from the same pool are
created with an empty schema cache, and still have to load the structure
of each table directly from the database.
With this change, a copy of the schema cache is associated with the
connection pool and applied to connections as they are created.
|
| |
|
|
| |
This reverts commit 796cab45561fce268aa74e6587cdb9cae3bb243e.
|
| |
|
|
|
|
|
| |
On reconnection failure, all the connection was released.
But, it is better to release only failed connection.
This patch changes not to release all the connection but release
only failed connection.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When trying to checkout connection from connection pool,
checkout()(and checkout_and_verify) verify whether the connection
is active or not.
And, if the connection is not active, connection adapters try to
reconnect to server. And, if database is down at this moment,
reconnect fails and exception is raised.
(Ex: Mysql2::Error: Can't connect to local MySQL server through socket xxx)
But, ConnectionPool does not catch the exception, but leaks current
disconnected connection to @connection.
So, if database's temporary down happens several times and exceeds
the number of connection pool(5 by default), activerecord will be
no more available, even if database server is already recovered.
This patch fix it by catching exception and releasing connection.
|
| | |
|
| |\
| |
| |
| | |
Fix issue with reaping_frequency type.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
When using DATABASE_URL to configure ActiveRecord, :reaping_frequency
does not get converted from a string to a numeric value. This value is
eventually passed to 'sleep' and must be numeric to avoid exceptions.
This commit converts :reaping_frequency to a float when present.
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The code in `ConnectionPool#release` assumed that a single thread only
ever holds a single connection, and thus that releasing a connection
only requires the owning thread_id.
There is a trivial counterexample to this assumption: code that checks
out additional connections from the pool in the same thread. For
instance:
connection_1 = ActiveRecord::Base.connection
connection_2 = ActiveRecord::Base.connection_pool.checkout
ActiveRecord::Base.connection_pool.checkin(connection_2)
connection_3 = ActiveRecord::Base.connection
At this point, connection_1 has been removed from the
`@reserved_connections` hash, causing a NEW connection to be returned as
connection_3 and the loss of any tracking info on connection_1. As long
as the thread in this example lives, connection_1 will be inaccessible
and un-reapable. If this block of code runs more times than the size of
the connection pool in a single thread, every subsequent connection
attempt will timeout, as all of the available connections have been
leaked.
Reverts parts of 9e457a8654fa89fe329719f88ae3679aefb21e56 and
essentially all of 4367d2f05cbeda855820e25a08353d4b7b3457ac
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch uniformizes warning messages. I used the most common style
already present in the code base:
* Capitalize the first word.
* End the message with a full stop.
* "Rails 5" instead of "Rails 5.0".
* Backticks for method names and inline code.
Also, converted a few long strings into the new heredoc convention.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The current style for warning messages without newlines uses
concatenation of string literals with manual trailing spaces
where needed.
Heredocs have better readability, and with `squish` we can still
produce a single line.
This is a similar use case to the one that motivated defining
`strip_heredoc`, heredocs are super clean.
|
| |
|
|
| |
This will avoid naming clash with user defined methods
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Inspired by @tenderlove's work in
c363fff29f060e6a2effe1e4bb2c4dd4cd805d6e, this reduces the number of
strings allocated when running callbacks for ActiveRecord instances. I
measured that using this script:
```
require 'objspace'
require 'active_record'
require 'allocation_tracer'
ActiveRecord::Base.establish_connection adapter: "sqlite3",
database: ":memory:"
ActiveRecord::Base.connection.instance_eval do
create_table(:articles) { |t| t.string :name }
end
class Article < ActiveRecord::Base; end
a = Article.create name: "foo"
a = Article.find a.id
N = 10
result = ObjectSpace::AllocationTracer.trace do
N.times { Article.find a.id }
end
result.sort.each do |k,v|
p k => v
end
puts "total: #{result.values.map(&:first).inject(:+)}"
```
When I run this against master and this branch I get this output:
```
pete@balloon:~/projects/rails/activerecord$ git checkout master
M Gemfile
Switched to branch 'master'
pete@balloon:~/projects/rails/activerecord$ bundle exec ruby benchmark_allocation_with_callback_send.rb > allocations_before
pete@balloon:~/projects/rails/activerecord$ git checkout remove-dynamic-send-on-built-in-callbacks
M Gemfile
Switched to branch 'remove-dynamic-send-on-built-in-callbacks'
pete@balloon:~/projects/rails/activerecord$ bundle exec ruby benchmark_allocation_with_callback_send.rb > allocations_after
pete@balloon:~/projects/rails/activerecord$ diff allocations_before allocations_after
39d38
<
{["/home/pete/projects/rails/activesupport/lib/active_support/callbacks.rb",
81]=>[40, 0, 0, 0, 0, 0]}
42c41
< total: 630
---
> total: 590
```
In addition to this, there are two micro-optimizations present:
* Using `block.call if block` vs `yield if block_given?` when the block was being captured already.
```
pete@balloon:~/projects$ cat benchmark_block_call_vs_yield.rb
require 'benchmark/ips'
def block_capture_with_yield &block
yield if block_given?
end
def block_capture_with_call &block
block.call if block
end
def no_block_capture
yield if block_given?
end
Benchmark.ips do |b|
b.report("block_capture_with_yield") { block_capture_with_yield }
b.report("block_capture_with_call") { block_capture_with_call }
b.report("no_block_capture") { no_block_capture }
end
pete@balloon:~/projects$ ruby benchmark_block_call_vs_yield.rb
Calculating -------------------------------------
block_capture_with_yield
124979 i/100ms
block_capture_with_call
138340 i/100ms
no_block_capture 136827 i/100ms
-------------------------------------------------
block_capture_with_yield
5703108.9 (±2.4%) i/s - 28495212 in 4.999368s
block_capture_with_call
6840730.5 (±3.6%) i/s - 34169980 in 5.002649s
no_block_capture 5821141.4 (±2.8%) i/s - 29144151 in 5.010580s
```
* Defining and calling methods instead of using send.
```
pete@balloon:~/projects$ cat benchmark_method_call_vs_send.rb
require 'benchmark/ips'
class Foo
def tacos
nil
end
end
my_foo = Foo.new
Benchmark.ips do |b|
b.report('send') { my_foo.send('tacos') }
b.report('call') { my_foo.tacos }
end
pete@balloon:~/projects$ ruby benchmark_method_call_vs_send.rb
Calculating -------------------------------------
send 97736 i/100ms
call 151142 i/100ms
-------------------------------------------------
send 2683730.3 (±2.8%) i/s - 13487568 in 5.029763s
call 8005963.9 (±2.7%) i/s - 40052630 in 5.006604s
```
The result of this is making typical ActiveRecord operations slightly faster:
https://gist.github.com/phiggins/e46e51dcc7edb45b5f98
|
| |
|
|
|
|
|
| |
Honoring an overidden `rack.test` allows testing closed connection between
multiple requests. This is useful if you're working on database resiliency, to
ensure the connection is in the expected state from one request to another on
the same worker.
|
| |
|
|
|
|
|
|
| |
seamusabshere/numerify-pool-checkout-timeout-from-urls-4-1-stable
Make sure :checkout_timeout and :dead_connection_timeout are numbers
Conflicts:
activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb
|
| |
|
|
|
|
|
| |
This makes the implicit description of how connection pooling works a
little more explicit. It converts the examples of a model hierarchy into
actual Ruby code and demonstrates how the key structure of the
database.yml relates to the `establish_connection` method.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
.. not a general timeout.
Now, if a thread checks out a connection then dies, we can immediately
recover that connection and re-use it.
This should alleviate the pool exhaustion discussed in #12867. More
importantly, it entirely avoids the potential issues of the reaper
attempting to check whether connections are still active: as long as the
owning thread is alive, the connection is its business alone.
As a no-op reap is now trivial (only entails checking a thread status
per connection), we can also perform one in-line any time we decide to
sleep for a connection.
|
| |
|
|
| |
or no conn available from the pool
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Fixes #11497
As `ActiveRecord::ConnectionAdapters::ConnectionManagement` middleware does not rescue from Exception (but only from StandardError),
the Connection Pool quickly runs out of connections when multiple erroneous Requests come in right after each other.
Recueing from all exceptions and not just StandardError, fixes this behaviour.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
mysql can't handle a parallel thread pinging the connection, so we can
get wrong results or segvs
This reverts commit 7cc588b684f6d1af3e7fab1edfa6715e269e41a2.
|
| | |
|
| |
|
|
| |
see: https://github.com/blog/1406-namespaced-gists
|
| |
|
|
| |
or the ConnectionPool silently fails to close connections inside the Thread
|
| | |
|
| |
|
|
|
|
| |
* dependencies/autoload
* concern
* deprecation
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary of the changes:
* Add thread_safe gem.
* Use thread safe cache for digestor caching.
* Replace manual synchronization with ThreadSafe::Cache in Relation::Delegation.
* Replace @attribute_method_matchers_cache Hash with ThreadSafe::Cache.
* Use TS::Cache to avoid the synchronisation overhead on listener retrieval.
* Replace synchronisation with TS::Cache usage.
* Use a preallocated array for performance/memory reasons.
* Update the controllers cache to the new AS::Dependencies::ClassCache API.
The original @controllers cache no longer makes much sense after @tenderlove's
changes in 7b6bfe84f3 and f345e2380c.
* Use TS::Cache in the connection pool to avoid locking overhead.
* Use TS::Cache in ConnectionHandler.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Keying these hashes by klass causes reloadable classes to never get
freed. Thanks to @thedarkone for pointing this out in
the comments on 221571beb6b4bb7437989bdefaf421f993ab6002.
This doesn't seem to make a massive difference to performance.
Benchmark
---------
require 'active_record'
require 'benchmark/ips'
class Post < ActiveRecord::Base
establish_connection adapter: 'sqlite3', database: ':memory:'
end
GC.disable
Benchmark.ips(20) do |r|
r.report { Post.connection }
end
Before
------
Calculating -------------------------------------
5632 i/100ms
-------------------------------------------------
218671.0 (±1.9%) i/s - 4364800 in 19.969401s
After
-----
Calculating -------------------------------------
8743 i/100ms
-------------------------------------------------
206525.9 (±17.8%) i/s - 4039266 in 19.992590s
|
| |
|
|
|
|
|
|
| |
Rather than just changing it and hoping for the best.
Requested by @jeremy:
https://github.com/rails/rails/commit/ba1544d71628abff2777c9c514142d7e9a159111#commitcomment-2106059
|
| |
|
|
|
|
|
|
|
|
| |
In the end I think the pain of implementing this seamlessly was not
worth the gain provided.
The intention was that it would allow plain ruby objects that might not
live in your main application to be subclassed and have persistence
mixed in. But I've decided that the benefit of doing that is not worth
the amount of complexity that the implementation introduced.
|
| | |
|
| |
|
|
|
| |
1. Unused variable
2. possibly useless use of a variable in
void context
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As a result of different commits, ConnectionPool had become
of two minds about exceptions, sometimes using PoolFullError
and sometimes using ConnectionTimeoutError. In fact, it was
using ConnectionTimeoutError internally, but then recueing
and re-raising as a PoolFullError.
There's no reason for this bifurcation, standardize on
ConnectionTimeoutError, which is the rails2 name and still
accurately describes semantics at this point.
History
In Rails2, ConnectionPool raises a ConnectionTimeoutError if
it can't get a connection within timeout.
Originally in master/rails3, @tenderlove had planned on removing
wait/blocking in connectionpool entirely, at that point he changed
exception to PoolFullError.
But then later wait/blocking came back, but exception remained
PoolFullError.
Then in 02b233556377 pmahoney introduced fair waiting logic, and
brought back ConnectionTimeoutError, introducing the weird bifurcation.
ConnectionTimeoutError accurately describes semantics as of this
point, and is backwards compat with rails2, there's no reason
for PoolFullError to be introduced, and no reason for two
different exception types to be used internally, no reason
to rescue one and re-raise as another. Unify!
|
| | |
|
| | |
|
| |
|
|
| |
We don't need separate @class_to_pool and @connection_pool hashes.
|
| | |
|
| |
|
|
|
|
| |
* Loop rather than recurse in retrieve_connection_pool
* Key the hash by class rather than class name. This avoids creating
unnecessary strings.
|
| |
|
|
|
| |
Get rid of ActiveModel::Configuration, make better use of
ActiveSupport::Concern + class_attribute, etc.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The core of this fix is a threadsafe, fair Queue class. It is
very similar to Queue in stdlib except that it supports waiting
with a timeout.
The issue this solves is that if several threads are contending for
database connections, an unfair queue makes is possible that a thread
will timeout even while other threads successfully acquire and release
connections. A fair queue means the thread that has been waiting the
longest will get the next available connection.
This includes a few test fixes to avoid test ordering issues that
cropped up during development of this patch.
|
| | |
|
