| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
|
|
| |
Instead of passing a separete name variable, we can make the resolver
merge a name on the config, and use that before creating the Specification.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
After PR https://github.com/rails/rails/pull/24844 the documentation for
`#retrieve_connection_pool` was out of date. This commit changes:
- the reference from `@class_to_pool` to `@owner_to_pool`.
- with newer Rubies, `#fetch` isn't significantly slower than `#[]`. Since Rails 5
requires Ruby >= 2.2.2, we can just use `#fetch` here.
|
| |
|
|
|
|
|
|
| |
Follow up of #24844.
The key of `@owner_to_pool` was changed from `klass.name` to
`spec.name`. By this change "memory leaks in development mode"
will not happen, bacause the equality of string is not changed
by reloading of model files.
|
| | |
|
| |
|
|
|
|
|
| |
Some slight documentation edits and fixes. Also, run remove unnecessary
`RuntimeError`.
r? @arthurnn
|
| |
|
|
| |
[skip ci]
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ConnectionHandler will not have any knowlodge of AR models now, it will
only know about the specs.
Like that we can decouple the two, and allow the same model to use more
than one connection.
Historically, folks used to create abstract AR classes on the fly in
order to have multiple connections for the same model, and override the
connection methods.
With this, now we can override the `specificiation_id` method in the
model, to return a key, that will be used to find the connection_pool
from the handler.
|
| |
|
|
|
|
| |
These should allow external code to run blocks of user code to do
"work", at a similar unit size to a web request, without needing to get
intimate with ActionDipatch.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The previous message was misleading (especially for Ops guys) when
diagnosing problems related to the database connection.
The message was suggesting that the connection cannot be obtained which
normally assumes the need to look at the database.
But this isn't the case as the connection could not be retrieved from
the application's internal connection pool.
The new message should make it more explicit and remove the confusion.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The focus of this change is to make the API more accessible.
References to method and classes should be linked to make it easy to
navigate around.
This patch makes exzessiv use of `rdoc-ref:` to provide more readable
docs. This makes it possible to document `ActiveRecord::Base#save` even
though the method is within a separate module
`ActiveRecord::Persistence`. The goal here is to bring the API closer to
the actual code that you would write.
This commit only deals with Active Record. The other gems will be
updated accordingly but in different commits. The pass through Active
Record is not completely finished yet. A follow up commit will change
the spots I haven't yet had the time to update.
/cc @fxn
|
| | |
|
| |
|
|
|
|
|
| |
The thread_safe gem is being deprecated and all its code has been merged
into the concurrent-ruby gem. The new class, Concurrent::Map, is exactly
the same as its predecessor except for fixes to two bugs discovered
during the merge.
|
| |\
| |
| | |
Improved ActiveRecord Connection Pool docs [ci skip]
|
| | | |
|
| |/
|
|
| |
[ci skip]
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit bdc1d329d4eea823d07cf010064bd19c07099ff3.
Before:
Calculating -------------------------------------
22.000 i/100ms
-------------------------------------------------
229.700 (± 0.4%) i/s - 1.166k
Total Allocated Object: 9939
After:
Calculating -------------------------------------
24.000 i/100ms
-------------------------------------------------
246.443 (± 0.8%) i/s - 1.248k
Total Allocated Object: 7939
```
begin
require 'bundler/inline'
rescue LoadError => e
$stderr.puts 'Bundler version 1.10 or later is required. Please update your Bundler'
raise e
end
gemfile(true) do
source 'https://rubygems.org'
# gem 'rails', github: 'rails/rails', ref: 'bdc1d329d4eea823d07cf010064bd19c07099ff3'
gem 'rails', github: 'rails/rails', ref: 'd2876141d08341ec67cf6a11a073d1acfb920de7'
gem 'arel', github: 'rails/arel'
gem 'sqlite3'
gem 'benchmark-ips'
end
require 'active_record'
require 'benchmark/ips'
ActiveRecord::Base.establish_connection('sqlite3::memory:')
ActiveRecord::Migration.verbose = false
ActiveRecord::Schema.define do
create_table :users, force: true do |t|
t.string :name, :email
t.boolean :admin
t.timestamps null: false
end
end
class User < ActiveRecord::Base
default_scope { where(admin: true) }
end
admin = true
1000.times do
attributes = {
name: "Lorem ipsum dolor sit amet, consectetur adipiscing elit.",
email: "foobar@email.com",
admin: admin
}
User.create!(attributes)
admin = !admin
end
GC.disable
Benchmark.ips(5, 3) do |x|
x.report { User.all.to_a }
end
key =
if RUBY_VERSION < '2.2'
:total_allocated_object
else
:total_allocated_objects
end
before = GC.stat[key]
User.all.to_a
after = GC.stat[key]
puts "Total Allocated Object: #{after - before}"
```
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Active Record detects when the process has forked and automatically
creates a new connection pool to avoid sharing file descriptors.
If the existing connection pool had a schema cache associated with it,
the new pool should copy it to avoid unnecessarily querying the database
for its schema.
The code to detect that the process has forked is in ConnectionHandler,
but the existing test for it was in the ConnectionManagement test file.
I moved it to the right place while I was writing the new test for this
change.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Renamed `@reserved_connections` -> `@thread_cached_conns`. New name
clearly conveys the purpose of the cache, which is to speed-up
`#connection` method.
The new `@thread_cached_conns` now also uses `Thread` objects as keys
(instead of previously `Thread.current.object_id`).
Since there is no longer any synchronization around
`@thread_cached_conns`, `disconnect!` and `clear_reloadable_connections!`
methods now pre-emptively obtain ownership (via `checkout`) of all
existing connections, before modifying internal data structures.
A private method `release` has been renamed `thread_conn_uncache` to
clear-up its purpose.
Fixed some brittle `thread.status == "sleep"` tests (threads can go
into sleep even without locks).
|
| | |
|
| |
|
|
| |
Move post checkout connection verification out of mutex.synchronize.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `db:schema:cache:dump` rake task dumps the database schema structure
to `db/schema_cache.dump`. If this file is present, the schema details
are loaded into the currently checked out connection by a railtie while
Rails is booting, to avoid having to query the database for its schema.
The schema cache dump is only applied to the initial connection used to
boot the application though; other connections from the same pool are
created with an empty schema cache, and still have to load the structure
of each table directly from the database.
With this change, a copy of the schema cache is associated with the
connection pool and applied to connections as they are created.
|
| |
|
|
| |
This reverts commit 796cab45561fce268aa74e6587cdb9cae3bb243e.
|
| |
|
|
|
|
|
| |
On reconnection failure, all the connection was released.
But, it is better to release only failed connection.
This patch changes not to release all the connection but release
only failed connection.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When trying to checkout connection from connection pool,
checkout()(and checkout_and_verify) verify whether the connection
is active or not.
And, if the connection is not active, connection adapters try to
reconnect to server. And, if database is down at this moment,
reconnect fails and exception is raised.
(Ex: Mysql2::Error: Can't connect to local MySQL server through socket xxx)
But, ConnectionPool does not catch the exception, but leaks current
disconnected connection to @connection.
So, if database's temporary down happens several times and exceeds
the number of connection pool(5 by default), activerecord will be
no more available, even if database server is already recovered.
This patch fix it by catching exception and releasing connection.
|
| | |
|
| |\
| |
| |
| | |
Fix issue with reaping_frequency type.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
When using DATABASE_URL to configure ActiveRecord, :reaping_frequency
does not get converted from a string to a numeric value. This value is
eventually passed to 'sleep' and must be numeric to avoid exceptions.
This commit converts :reaping_frequency to a float when present.
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The code in `ConnectionPool#release` assumed that a single thread only
ever holds a single connection, and thus that releasing a connection
only requires the owning thread_id.
There is a trivial counterexample to this assumption: code that checks
out additional connections from the pool in the same thread. For
instance:
connection_1 = ActiveRecord::Base.connection
connection_2 = ActiveRecord::Base.connection_pool.checkout
ActiveRecord::Base.connection_pool.checkin(connection_2)
connection_3 = ActiveRecord::Base.connection
At this point, connection_1 has been removed from the
`@reserved_connections` hash, causing a NEW connection to be returned as
connection_3 and the loss of any tracking info on connection_1. As long
as the thread in this example lives, connection_1 will be inaccessible
and un-reapable. If this block of code runs more times than the size of
the connection pool in a single thread, every subsequent connection
attempt will timeout, as all of the available connections have been
leaked.
Reverts parts of 9e457a8654fa89fe329719f88ae3679aefb21e56 and
essentially all of 4367d2f05cbeda855820e25a08353d4b7b3457ac
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch uniformizes warning messages. I used the most common style
already present in the code base:
* Capitalize the first word.
* End the message with a full stop.
* "Rails 5" instead of "Rails 5.0".
* Backticks for method names and inline code.
Also, converted a few long strings into the new heredoc convention.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The current style for warning messages without newlines uses
concatenation of string literals with manual trailing spaces
where needed.
Heredocs have better readability, and with `squish` we can still
produce a single line.
This is a similar use case to the one that motivated defining
`strip_heredoc`, heredocs are super clean.
|
| |
|
|
| |
This will avoid naming clash with user defined methods
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Inspired by @tenderlove's work in
c363fff29f060e6a2effe1e4bb2c4dd4cd805d6e, this reduces the number of
strings allocated when running callbacks for ActiveRecord instances. I
measured that using this script:
```
require 'objspace'
require 'active_record'
require 'allocation_tracer'
ActiveRecord::Base.establish_connection adapter: "sqlite3",
database: ":memory:"
ActiveRecord::Base.connection.instance_eval do
create_table(:articles) { |t| t.string :name }
end
class Article < ActiveRecord::Base; end
a = Article.create name: "foo"
a = Article.find a.id
N = 10
result = ObjectSpace::AllocationTracer.trace do
N.times { Article.find a.id }
end
result.sort.each do |k,v|
p k => v
end
puts "total: #{result.values.map(&:first).inject(:+)}"
```
When I run this against master and this branch I get this output:
```
pete@balloon:~/projects/rails/activerecord$ git checkout master
M Gemfile
Switched to branch 'master'
pete@balloon:~/projects/rails/activerecord$ bundle exec ruby benchmark_allocation_with_callback_send.rb > allocations_before
pete@balloon:~/projects/rails/activerecord$ git checkout remove-dynamic-send-on-built-in-callbacks
M Gemfile
Switched to branch 'remove-dynamic-send-on-built-in-callbacks'
pete@balloon:~/projects/rails/activerecord$ bundle exec ruby benchmark_allocation_with_callback_send.rb > allocations_after
pete@balloon:~/projects/rails/activerecord$ diff allocations_before allocations_after
39d38
<
{["/home/pete/projects/rails/activesupport/lib/active_support/callbacks.rb",
81]=>[40, 0, 0, 0, 0, 0]}
42c41
< total: 630
---
> total: 590
```
In addition to this, there are two micro-optimizations present:
* Using `block.call if block` vs `yield if block_given?` when the block was being captured already.
```
pete@balloon:~/projects$ cat benchmark_block_call_vs_yield.rb
require 'benchmark/ips'
def block_capture_with_yield &block
yield if block_given?
end
def block_capture_with_call &block
block.call if block
end
def no_block_capture
yield if block_given?
end
Benchmark.ips do |b|
b.report("block_capture_with_yield") { block_capture_with_yield }
b.report("block_capture_with_call") { block_capture_with_call }
b.report("no_block_capture") { no_block_capture }
end
pete@balloon:~/projects$ ruby benchmark_block_call_vs_yield.rb
Calculating -------------------------------------
block_capture_with_yield
124979 i/100ms
block_capture_with_call
138340 i/100ms
no_block_capture 136827 i/100ms
-------------------------------------------------
block_capture_with_yield
5703108.9 (±2.4%) i/s - 28495212 in 4.999368s
block_capture_with_call
6840730.5 (±3.6%) i/s - 34169980 in 5.002649s
no_block_capture 5821141.4 (±2.8%) i/s - 29144151 in 5.010580s
```
* Defining and calling methods instead of using send.
```
pete@balloon:~/projects$ cat benchmark_method_call_vs_send.rb
require 'benchmark/ips'
class Foo
def tacos
nil
end
end
my_foo = Foo.new
Benchmark.ips do |b|
b.report('send') { my_foo.send('tacos') }
b.report('call') { my_foo.tacos }
end
pete@balloon:~/projects$ ruby benchmark_method_call_vs_send.rb
Calculating -------------------------------------
send 97736 i/100ms
call 151142 i/100ms
-------------------------------------------------
send 2683730.3 (±2.8%) i/s - 13487568 in 5.029763s
call 8005963.9 (±2.7%) i/s - 40052630 in 5.006604s
```
The result of this is making typical ActiveRecord operations slightly faster:
https://gist.github.com/phiggins/e46e51dcc7edb45b5f98
|
| |
|
|
|
|
|
| |
Honoring an overidden `rack.test` allows testing closed connection between
multiple requests. This is useful if you're working on database resiliency, to
ensure the connection is in the expected state from one request to another on
the same worker.
|
| |
|
|
|
|
|
|
| |
seamusabshere/numerify-pool-checkout-timeout-from-urls-4-1-stable
Make sure :checkout_timeout and :dead_connection_timeout are numbers
Conflicts:
activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb
|
| |
|
|
|
|
|
| |
This makes the implicit description of how connection pooling works a
little more explicit. It converts the examples of a model hierarchy into
actual Ruby code and demonstrates how the key structure of the
database.yml relates to the `establish_connection` method.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
.. not a general timeout.
Now, if a thread checks out a connection then dies, we can immediately
recover that connection and re-use it.
This should alleviate the pool exhaustion discussed in #12867. More
importantly, it entirely avoids the potential issues of the reaper
attempting to check whether connections are still active: as long as the
owning thread is alive, the connection is its business alone.
As a no-op reap is now trivial (only entails checking a thread status
per connection), we can also perform one in-line any time we decide to
sleep for a connection.
|
| |
|
|
| |
or no conn available from the pool
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Fixes #11497
As `ActiveRecord::ConnectionAdapters::ConnectionManagement` middleware does not rescue from Exception (but only from StandardError),
the Connection Pool quickly runs out of connections when multiple erroneous Requests come in right after each other.
Recueing from all exceptions and not just StandardError, fixes this behaviour.
|
| | |
|
