| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The following PR adds behavior to Rails to allow an application to
automatically switch it's connection from the primary to the replica.
A request will be sent to the replica if:
* The request is a read request (`GET` or `HEAD`)
* AND It's been 2 seconds since the last write to the database (because
we don't want to send a user to a replica if the write hasn't made it
to the replica yet)
A request will be sent to the primary if:
* It's not a GET/HEAD request (ie is a POST, PATCH, etc)
* Has been less than 2 seconds since the last write to the database
The implementation that decides when to switch reads (the 2 seconds) is
"safe" to use in production but not recommended without adequate testing
with your infrastructure. At GitHub in addition to the a 5 second delay
we have a curcuit breaker that checks the replication delay
and will send the query to a replica before the 5 seconds has passed.
This is specific to our application and therefore not something Rails
should be doing for you. You'll need to test and implement more robust
handling of when to switch based on your infrastructure. The auto
switcher in Rails is meant to be a basic implementation / API that acts
as a guide for how to implement autoswitching.
The impementation here is meant to be strict enough that you know how to
implement your own resolver and operations classes but flexible enough
that we're not telling you how to do it.
The middleware is not included automatically and can be installed in
your application with the classes you want to use for the resolver and
operations passed in. If you don't pass any classes into the middleware
the Rails default Resolver and Session classes will be used.
The Resolver decides what parameters define when to
switch, Operations sets timestamps for the Resolver to read from. For
example you may want to use cookies instead of a session so you'd
implement a Resolver::Cookies class and pass that into the middleware
via configuration options.
```
config.active_record.database_selector = { delay: 2.seconds }
config.active_record.database_resolver = MyResolver
config.active_record.database_operations = MyResolver::MyCookies
```
Your classes can inherit from the existing classes and reimplment the
methods (or implement more methods) that you need to do the switching.
You only need to implement methods that you want to change. For example
if you wanted to set the session token for the last read from a replica
you would reimplement the `read_from_replica` method in your resolver
class and implement a method that updates a new timestamp in your
operations class.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While the three-tier config makes it easier to define databases for
multiple database applications, it quickly became clear to offer full
support for multiple databases we need to change the way the connections
hash was handled.
A three-tier config means that when Rails needed to choose a default
configuration (in the case a user doesn't ask for a specific
configuration) it wasn't clear to Rails which the default was. I
[bandaid fixed this so the rake tasks could work](#32271) but that fix
wasn't correct because it actually doubled up the configuration hashes.
Instead of attemping to manipulate the hashes @tenderlove and I decided
that it made more sense if we converted the hashes to objects so we can
easily ask those object questions. In a three tier config like this:
```
development:
primary:
database: "my_primary_db"
animals:
database; "my_animals_db"
```
We end up with an object like this:
```
@configurations=[
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbded10
@env_name="development",@spec_name="primary",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>,
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbdea90
@env_name="development",@spec_name="animals",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>
]>
```
The configurations setter takes the database configuration set by your
application and turns them into an
`ActiveRecord::DatabaseConfigurations` object that has one getter -
`@configurations` which is an array of all the database objects.
The configurations getter returns this object by default since it acts
like a hash in most of the cases we need. For example if you need to
access the default `development` database we can simply request it as we
did before:
```
ActiveRecord::Base.configurations["development"]
```
This will return primary development database configuration hash:
```
{ "database" => "my_primary_db" }
```
Internally all of Active Record has been converted to use the new
objects. I've built this to be backwards compatible but allow for
accessing the hash if needed for a deprecation period. To get the
original hash instead of the object you can either add `to_h` on the
configurations call or pass `legacy: true` to `configurations.
```
ActiveRecord::Base.configurations.to_h
=> { "development => { "database" => "my_primary_db" } }
ActiveRecord::Base.configurations(legacy: true)
=> { "development => { "database" => "my_primary_db" } }
```
The new configurations object allows us to iterate over the Active
Record configurations without losing the known environment or
specification name for that configuration. You can also select all the
configs for an env or env and spec. With this we can always ask
any object what environment it belongs to:
```
db_configs = ActiveRecord::Base.configurations.configurations_for("development")
=> #<ActiveRecord::DatabaseConfigurations:0x00007fd1acbdf800
@configurations=[
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbded10
@env_name="development",@spec_name="primary",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>,
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbdea90
@env_name="development",@spec_name="animals",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>
]>
db_config.env_name
=> "development"
db_config.spec_name
=> "primary"
db_config.config
=> { "adapter"=>"sqlite3", "database"=>"db/development.sqlite3" }
```
The configurations object is more flexible than the configurations hash
and will allow us to build on top of the connection management in order
to add support for primary/replica connections, sharding, and
constructing queries for associations that live in multiple databases.
|
| |
|
|
|
|
|
| |
To solve the problem #32299, just enough to introduce
`fk_ignore_pattern` option.
I don't think there is a need to expose these constants.
|
| |\
| |
| | |
Expose foreign key name ignore pattern in configuration
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
When dumping the database schema, Rails will dump foreign key names only
if those names were not generate by Rails. Currently this is determined
by checking if the foreign key name is `fk_rails_` followed by
a 10-character hash.
At [Cookpad](https://github.com/cookpad), we use
[Departure](https://github.com/departurerb/departure) (Percona's
pt-online-schema-change runner for ActiveRecord migrations) to run migrations.
Often, `pt-osc` will make a copy of a table in order to run a long migration
without blocking it. In this copy process, foreign keys are copied too,
but [their name is prefixed with an underscore to prevent name collision
](https://www.percona.com/doc/percona-toolkit/LATEST/pt-online-schema-change.html#cmdoption-pt-online-schema-change-alter-foreign-keys-method).
In the process described above, we often end up with a development
database that contains foreign keys which name starts with `_fk_rails_`.
That name does not match the ignore pattern, so next time Rails dumps
the database schema (eg. when running `rake db:migrate`), our
`db/schema.rb` file ends up containing those unwanted foreign key names.
This also produces an unwanted git diff that we'd prefer not to commit.
In this PR, I'd like to suggest a way to expose the foreign key name
ignore pattern to the Rails configuration, so that individual projects
can decide on a different pattern of foreign keys that will not get
their names dumped in `schema.rb`.
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Moves the configs_for and DatabaseConfig struct into it's own file. I
was considering doing this in a future refactoring but our set up forced
me to move it now. You see there are `mattr_accessor`'s on the Core
module that have default settings. For example the `schema_format`
defaults to Ruby. So if I call `configs_for` or any methods in the Core
module it will reset the `schema_format` to `:ruby`. By moving it to
it's own class we can keep the logic contained and avoid this
unfortunate issue.
The second change here does a double loop over the yaml files. Bear with
me...
Our tests dictate that we need to load an environment before our rake
tasks because we could have something in an environment that the
database.yml depends on. There are side-effects to this and I think
there's a deeper bug that needs to be fixed but that's for another
issue. The gist of the problem is when I was creating the dynamic rake
tasks if the yaml that that rake task is calling evaluates code (like
erb) that calls the environment configs the code will blow up because
the environment is not loaded yet.
To avoid this issue we added a new method that simply loads the yaml and
does not evaluate the erb or anything in it. We then use that yaml to
create the task name. Inside the task name we can then call
`load_config` and load the real config to actually call the code
internal to the task. I admit, this is gross, but refactoring can't all
be pretty all the time and I'm working hard with `@tenderlove` to
refactor much more of this code to get to a better place re connection
management and rake tasks.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Provides both a forked process and threaded parallelization options. To
use add `parallelize` to your test suite.
Takes a `workers` argument that controls how many times the process
is forked. For each process a new database will be created suffixed
with the worker number; test-database-0 and test-database-1
respectively.
If `ENV["PARALLEL_WORKERS"]` is set the workers argument will be ignored
and the environment variable will be used instead. This is useful for CI
environments, or other environments where you may need more workers than
you do for local testing.
If the number of workers is set to `1` or fewer, the tests will not be
parallelized.
The default parallelization method is to fork processes. If you'd like to
use threads instead you can pass `with: :threads` to the `parallelize`
method. Note the threaded parallelization does not create multiple
database and will not work with system tests at this time.
parallelize(workers: 2, with: :threads)
The threaded parallelization uses Minitest's parallel exector directly.
The processes paralleliztion uses a Ruby Drb server.
For parallelization via threads a setup hook and cleanup hook are
provided.
```
class ActiveSupport::TestCase
parallelize_setup do |worker|
# setup databases
end
parallelize_teardown do |worker|
# cleanup database
end
parallelize(workers: 2)
end
```
[Eileen M. Uchitelle, Aaron Patterson]
|
| |
|
|
| |
Follow up of #31606.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Attribute modules (`Attribute`, `Attributes`, `AttributeSet`) uses
`Type`, but referencing `Type` before the modules still fail.
```
% ./bin/test -w test/cases/attribute_test.rb -n test_with_value_from_user_validates_the_value
Run options: -n test_with_value_from_user_validates_the_value --seed 31876
E
Error:
ActiveModel::AttributeTest#test_with_value_from_user_validates_the_value:
NameError: uninitialized constant ActiveModel::AttributeTest::Type
/Users/kamipo/src/github.com/rails/rails/activemodel/test/cases/attribute_test.rb:233:in `block in <class:AttributeTest>'
bin/test test/cases/attribute_test.rb:232
Finished in 0.002985s, 335.0479 runs/s, 335.0479 assertions/s.
1 runs, 1 assertions, 0 failures, 1 errors, 0 skips
```
Probably we need more autoloading at least `Type`.
|
| |
|
|
|
| |
Use these to back the attributes API. Stop automatically including
ActiveModel::Dirty in ActiveModel::Attributes, and make it optional.
|
| |
|
|
| |
This basically reverts 9d4f79d3d394edb74fa2192e5d9ad7b09ce50c6d
|
| | |
|
| | |
|
| |
|
|
|
|
| |
".. with __dir__ we can restore order in the Universe." - by @fxn
Related to 5b8738c2df003a96f0e490c43559747618d10f5f
|
| |
|
| |
Specify complete path to AR::LegacyYamlAdapter
|
| |
|
|
|
|
|
|
| |
Per https://www.timeanddate.com/counters/firstnewyear.html, it's already
2017 in a lot of places, so we should bump the Rails license years to
2017.
[ci skip]
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes a bug that can occur when ActiveJob tries to access ActiveRecord.
Specifically, I had an Active Job process fail on Sidekiq with this error:
```
ActiveJob::DeserializationError: Error while trying to deserialize arguments:
uninitialized constant ActiveRecord::Core::ClassMethods::TableMetadata
Did you mean? ActiveRecord::TableMetadata
```
raised by these lines of code:
```
[GEM_ROOT]/gems/activerecord-5.0.0.1/lib/active_record/core.rb:300 :in `table_metadata`
298
299 def table_metadata # :nodoc:
300 TableMetadata.new(self, arel_table)
301 end
302 end
[GEM_ROOT]/gems/activerecord-5.0.0.1/lib/active_record/core.rb:273 :in `predicate_builder`
[GEM_ROOT]/gems/activerecord-5.0.0.1/lib/active_record/core.rb:290 :in `relation`
```
The problem seems to be that, inside ActiveRecord::Core, the `TableMetadata`
class has not been loaded and, therefore, Rails tries to access the constant
`ActiveRecord::Core::ClassMethods::TableMetadata` which does not exist.
Eager loading `ActiveRecord::TableMetadata` should fix the issue.
@rafaelfranca -- see our Campfire discussion
|
| |
|
|
|
| |
The current code base is not uniform. After some discussion,
we have chosen to go with double quotes by default.
|
| |
|
|
|
|
| |
These should allow external code to run blocks of user code to do
"work", at a similar unit size to a web request, without needing to get
intimate with ActionDipatch.
|
| |\
| |
| | |
Prevent destructive action on production database
|
| | |
| |
| |
| |
| |
| |
| | |
This PR introduces a key/value type store to Active Record that can be used for storing internal values. It is an alternative implementation to #21237 cc @sgrif @matthewd.
It is possible to run your tests against your production database by accident right now. While infrequently, but as an anecdotal data point, Heroku receives a non-trivial number of requests for a database restore due to this happening. In these cases the loss can be large.
To prevent against running tests against production we can store the "environment" version that was used when migrating the database in a new internal table. Before executing tests we can see if the database is a listed in `protected_environments` and abort. There is a manual escape valve to force this check from happening with environment variable `DISABLE_DATABASE_ENVIRONMENT_CHECK=1`.
|
| |/ |
|
| | |
|
| |
|
|
| |
Fixes #20626
|
| |
|
|
|
|
|
|
|
|
| |
[fixes #18606]
Make belongs_to use touch over touch_later when running the callbacks.
Add more tests and small method rename
Thanks Jeremy for the feedback.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I should have done this in the first place. We are now serializing an
explicit version so we can make more careful changes in the future. This
will load Active Record objects which were serialized in Rails 4.1.
There will be bugs, as YAML serialization was at least partially broken
back then. There will also be edge cases that we might not be able to
handle, especially if the type of a column has changed.
In addition, we're passing this as `from_database`, since that is
required for serialized columns at minimum. All other types were
serializing the cast value. At a glance, there should be no types for
which this is a problem.
Finally, dirty checking information will be lost on records serialized
in 4.1, so no columns will be marked as changed.
|
| | |
|
| |
|
|
|
|
| |
Update SecureToken Docs
Add Changelog entry for has_secure_token [ci skip]
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This class cares far too much about the internals of other parts of
Active Record. This is an attempt to break out a meaningful object which
represents the needs of the predicate builder. I'm not fully satisfied
with the name, but the general concept is an object which represents a
table, the associations to/from that table, and the types associated
with it. Many of these exist at the `ActiveRecord::Base` class level,
not as properties of the table itself, hence the need for another
object. Currently it provides these by holding a reference to the class,
but that will likely change in the future. This allows the predicate
builder to remain wholy concerned with building predicates.
/cc @mrgilman
|
| |
|
|
|
|
|
| |
This reverts commit 6f3c64eeb1dc8288dae49f114aaf619adc7dcb7f.
Conflicts:
activerecord/CHANGELOG.md
|
| |
|
|
|
|
| |
This reverts commit a03097759bd7103bb9db253e7ba095f011453f75.
This needs more work before it would work correctly on master.
|
| |
|
|
|
|
|
|
| |
Fixed issue with ActiveRecord serialize object as JSON
Conflicts:
activerecord/CHANGELOG.md
activerecord/lib/active_record/attribute_methods/serialization.rb
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Moved `Builder` to its own file, as it started looking very weird once I
added private methods to the `AttributeSet` class and the `Builder`
class started to grow.
Would like to refactor `fetch_value` to change to
```ruby
self[name].value(&block)
```
But that requires the attributes to know about their name, which they
currently do not.
|
| |
|
|
|
| |
Mostly delegation to start, but we can start moving a lot of behavior in
bulk to this object.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There's a lot more that can be moved to these, but this felt like a good
place to introduce the object. Plans are:
- Remove all knowledge of type casting from the columns, beyond a
reference to the cast_type
- Move type_cast_for_database to these objects
- Potentially make them mutable, introduce a state machine, and have
dirty checking handled here as well
- Move `attribute`, `decorate_attribute`, and anything else that
modifies types to mess with this object, not the columns hash
- Introduce a collection object to manage these, reduce allocations, and
not require serializing the types
|
| | |
|
| | |
|
| |
|
|
| |
map to integers in the database, but can be queried by name
|
| | |
|
| |
|
|
| |
/cc @tenderlove
|
| |
|
|
| |
They were deprecated in 4.0, planned to remove in 4.1
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Suppose Man has_many interests, and inverse_of is used.
Man.first.interests.first.man will correctly execute two queries,
avoiding the need for a third query when Interest#man is called. This is
because CollectionAssociation#first calls set_inverse_instance.
However Man.first.interests.where("1=1").first.man will execute three
queries, even though this is obviously a subset of the records in the
association.
This is because calling where("1=1") spawns a new Relation object from
the CollectionProxy object, and the Relation has no knowledge of the
association, so it cannot set the inverse instance.
This commit solves the problem by making relations spawned from
CollectionProxies return a new Relation subclass called
AssociationRelation, which does know about associations. Records loaded
from this class will get the inverse instance set properly.
Fixes #5717.
Live commit from La Conf! :sparkles:
|
| |\
| |
| |
| |
| |
| |
| | |
Statement cache
Conflicts:
activerecord/CHANGELOG.md
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
