| Commit message (Collapse) | Author | Age | Files | Lines |
... | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This is a 4th attempt to make counter cache transactional completely.
Past attempts: #9236, #14849, #23357.
All existing counter cache issues (increment/decrement twice, lost
increment) are caused due to updating counter cache on the outside of
the record saving transaction by assigning belongs_to record, even
though assigning that doesn't cause the record saving.
We have the `@_after_replace_counter_called` guard condition to mitigate
double increment/decrement issues, but we can't completely prevent that
inconsistency as long as updating counter cache on the outside of the
transaction, since saving the record is not always happened after that.
We already have handling counter cache after create/update/destroy,
https://github.com/rails/rails/blob/1b90f614b1b3d06b7f02a8b9ea6cd84f15d58643/activerecord/lib/active_record/counter_cache.rb#L162-L189
https://github.com/rails/rails/blob/1b90f614b1b3d06b7f02a8b9ea6cd84f15d58643/activerecord/lib/active_record/associations/builder/belongs_to.rb#L33-L59
so just removing assigning logic on the belongs_to association makes
counter cache transactional completely.
Closes #14849.
Closes #23357.
Closes #31493.
Closes #31494.
Closes #32372.
Closes #33113.
Closes #33117
Closes #33129.
Closes #33458.
|
|\ \ \ \
| |_|/ /
|/| | | |
Deprecate ActiveRecord::Result#to_hash in favor of #to_a
|
| | |/
| |/|
| | |
| | |
| | |
| | |
| | | |
method returns an array of hashes, not a hash
e.g. Hash.try_convert(result) calls #to_hash and raises a TypeError
[Gannon McGibbon + Kevin Cheng]
|
| |/
|/|
| |
| |
| |
| | |
When you pass an empty array to find we know we shoudl return an empty
array but it is surprising that we are returning the original empty
array instead of a new one.
|
|/
|
|
|
|
|
|
|
| |
`topic` and `reply` belongs_to associations on `SillyReply` are defined
with the same `foreign_key` (`parent_id`) and `counter_cache`
(`replies_count`) columns.
This would cause unintentional side-effect (e.g. saving `SillyReply`
object would cause double increment `replies_count`), so it is better to
avoid that side-effect.
|
|
|
|
|
|
|
| |
block
`ActiveRecord::MigrationContext.new` just create an instance, doesn't
have any side-effect.
|
|\
| |
| |
| |
| | |
faucct/bugfix/preload_multiple_instances_of_same_record
ActiveRecord::Associations::Preloader should preload all instances of the same record
|
| |
| |
| |
| | |
same record
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
MySQL supports JOINs to UPDATE, so if column name isn't qualified by
table name, it would cause an ambiguous error:
```
Mysql2::Error: Column 'integer' in field list is ambiguous: UPDATE `pets` INNER JOIN `toys` ON `toys`.`pet_id` = `pets`.`pet_id` SET `integer` = COALESCE(`integer`, 0) + 1 WHERE `toys`.`name` = ?
```
|
| |
| |
| |
| |
| |
| |
| |
| | |
`relations_test.rb`
`persistence_test.rb` and `relations_test.rb` have too many lines, so
I'd like to extract relation around tests to dedicated files before
newly test added.
|
|\ \
| |/
|/| |
Fallback to unprepared statement only when bind params limit is exceeded
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This is a follow up and/or an alternative of #33844.
Unlike #33844, this would attempt to construct unprepared statement only
when bind params limit (mysql2 65535, pg 65535, sqlite3 249999) is
exceeded.
I only defined 65535 as the limit, not defined 249999 for sqlite3, since
it is an edge case, I'm not excited to add less worth extra code.
|
| |
| |
| |
| |
| | |
Follow up #33874.
Related #23393.
|
|/ |
|
|\
| |
| |
| | |
Allow subclasses to redefine autosave callbacks for associated records
|
| | |
|
|\ \
| | |
| | | |
Eager loading/preloading should be worked regardless of large number of records
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Since 213796f, bind params are used for IN clause if enabled prepared
statements.
Unfortunately, most adapter modules have a limitation for # of bind
params (mysql2 65535, pg 65535, sqlite3 250000). So if eager loading
large number of records at once, that query couldn't be sent to the
database.
Since eager loading/preloading queries are auto-generated by Active
Record itself, so it should be worked regardless of large number of
records like as before.
Fixes #33702.
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
- Move
```
filter_attributes = self.filter_attributes.map(&:to_s).to_set
filter_attributes.include?(attribute_name) && !read_attribute(attribute_name).nil?
```
to private method.
- Fix tests in `activerecord/test/cases/filter_attributes_test.rb`
- Ensure that `teardown` sets `ActiveRecord::Base.filter_attributes` to
previous state.
- Ensure that `Admin::Account.filter_attributes` is set to previous
state in the "filter_attributes could be overwritten by models" test.
Follow up #33756
|
|/ /
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Use utf8mb4 character set by default
`utf8mb4` character set supports supplementary characters including emoji.
`utf8` character set with 3-Byte encoding is not enough to support them.
There was a downside of 4-Byte length character set with MySQL 5.5 and 5.6:
"ERROR 1071 (42000): Specified key was too long; max key length is 767 bytes"
for Rails string data type which is mapped to varchar(255) type.
MySQL 5.7 supports 3072 byte key prefix length by default.
* Remove `DEFAULT COLLATE` from Active Record unit test databases
There should be no "one size fits all" collation in MySQL 5.7.
Let MySQL server choose the default collation for Active Record
unit test databases.
Users can choose their best collation for their databases
by setting `options[:collation]` based on their requirements.
* InnoDB FULLTEXT indexes support since MySQL 5.6
it does not have to use MyISAM storage engine whose maximum key length is 1000 bytes.
Using MyISAM storag engine with utf8mb4 character set would cause
"Specified key was too long; max key length is 1000 bytes"
https://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html
* References
"10.9.1 The utf8mb4 Character Set (4-Byte UTF-8 Unicode Encoding)"
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8mb4.html
"10.9.2 The utf8mb3 Character Set (3-Byte UTF-8 Unicode Encoding)"
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html
"14.8.1.7 Limits on InnoDB Tables"
https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
> If innodb_large_prefix is enabled (the default), the index key prefix limit is 3072 bytes
> for InnoDB tables that use DYNAMIC or COMPRESSED row format.
* CI against MySQL 5.7
Followed this instruction and changed root password to empty string.
https://docs.travis-ci.com/user/database-setup/#MySQL-57
* The recommended minimum version of MySQL is 5.7.9
to support utf8mb4 character set and `innodb_default_row_format`
MySQL 5.7.9 introduces `innodb_default_row_format` to support 3072 byte length index by default.
Users do not have to change MySQL database configuration to support Rails string type.
https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_default_row_format
https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
> If innodb_large_prefix is enabled (the default),
> the index key prefix limit is 3072 bytes for InnoDB tables that use DYNAMIC or COMPRESSED row format.
* The recommended minimum version of MariaDB is 10.2.2
MariaDB 10.2.2 is the first version of MariaDB supporting `innodb_default_row_format`
Also MariaDB says "MySQL 5.7 is compatible with MariaDB 10.2".
- innodb_default_row_format
https://mariadb.com/kb/en/library/xtradbinnodb-server-system-variables/#innodb_default_row_format
- "MariaDB versus MySQL - Compatibility"
https://mariadb.com/kb/en/library/mariadb-vs-mysql-compatibility/
> MySQL 5.7 is compatible with MariaDB 10.2
- "Supported Character Sets and Collations"
https://mariadb.com/kb/en/library/supported-character-sets-and-collations/
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
In order to avoid double assignments of nested_attributes for `has_many`
relations during record initialization, nested_attributes in `create_with`
should not be passed into `klass.new` and have them populate during
`initialize_internals_callback` with scope attributes.
However, `create_with` keys should always have precedence over where
clauses, so if there are same keys in both `create_with` and
`where_values_hash`, the value in `create_with` should be the one that's
used.
|
| |
| |
| |
| | |
Rather than a configuration on the connection.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
sensitive value of database column when call `#inspect`
* Why
Some sensitive data will be exposed in log accidentally by calling `#inspect`, e.g.
```ruby
@account = Account.find params[:id]
payload = { account: @account }
logger.info "payload will be #{ payload }"
```
All the information of `@account` will be exposed in log.
* Solution
Add a class attribute filter_attributes to specify which values of columns shouldn't be exposed.
This attribute equals to `Rails.application.config.filter_parameters` by default.
```ruby
Rails.application.config.filter_parameters += [:credit_card_number]
Account.last.insepct # => #<Account id: 123, credit_card_number: [FILTERED] ...>
```
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
`DatabaseLimits` and those methods were introduced at 3809c80, but most
methods were never used and never tested from the beginning (except
`table_alias_length`, `index_name_length`, and `in_clause_length` (since
66c09372)).
There is no reason to maintain unused those methods for about 8 years.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
After #33637 some tests in `activerecord/test/cases/tasks/database_tasks_test.rb`
don't assert anything.
We used to stub `ActiveRecord::Base::configurations` method in those
tests like `ActiveRecord::Base.stub(:configurations, @configurations) {}`.
Since #33637 `ActiveRecord::Base::configurations` is a `ActiveRecord::DatabaseConfigurations`
object(not a Hash object) we can't do so anymore.
`ActiveRecord::DatabaseConfigurations` object builds during `ActiveRecord::Base::configurations=`.
We can replace `ActiveRecord::Base.stub(:configurations, @configurations) {}` to
```
begin
old_configurations = ActiveRecord::Base.configurations
ActiveRecord::Base.configurations = @configurations
# ...
ensure
ActiveRecord::Base.configurations = old_configurations
end
```
Also I fixed tests in `activerecord/test/cases/tasks/legacy_database_tasks_test.rb`
But currently It looks like duplication of
`activerecord/test/cases/tasks/database_tasks_test.rb`.
We should improve those tests or remove them.
I've tried (in `activerecord/test/cases/tasks/legacy_database_tasks_test.rb` file):
```
def with_stubbed_configurations
old_configurations = ActiveRecord::Base.configurations.to_h
ActiveRecord::Base.configurations = @configurations
ActiveRecord::Base.stub(:configurations, ActiveRecord::Base.configurations.to_h) do
yield
end
ensure
ActiveRecord::Base.configurations = old_configurations
end
```
but it causes erros in tests cases.
After discussion we decided to remove
`activerecord/test/cases/tasks/legacy_database_tasks_test.rb`
Related to #33637
|
|\ \
| | |
| | |
| | |
| | | |
tgxworld/regression_setting_children_record_in_parent_before_save
Fix regression setting children record in parent before_save callback.
|
| | | |
|
|/ /
| |
| |
| | |
[Gannon McGibbon + Kenji Suzuki]
|
| |
| |
| |
| | |
Since #29860, `travel_back` automatically called at the end of the test.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This restores an ability that `update` with ids on a relation which is
described at https://github.com/rails/rails/issues/33470#issuecomment-411203013.
I personally think that the `update` with two arguments on a relation is
not a designed feature, since that is totally not using a relation
state, and also is not documented.
But removing any feature should not be suddenly happened in a stable
version even if that is not documented.
|
|\ \
| | |
| | | |
Refactor Active Record configurations
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
While the three-tier config makes it easier to define databases for
multiple database applications, it quickly became clear to offer full
support for multiple databases we need to change the way the connections
hash was handled.
A three-tier config means that when Rails needed to choose a default
configuration (in the case a user doesn't ask for a specific
configuration) it wasn't clear to Rails which the default was. I
[bandaid fixed this so the rake tasks could work](#32271) but that fix
wasn't correct because it actually doubled up the configuration hashes.
Instead of attemping to manipulate the hashes @tenderlove and I decided
that it made more sense if we converted the hashes to objects so we can
easily ask those object questions. In a three tier config like this:
```
development:
primary:
database: "my_primary_db"
animals:
database; "my_animals_db"
```
We end up with an object like this:
```
@configurations=[
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbded10
@env_name="development",@spec_name="primary",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>,
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbdea90
@env_name="development",@spec_name="animals",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>
]>
```
The configurations setter takes the database configuration set by your
application and turns them into an
`ActiveRecord::DatabaseConfigurations` object that has one getter -
`@configurations` which is an array of all the database objects.
The configurations getter returns this object by default since it acts
like a hash in most of the cases we need. For example if you need to
access the default `development` database we can simply request it as we
did before:
```
ActiveRecord::Base.configurations["development"]
```
This will return primary development database configuration hash:
```
{ "database" => "my_primary_db" }
```
Internally all of Active Record has been converted to use the new
objects. I've built this to be backwards compatible but allow for
accessing the hash if needed for a deprecation period. To get the
original hash instead of the object you can either add `to_h` on the
configurations call or pass `legacy: true` to `configurations.
```
ActiveRecord::Base.configurations.to_h
=> { "development => { "database" => "my_primary_db" } }
ActiveRecord::Base.configurations(legacy: true)
=> { "development => { "database" => "my_primary_db" } }
```
The new configurations object allows us to iterate over the Active
Record configurations without losing the known environment or
specification name for that configuration. You can also select all the
configs for an env or env and spec. With this we can always ask
any object what environment it belongs to:
```
db_configs = ActiveRecord::Base.configurations.configurations_for("development")
=> #<ActiveRecord::DatabaseConfigurations:0x00007fd1acbdf800
@configurations=[
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbded10
@env_name="development",@spec_name="primary",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>,
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbdea90
@env_name="development",@spec_name="animals",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>
]>
db_config.env_name
=> "development"
db_config.spec_name
=> "primary"
db_config.config
=> { "adapter"=>"sqlite3", "database"=>"db/development.sqlite3" }
```
The configurations object is more flexible than the configurations hash
and will allow us to build on top of the connection management in order
to add support for primary/replica connections, sharding, and
constructing queries for associations that live in multiple databases.
|
| | |
| | |
| | |
| | |
| | | |
Without this, `inverse_associations_test.rb` breaks when running in isolation.
https://travis-ci.org/rails/rails/jobs/422266840#L1894-L1899
|
|\ \ \
| |/ /
|/| | |
Find inverse associations with plural names
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Previously ActiveRecord couldn't find inverse associations if they were plural, which is a pretty standard use case. This commit changes the behavior to first attempt to find the singular version, then attempt to find the plural version. That makes it work and find plural associations as in the example below:
```
class Post
has_many :comments
end
class Comment
belongs_to :post
end
```
Previously the `:post` association reflection would only attempt to find a `comment` inverse, as opposed to both a `comment` and `comments` inverse.
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Issue #27852 reports that when `rails db:create` fails, it causes
leaking of user's DB credentials to $stderr.
We print a DB's configuration hash in order to help users more quickly
to figure out what could be wrong with his configuration.
This commit changes message from
"Couldn't create database for #{configuration.inspect}" to
"Couldn't create '#{configuration['database']}' database. Please check your configuration.".
There are two PRs that fixing it #27878, #27879, but they need a bit more work.
I decided help to finish this and added Author of those PRs credit in this commit.
Since it is a security issue, I think we should backport it to
`5-2-stable`, and `5-1-stable`.
Guided by https://edgeguides.rubyonrails.org/maintenance_policy.html#security-issues
Fixes #27852
Closes #27879
Related to #27878
[Alexander Marrs & bogdanvlviv]
|
|\ \ \
| | | |
| | | | |
Finish converting whitelist and blacklist references
|
| | | | |
|
| | | | |
|
| | | | |
|
| |/ /
|/| |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
`changes_applied` should be called before continuing around callback
chain. Otherwise the mutation tracker returns old value for methods like
`changed`? or `id_in_database` in around callbacks. Also methods depend
on `id_in_database`, like `update_column`, are not working in
`around_create` callbacks.
```
class Foo < ActiveRecord::Base
around_create :around_create_callback
def around_create_callback
...
yield
p id_in_database # => nil
update_column(:generated_column, generate_value) # silently fails
end
...
end
```
|
| | | |
|
|\ \ \
| |/ /
|/| | |
Add test case to test enum in has_many
|
| | |
| | |
| | |
| | |
| | |
| | | |
There is test in has_one to test enum, but there is no for has_many.
[Rich Chen]
|
|\ \ \
| |/ /
|/| | |
Prevent deadlocks when waiting for connection from pool.
|
| | |
| | |
| | |
| | |
| | | |
When a thread that had the load interlock but was blocked waiting to check a connection out of the connection pool but all of the threads using the available connections were blocked waiting to obtain the load interlock an `ActiveRecord::ConnectionTimeoutError` exception was be thrown by the thread waiting for the connection.
When waiting for the connection to check out we should allow loading to proceed to avoid this deadlock.
|
| | |
| | |
| | |
| | | |
`Topic` isn't loaded in this test file.
|
|\ \ \
| | | |
| | | | |
Omit BEGIN/COMMIT statements for empty transactions
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
If a transaction is opened and closed without any queries being run, we
can safely omit the `BEGIN` and `COMMIT` statements, as they only exist
to modify the connection's behaviour inside the transaction. This
removes the overhead of those statements when saving a record with no
changes, which makes workarounds like `save if changed?` unnecessary.
This implementation buffers transactions inside the transaction manager
and materializes them the next time the connection is used. For this to
work, the adapter needs to guard all connection use with a call to
`materialize_transactions`. Because of this, adapters must opt in to get
this new behaviour by implementing `supports_lazy_transactions?`.
If `raw_connection` is used to get a reference to the underlying
database connection, the behaviour is disabled and transactions are
opened eagerly, as we can't know how the connection will be used.
However when the connection is checked back into the pool, we can assume
that the application won't use the reference again and reenable lazy
transactions. This prevents a single `raw_connection` call from
disabling lazy transactions for the lifetime of the connection.
|