| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- The `insert_fixtures` method can be optimized by making a single multi statement query for all fixtures having the same connection instead of doing a single query per table
- The previous code was bulk inserting fixtures for a single table, making X query for X fixture files
- This patch builds a single **multi statement query** for every tables. Given a set of 3 fixtures (authors, dogs, computers):
```ruby
# before
%w(authors dogs computers).each do |table|
sql = build_sql(table)
connection.query(sql)
end
# after
sql = build_sql(authors, dogs, computers)
connection.query(sql)
```
- `insert_fixtures` is now deprecated, `insert_fixtures_set` is the new way to go with performance improvement
- My tests were done with an app having more than 700 fixtures, the time it takes to insert all of them was around 15s. Using a single multi statement query, it took on average of 8 seconds
- In order for a multi statement to be executed, mysql needs to be connected with the `MULTI_STATEMENTS` [flag](https://dev.mysql.com/doc/refman/5.7/en/c-api-multiple-queries.html), which is done before inserting the fixtures by reconnecting to da the database with the flag declared. Reconnecting to the database creates some caveats:
1. We loose all open transactions; Inside the original code, when inserting fixtures, a transaction is open. Multple delete statements are [executed](https://github.com/rails/rails/blob/a681eaf22955734c142609961a6d71746cfa0583/activerecord/lib/active_record/fixtures.rb#L566) and finally the fixtures are inserted. The problem with this patch is that we need to open the transaction only after we reconnect to the DB otherwise reconnecting drops the open transaction which doesn't commit all delete statements and inserting fixtures doesn't work since we duplicated them (Primary key duplicate exception)...
- In order to fix this problem, the transaction is now open directly inside the `insert_fixtures` method, right after we reconnect to the db
- As an effect, since the transaction is open inside the `insert_fixtures` method, the DELETE statements need to be executed here since the transaction is open later
2. The same problem happens for the `disable_referential_integrity` since we reconnect, the `FOREIGN_KEY_CHECKS` is reset to the original value
- Same solution as 1. , the disable_referential_integrity can be called after we reconnect to the transaction
3. When the multi statement query is executed, no other queries can be performed until we paginate over the set of results, otherwise mysql throws a "Commands out of sync" [Ref](https://dev.mysql.com/doc/refman/5.7/en/commands-out-of-sync.html)
- Iterating over the set of results until `mysql_client.next_result` is false. [Ref](https://github.com/brianmario/mysql2#multiple-result-sets)
- Removed the `active_record.sql "Fixture delete"` notification, the delete statements are now inside the INSERT's one
- On mysql the `max_allowed_packet` is looked up:
1. Before executing the multi-statements query, we check the packet length of each statements, if the packet is bigger than the max_allowed_packet config, an `ActiveRecordError` is raised
2. Otherwise we concatenate the current sql statement into the previous and so on until the packet is `< max_allowed_packet`
|
|\
| |
| | |
Quote colum_names when building select:
|
| |
| |
| |
| |
| |
| |
| |
| | |
- #30980 introcuded a change to not use `Arel.star` when model have ignored columns, a query used to look like `SELECT *. FROM developers` whereas now it would like `SELECT column1, column2 FROM developers`
- If a column has the same name has a reserved database specific keyword (such as key, where ...) then the query would fail because the names aren't quoted
- Quoting almost always happen unless we use a `from` clause in the query https://github.com/rails/rails/blob/9965b98dc0d58a86e10b4343bb6e15e01661a8c3/activerecord/lib/active_record/relation/query_methods.rb#L1052
- This PR cast all columns name to symbols in order for the quoting logic to be picked up https://github.com/rails/rails/blob/9965b98dc0d58a86e10b4343bb6e15e01661a8c3/activerecord/lib/active_record/relation/query_methods.rb#L1054-L1055
- A reproduction script can be found here https://gist.github.com/Edouard-chin/f56d464a0adcb76962afc1a9134a1536
|
|/
|
|
| |
Follow up of #31390.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`connection.primary_key` also return composite primary keys, so
`from_primary_key_column` may not be found even if `from_primary_key` is
presented.
```
% ARCONN=sqlite3 be ruby -w -Itest
test/cases/adapters/sqlite3/sqlite3_adapter_test.rb -n
test_copy_table_with_composite_primary_keys
Using sqlite3
Run options: -n test_copy_table_with_composite_primary_keys --seed 19041
# Running:
E
Error:
ActiveRecord::ConnectionAdapters::SQLite3AdapterTest#test_copy_table_with_composite_primary_keys:
NoMethodError: undefined method `type' for nil:NilClass
/path/to/rails/activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb:411:in
`block in copy_table'
```
This change fixes `copy_table` to do not lose composite primary keys.
|
|
|
|
|
|
|
|
|
|
|
|
| |
This regression was caused at 213796fb due to polymorphic predicates are
combined by `Arel::Nodes::And`. But I'd like to keep that combined
because it would help inverting polymorphic predicates correctly
(e9ba12f7), and we can collect equality nodes regardless of combined by
`Arel::Nodes::And` (`a AND (b AND c) AND d` == `a AND b AND c AND d`).
This change fixes the regression to collect equality nodes in
`Arel::Nodes::And` as well.
Fixes #31338.
|
|\
| |
| | |
Fix sqlite migrations with custom primary keys
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Previously, if a record was created with a custom primary key, that
table could not be migrated using sqlite. While attempting to copy the
table, the type of the primary key was ignored.
Once that was corrected, copying the indexes would fail because custom
primary keys are autoindexed by sqlite by default.
To correct that, this skips copying the index if the index name begins
with "sqlite_". This is a reserved word that indicates that the
index is an internal schema object. SQLite prohibits applications from
creating objects whose names begin with "sqlite_", so this string should
be safe to use as a check.
ref https://www.sqlite.org/fileformat2.html#intschema
|
| |
| |
| |
| |
| | |
`get_all_versions` doesn't use passed `connection`. So it should be
caught `NoDatabaseError` from `SchemaMigration.table_exists?`.
|
|\ \
| | |
| | |
| | |
| | | |
y-yagi/ignore_no_database_error_when_loading_schema_cache
Ignore `NoDatabaseError` when loading schema cache
|
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This is necessary in order to make the processing dependent on
`Migrator.current_version` work even without database.
Context: https://github.com/rails/rails/pull/31135#issuecomment-348404326
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This is a regression since Rails 4.2.
SQLite3 integer is stored in 1, 2, 3, 4, 6, or 8 bytes depending on the
magnitude of the value. Assuming default valid value as 4 bytes caused
that actual valid value in INTEGER storage class cannot be stored and
existing value cannot be found.
https://www.sqlite.org/datatype3.html
We should allow valid value in INTEGER storage class in SQLite3 to fix
the regression.
Fixes #22594.
|
| | |
| | |
| | |
| | |
| | | |
Actually SQLite3 doesn't have JSON storage class (so it is stored as a
TEXT like Date and Time). But emulating JSON types is convinient for
making database agnostic migrations.
|
|/ /
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If do not execute a type changing first, filling in default value may be
failed.
```
% ARCONN=postgresql be ruby -w -Itest test/cases/migration/compatibility_test.rb -n test_legacy_change_column_with_null_executes_update
Using postgresql
Run options: -n test_legacy_change_column_with_null_executes_update --seed 20459
E
Error:
ActiveRecord::Migration::CompatibilityTest#test_legacy_change_column_with_null_executes_update:
StandardError: An error has occurred, this and all later migrations canceled:
PG::StringDataRightTruncation: ERROR: value too long for type character varying(5)
: UPDATE "testings" SET "foo"='foobar' WHERE "foo" IS NULL
```
|
| |
| |
| |
| |
| |
| |
| | |
`options_for_index_columns`
And placed `add_options_for_index_columns` in `schema_statements.rb`
consistently to ease to find related code.
|
| | |
|
|\ \
| | |
| | | |
Extract sql fragment generators from PostgreSQL adapter
|
| | | |
|
| | | |
|
| |/
|/|
| |
| | |
Add validate_constraint and update naming
|
| | |
|
|\ \
| | |
| | |
| | |
| | | |
gregnavis/support-postgresql-operator-classes-in-indexes
Add support for PostgreSQL operator classes to add_index
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Add support for specifying non-default operator classes in PostgreSQL
indexes. An example CREATE INDEX query that becomes possible is:
CREATE INDEX users_name ON users USING gist (name gist_trgm_ops);
Previously it was possible to specify the `gist` index but not the
custom operator class. The `add_index` call for the above query is:
add_index :users, :name, using: :gist, opclasses: {name: :gist_trgm_ops}
|
| | |
| | |
| | |
| | |
| | | |
making changes (#31306)
It makes more sense than ignoring invalid IDs.
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
In 35836019, class level `update` and `destroy` suppressed
`RecordNotFound` to ensure returning affected objects. But
`RecordNotFound` is a common exception caught by a `rescue_from`
handler. So changing the behavior when a typical `params[:id]` is passed
has a compatibility problem. The previous behavior should not be
changed.
Fixes #31301.
|
|/ /
| |
| |
| |
| |
| | |
In #30510, `StatementPool` in `AbstractMysqlAdapter` was hidden in the
doc. But that class is also had in sqlite3 and postgresql adapters and
the base class is :nodoc: class.
|
|\ \
| | |
| | |
| | |
| | | |
chopraanmol1/bug_fix_has_one_inverse_owner_reload_from_validation
Inverse instance should not be reloaded during autosave if called in validation
|
| | |
| | |
| | |
| | |
| | | |
Record saved in save_has_one_association already make call to association.loaded! via record's before_save callback of save_belongs_to_association,
but this will reload object if accessed in record's validation.
|
|\ \ \
| | | |
| | | |
| | | | |
Scoping reserved names
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Due to inconsistent behavior when chaining scopes and one scope named after a Relation method
Validation code added in 2 places:
- scope, to prevent problematic scope names.
- enum, cause it tries to auto define scope.
|
|\ \ \ \
| | | | |
| | | | | |
[ci skip]Update docs `ActiveRecord::FinderMethods#find`
|
| |/ / /
| | | |
| | | |
| | | | |
ref https://github.com/rails/rails/pull/22653
|
|/ / / |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
There are two concerns which are both being combined into one here, but
both have the same goal. There are certain attributes which we want to
always consider initialized. Previously, they were handled separately.
The primary key (which is assumed to be backed by a database column)
needs to be initialized, because there is a ton of code in Active Record
that assumes `foo.id` will never raise. Additionally, we want attributes
which aren't backed by a database column to always be initialized, since
we would never receive a database value for them.
Ultimately these two concerns can be combined into one. The old
implementation hid a lot of inherent complexity, and is hard to optimize
from the outside. We can simplify things significantly by just passing
in a hash.
This has slightly different semantics from the old behavior, in that
`Foo.select(:bar).first.id` will return the default value for the
primary key, rather than `nil` unconditionally -- however, the default
value is always `nil` in practice.
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Since #31173, mysql2 adapter depends on `automatic_close` which is
introduced since mysql2 0.4.3. So the adapter with the mysql2 version
before doesn't work with fork now.
```
% ARCONN=mysql2 be ruby -w -Itest test/cases/connection_adapters/connection_handler_test.rb -n test_forked_child_doesnt_mangle_parent_connection
Using mysql2
Run options: -n test_forked_child_doesnt_mangle_parent_connection --seed 19988
/Users/kamipo/src/github.com/rails/rails/activerecord/lib/active_record/connection_adapters/mysql2_adapter.rb:108:in `discard!': undefined method `automatic_close=' for #<Mysql2::Client:0x00007fedaa91dfd0> (NoMethodError)
```
This drops mysql2 version less than 0.4.3 to guarantee fork safety.
|
| | | |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
statement due to user request (#31235)
This changes `StatementTimeout` to `QueryCanceled` for PostgreSQL.
In MySQL, errno 1317 (`ER_QUERY_INTERRUPTED`) is only used when the
query is manually cancelled.
But in PostgreSQL, `QUERY_CANCELED` error code (57014) which is used
`StatementTimeout` is also used when the both case. And, we can not tell
which reason happened.
So I decided to introduce new error class `QueryCanceled` closer to the
error code name.
|
| | |
| | |
| | |
| | |
| | |
| | | |
Since #31129, new error class `StatementTimeout` has been added.
`TransactionTimeout` is caused by the timeout shorter than
`StatementTimeout`, but its name is too generic. I think that it should
be a name that understands the difference with `StatementTimeout`.
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
rails/kamipo/ordinal_methods_should_respect_loaded_records"
This reverts commit 0f79ab91150b4cdb6c018530978a3395962c7a02, reversing
changes made to d575f7f2e737739302a0e8210d01c10f5d4e2c35.
This PR philosophically conflicts with #30800 and Matthew thinks we
should hold off merging this until we find concensus. Reverting since
we're about to cut a release for 5.2.
|
|\ \ \
| | | |
| | | | |
Flush idle database connections
|
| |/ / |
|
| |/
|/| |
|
|\ \
| | |
| | | |
Provide arguments to RecordNotFound
|
| | | |
|
|\ \ \
| | | |
| | | | |
Add :nodoc: to activerecord [ci skip]
|
| | | | |
|
|\ \ \ \
| | | | |
| | | | |
| | | | |
| | | | | |
rails/kamipo/ordinal_methods_should_respect_loaded_records
Ordinal methods should respect loaded records
|
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
We should reset partially loaded `@offsets` cache when latest records
has loaded because the cache has been staled and it may not be
consistent with latest records.
|
|\ \ \ \ \
| |_|_|_|/
|/| | | | |
Improve AR connection fork safety
|
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Use whatever adapter-provided means we have available to ensure forked
children don't send quit/shutdown/goodbye messages to the server on
connections that belonged to their parent.
|