| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Since #23461, all adapters supports prepared statements, so that clears
the prepared statements cache is no longer database specific.
Actually, I struggled to identify the cause of random CI failure in
#23461, that was missing `@statements.clear` in `clear_cache!`.
This extracts `clear_cache!` to ensure the common concerns in the
abstract adapter.
|
|
|
|
|
| |
Adds a method to ActiveRecord allowing records to be inserted in bulk without instantiating ActiveRecord models. This method supports options for handling uniqueness violations by skipping duplicate records or overwriting them in an UPSERT operation.
ActiveRecord already supports bulk-update and bulk-destroy actions that execute SQL UPDATE and DELETE commands directly. It also supports bulk-read actions through `pluck`. It makes sense for it also to support bulk-creation.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In MySQL, the default collation is case insensitive. Since the
uniqueness validator enforces case sensitive comparison by default, it
frequently causes mismatched collation issues (performance, weird
behavior, etc) to MySQL users.
https://grosser.it/2009/12/11/validates_uniqness_of-mysql-slow/
https://github.com/rails/rails/issues/1399
https://github.com/rails/rails/pull/13465
https://github.com/gitlabhq/gitlabhq/commit/c1dddf8c7d947691729f6d64a8ea768b5c915855
https://github.com/huginn/huginn/pull/1330#discussion_r55152573
I'd like to deprecate the implicit default enforcing since I frequently
experienced the problems in code reviews.
Note that this change has no effect to sqlite3, postgresql, and
oracle-enhanced adapters which are implemented as case sensitive by
default, only affect to mysql2 adapter (I can take a work if sqlserver
adapter will support Rails 6.0).
|
|
|
|
|
| |
Also, improving an argument error message for `limit`, extracting around
`type_to_sql` code into schema statements, and more exercise tests.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The CI failure for `test_errors_for_bigint_fks_on_integer_pk_table` is
due to the poor regex that extract all ``` `(\w+)` ``` like parts from
the message (`:foreign_key` should be `"old_car_id"`, but `"engines"`):
https://travis-ci.org/rails/rails/jobs/494123455#L1703
I've improved the regex more strictly and have more exercised mismatched
foreign key tests.
Fixes #35294
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I've found a few places in Rails code base where I think it makes sense
to calculate elapsed time more precisely by using
`Concurrent.monotonic_time`:
- Fix calculation of elapsed time in `ActiveSupport::Cache::MemoryStore#prune`
- Fix calculation of elapsed time in
`ActiveRecord::ConnectionAdapters::ConnectionPool::Queue#wait_poll`
- Fix calculation of elapsed time in
`ActiveRecord::ConnectionAdapters::ConnectionPool#attempt_to_checkout_all_existing_connections`
- Fix calculation of elapsed time in `ActiveRecord::ConnectionAdapters::Mysql2Adapter#explain`
See
https://docs.ruby-lang.org/en/2.5.0/Process.html#method-c-clock_gettime
https://blog.dnsimple.com/2018/03/elapsed-time-with-ruby-the-right-way
Related to 7c4542146f0dde962205e5a90839349631ae60fb
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In MySQL, the text column size is 65,535 bytes by default (1 GiB in
PostgreSQL). It is sometimes too short when people want to use a text
column, so they sometimes change the text size to mediumtext (16 MiB) or
longtext (4 GiB) by giving the `limit` option.
Unlike MySQL, PostgreSQL doesn't allow the `limit` option for a text
column (raises ERROR: type modifier is not allowed for type "text").
So `limit: 4294967295` (longtext) couldn't be used in Action Text.
I've allowed changing text and blob size without giving the `limit`
option, it prevents that migration failure on PostgreSQL.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-14.html
> Error messages relating to creating and dropping foreign keys
> were improved to be more specific and informative. (Bug #28526309, Bug #92087)
https://dev.mysql.com/doc/refman/8.0/en/server-error-reference.html
> Error number: 3780; Symbol: ER_FK_INCOMPATIBLE_COLUMNS; SQLSTATE: HY000
> Message: Referencing column '%s' and referenced column '%s' in foreign key constraint '%s' are incompatible.
> ER_FK_INCOMPATIBLE_COLUMNS was added in 8.0.14.
|
|
|
|
| |
predicate construction
|
| |
|
|
|
|
| |
Since we already bumped the minimum version of MySQL to 5.5.8 at #33853.
|
|
|
|
| |
The indexing issue on `utf8mb4` columns is resolved since MySQL 5.7.9.
|
|
|
|
|
| |
Move `ActiveRecord::StatementInvalid` SQL to error property.
Also add bindings as an error property.
|
|
|
|
|
|
|
| |
MySQL 8.0.13 and higher supports functional key parts that index
expression values rather than column or column prefix values.
https://dev.mysql.com/doc/refman/8.0/en/create-index.html
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
when user has no parent table access privileges
Refer https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-13.html#mysqld-8-0-13-errors
>>
* Previously, the ER_NO_REFERENCED_ROW_2 and ER_ROW_IS_REFERENCED_2 error messages
for foreign key operations were displayed and revealed information about parent tables,
even when the user had no parent table access privileges. Error handling for this situation has been revised:
* If the user does have table-level privileges for all parent tables,
ER_NO_REFERENCED_ROW_2 and ER_ROW_IS_REFERENCED_2 are displayed, the same as before.
* If the user does not have table-level privileges for all parent tables,
more generic error messages are displayed instead (ER_NO_REFERENCED_ROW and ER_ROW_IS_REFERENCED).
<<
This pull request addresses these 3 failures:
```ruby
$ ARCONN=mysql2 bundle exec ruby -w -Itest test/cases/adapter_test.rb -n /foreign/
Using mysql2
Run options: -n /foreign/ --seed 14251
F
Failure:
ActiveRecord::AdapterForeignKeyTest#test_foreign_key_violations_are_translated_to_specific_exception_with_validate_false [test/cases/adapter_test.rb:348]:
[ActiveRecord::InvalidForeignKey] exception expected, not
Class: <ActiveRecord::StatementInvalid>
Message: <"Mysql2::Error: Cannot add or update a child row: a foreign key constraint fails: INSERT INTO `fk_test_has_fk` (`fk_id`) VALUES (1231231231)">
... snip ...
rails test test/cases/adapter_test.rb:343
F
Failure:
ActiveRecord::AdapterForeignKeyTest#test_foreign_key_violations_on_delete_are_translated_to_specific_exception [test/cases/adapter_test.rb:368]:
[ActiveRecord::InvalidForeignKey] exception expected, not
Class: <ActiveRecord::StatementInvalid>
Message: <"Mysql2::Error: Cannot delete or update a parent row: a foreign key constraint fails: DELETE FROM fk_test_has_pk WHERE pk_id = 1">
... snip ...
rails test test/cases/adapter_test.rb:365
F
Failure:
ActiveRecord::AdapterForeignKeyTest#test_foreign_key_violations_on_insert_are_translated_to_specific_exception [test/cases/adapter_test.rb:358]:
[ActiveRecord::InvalidForeignKey] exception expected, not
Class: <ActiveRecord::StatementInvalid>
Message: <"Mysql2::Error: Cannot add or update a child row: a foreign key constraint fails: INSERT INTO fk_test_has_fk (fk_id) VALUES (0)">
... snip ...
rails test test/cases/adapter_test.rb:357
Finished in 0.087370s, 34.3366 runs/s, 34.3366 assertions/s.
3 runs, 3 assertions, 3 failures, 0 errors, 0 skips
$
```
|
|
|
|
|
|
|
| |
I don't prefer to extract it for one adapter even though all adapters
also does.
Related to #34227.
|
|
|
|
|
|
|
|
|
| |
Will allow sub classes to override the protected
`#check_version` method hook if desired.
For example, this will be most helpful in sub classes that wish
to support lazy initialization because the version check can
be postponed until the connection is ready to be initialized.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In Ruby 2.3 or later, `String#+@` is available and `+@` is faster than `dup`.
```ruby
# frozen_string_literal: true
require "bundler/inline"
gemfile(true) do
source "https://rubygems.org"
gem "benchmark-ips"
end
Benchmark.ips do |x|
x.report('+@') { +"" }
x.report('dup') { "".dup }
x.compare!
end
```
```
$ ruby -v benchmark.rb
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
Warming up --------------------------------------
+@ 282.289k i/100ms
dup 187.638k i/100ms
Calculating -------------------------------------
+@ 6.775M (± 3.6%) i/s - 33.875M in 5.006253s
dup 3.320M (± 2.2%) i/s - 16.700M in 5.032125s
Comparison:
+@: 6775299.3 i/s
dup: 3320400.7 i/s - 2.04x slower
```
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* MySQL 5.1 does not support `utf8mb4` character set
* MySQL 5.1 had been already EOLed on Dec 2013
https://www.mysql.com/support/eol-notice.html
> Per Oracle's Lifetime Support policy, as of December 31, 2013, MySQL 5.1
> is covered under Oracle Sustaining Support.
* MySQL 5.5.8 is the first General Availability of MySQL 5.5
https://dev.mysql.com/doc/relnotes/mysql/5.5/en/news-5-5-8.html
|
|
|
|
| |
are not supported
|
|
|
|
|
|
|
|
|
|
|
| |
Once #33608 merged If users create a new database using MySQL 5.1.x, it will fail to create databases
since MySQL 5.1 does not know `utf8mb4` character set.
This pull request removes `encoding: utf8mb4` from `mysql.yml.tt`
to let create_database method handles default character set by MySQL server version.
`supports_longer_index_key_prefix?` method will need to validate if MySQL 5.5 and 5.6 server configured
correctly to support longer index key prefix, but not yet.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Use utf8mb4 character set by default
`utf8mb4` character set supports supplementary characters including emoji.
`utf8` character set with 3-Byte encoding is not enough to support them.
There was a downside of 4-Byte length character set with MySQL 5.5 and 5.6:
"ERROR 1071 (42000): Specified key was too long; max key length is 767 bytes"
for Rails string data type which is mapped to varchar(255) type.
MySQL 5.7 supports 3072 byte key prefix length by default.
* Remove `DEFAULT COLLATE` from Active Record unit test databases
There should be no "one size fits all" collation in MySQL 5.7.
Let MySQL server choose the default collation for Active Record
unit test databases.
Users can choose their best collation for their databases
by setting `options[:collation]` based on their requirements.
* InnoDB FULLTEXT indexes support since MySQL 5.6
it does not have to use MyISAM storage engine whose maximum key length is 1000 bytes.
Using MyISAM storag engine with utf8mb4 character set would cause
"Specified key was too long; max key length is 1000 bytes"
https://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html
* References
"10.9.1 The utf8mb4 Character Set (4-Byte UTF-8 Unicode Encoding)"
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8mb4.html
"10.9.2 The utf8mb3 Character Set (3-Byte UTF-8 Unicode Encoding)"
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html
"14.8.1.7 Limits on InnoDB Tables"
https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
> If innodb_large_prefix is enabled (the default), the index key prefix limit is 3072 bytes
> for InnoDB tables that use DYNAMIC or COMPRESSED row format.
* CI against MySQL 5.7
Followed this instruction and changed root password to empty string.
https://docs.travis-ci.com/user/database-setup/#MySQL-57
* The recommended minimum version of MySQL is 5.7.9
to support utf8mb4 character set and `innodb_default_row_format`
MySQL 5.7.9 introduces `innodb_default_row_format` to support 3072 byte length index by default.
Users do not have to change MySQL database configuration to support Rails string type.
https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_default_row_format
https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
> If innodb_large_prefix is enabled (the default),
> the index key prefix limit is 3072 bytes for InnoDB tables that use DYNAMIC or COMPRESSED row format.
* The recommended minimum version of MariaDB is 10.2.2
MariaDB 10.2.2 is the first version of MariaDB supporting `innodb_default_row_format`
Also MariaDB says "MySQL 5.7 is compatible with MariaDB 10.2".
- innodb_default_row_format
https://mariadb.com/kb/en/library/xtradbinnodb-server-system-variables/#innodb_default_row_format
- "MariaDB versus MySQL - Compatibility"
https://mariadb.com/kb/en/library/mariadb-vs-mysql-compatibility/
> MySQL 5.7 is compatible with MariaDB 10.2
- "Supported Character Sets and Collations"
https://mariadb.com/kb/en/library/supported-character-sets-and-collations/
|
|
|
|
| |
Rather than a configuration on the connection.
|
|\
| |
| | |
Omit BEGIN/COMMIT statements for empty transactions
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If a transaction is opened and closed without any queries being run, we
can safely omit the `BEGIN` and `COMMIT` statements, as they only exist
to modify the connection's behaviour inside the transaction. This
removes the overhead of those statements when saving a record with no
changes, which makes workarounds like `save if changed?` unnecessary.
This implementation buffers transactions inside the transaction manager
and materializes them the next time the connection is used. For this to
work, the adapter needs to guard all connection use with a call to
`materialize_transactions`. Because of this, adapters must opt in to get
this new behaviour by implementing `supports_lazy_transactions?`.
If `raw_connection` is used to get a reference to the underlying
database connection, the behaviour is disabled and transactions are
opened eagerly, as we can't know how the connection will be used.
However when the connection is checked back into the pool, we can assume
that the application won't use the reference again and reenable lazy
transactions. This prevents a single `raw_connection` call from
disabling lazy transactions for the lifetime of the connection.
|
|/
|
|
| |
https://github.com/rails/rails/issues/31190
|
|
|
|
| |
violations on delete
|
|
|
|
|
|
| |
#33363 has two regressions. First one is that `insert_fixtures_set` is
failed if flags is an array. Second one is that connection flags are not
restored if `set_server_option` is not supported.
|
| |
|
|
|
|
| |
There's no need to wrap the statement in a hash with a single key.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
to support Oracle database support identity data type
Oracle database does not support `INSERT .. DEFAULT VALUES`
then every insert statement needs at least one column name specified.
When `prefetch_primary_key?` returns `true` insert statement
always have the primary key name since the primary key value is selected
from the associated sequence. However, supporting identity data type
will make `prefetch_primary_key?` returns `false`
then no primary key column name added.
As a result, `empty_insert_statement_value` raises `NotImplementedError`
To address this error `empty_insert_statement_value` can take
one argument `primary_key` to generate insert statement like this.
`INSERT INTO "POSTS" ("ID") VALUES(DEFAULT)`
It needs arity change for the public method but no actual behavior changes for the bundled adapters.
Oracle enhanced adapter `empty_insert_statement_value` implementation will be like this:
```
def empty_insert_statement_value(primary_key)
raise NotImplementedError unless primary_key
"(#{quote_column_name(primary_key)}) VALUES(DEFAULT)"
end
```
[Raise NotImplementedError when using empty_insert_statement_value with Oracle](https://github.com/rails/rails/pull/28029)
[Add support for INSERT .. DEFAULT VALUES](https://community.oracle.com/ideas/13845)
|
|
|
|
|
|
|
| |
Prevent `ActiveRecord::FinderMethods#limited_ids_for` from using correct primary
key values even if `ORDER BY` columns include other table's primary key.
Fixes #28364.
|
|
|
|
|
| |
Some places we can't remove because Ruby still don't have a method
equivalent to strip_heredoc to be called in an already existent string.
|
| |
|
|
|
|
|
|
| |
Since #29504, mysql2 adapter lost ability to insert zero value on
primary key due to enforce `NO_AUTO_VALUE_ON_ZERO` disabled.
That is for using `DEFAULT` on auto increment column, but we can use
`NULL` instead in that case.
|
|\
| |
| | |
Build a multi-statement query when inserting fixtures
|
| |
| |
| |
| |
| | |
- mysql will add a 2 bytes margin to the statement, so given a `max_allowed_packet` set to 1024 bytes, a 1024 bytes fixtures will no be inserted (mysql will throw an error)
- Preventing this by decreasing the max_allowed_packet by 2 bytes when doing the comparison with the actual statement size
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- The `insert_fixtures` method can be optimized by making a single multi statement query for all fixtures having the same connection instead of doing a single query per table
- The previous code was bulk inserting fixtures for a single table, making X query for X fixture files
- This patch builds a single **multi statement query** for every tables. Given a set of 3 fixtures (authors, dogs, computers):
```ruby
# before
%w(authors dogs computers).each do |table|
sql = build_sql(table)
connection.query(sql)
end
# after
sql = build_sql(authors, dogs, computers)
connection.query(sql)
```
- `insert_fixtures` is now deprecated, `insert_fixtures_set` is the new way to go with performance improvement
- My tests were done with an app having more than 700 fixtures, the time it takes to insert all of them was around 15s. Using a single multi statement query, it took on average of 8 seconds
- In order for a multi statement to be executed, mysql needs to be connected with the `MULTI_STATEMENTS` [flag](https://dev.mysql.com/doc/refman/5.7/en/c-api-multiple-queries.html), which is done before inserting the fixtures by reconnecting to da the database with the flag declared. Reconnecting to the database creates some caveats:
1. We loose all open transactions; Inside the original code, when inserting fixtures, a transaction is open. Multple delete statements are [executed](https://github.com/rails/rails/blob/a681eaf22955734c142609961a6d71746cfa0583/activerecord/lib/active_record/fixtures.rb#L566) and finally the fixtures are inserted. The problem with this patch is that we need to open the transaction only after we reconnect to the DB otherwise reconnecting drops the open transaction which doesn't commit all delete statements and inserting fixtures doesn't work since we duplicated them (Primary key duplicate exception)...
- In order to fix this problem, the transaction is now open directly inside the `insert_fixtures` method, right after we reconnect to the db
- As an effect, since the transaction is open inside the `insert_fixtures` method, the DELETE statements need to be executed here since the transaction is open later
2. The same problem happens for the `disable_referential_integrity` since we reconnect, the `FOREIGN_KEY_CHECKS` is reset to the original value
- Same solution as 1. , the disable_referential_integrity can be called after we reconnect to the transaction
3. When the multi statement query is executed, no other queries can be performed until we paginate over the set of results, otherwise mysql throws a "Commands out of sync" [Ref](https://dev.mysql.com/doc/refman/5.7/en/commands-out-of-sync.html)
- Iterating over the set of results until `mysql_client.next_result` is false. [Ref](https://github.com/brianmario/mysql2#multiple-result-sets)
- Removed the `active_record.sql "Fixture delete"` notification, the delete statements are now inside the INSERT's one
- On mysql the `max_allowed_packet` is looked up:
1. Before executing the multi-statements query, we check the packet length of each statements, if the packet is bigger than the max_allowed_packet config, an `ActiveRecordError` is raised
2. Otherwise we concatenate the current sql statement into the previous and so on until the packet is `< max_allowed_packet`
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If `collation` is given without `charset`, it may generate invalid SQL.
For example `create_database(:matt_aimonetti, collation: "utf8mb4_bin")`:
```
> CREATE DATABASE `matt_aimonetti` DEFAULT CHARACTER SET `utf8` COLLATE `utf8mb4_bin`;
ERROR 1253 (42000): COLLATION 'utf8mb4_bin' is not valid for CHARACTER SET 'utf8'
```
In MySQL, charset is used to find the default collation. If `collation`
is given explicitly, it is not necessary to give extra charset.
|
| | |
|
| |
| |
| |
| | |
Follow up of #31177.
|
|\ \
| | |
| | |
| | |
| | | |
albertoalmagro/remove-default-mysql-engine-from-ar-5-2
Remove default ENGINE=InnoDB for Mysql2 adapter
|
| |/
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Before this commit ENGINE=InnoDB was added by default to Mysql2 adapter
+create_table+ if no +options+ option was provided. This default ENGINE
was lost as soon as something was passed in at +options+ option, making
its goal and propagation inconsistent, as the programmer needed to
remember including ENGINE=InnoDB when something was passed in.
This commit removes default ENGINE as its use isn't needed anymore for
current MySQL and MariaDB versions. It adds compatibility support and
tests to ensure that default ENGINE is still present for migrations
with version 5.1 and before. It also ensures we still dump the ENGINE
option to +schema.rb+ in order to avoid inconsistencies.
|
| |
| |
| |
| | |
Otherwise it will occur syntax error if primary key is a reserved word.
|