| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
| |
The indexing issue on `utf8mb4` columns is resolved since MySQL 5.7.9.
|
|
|
|
|
| |
Move `ActiveRecord::StatementInvalid` SQL to error property.
Also add bindings as an error property.
|
|
|
|
|
|
|
| |
MySQL 8.0.13 and higher supports functional key parts that index
expression values rather than column or column prefix values.
https://dev.mysql.com/doc/refman/8.0/en/create-index.html
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
when user has no parent table access privileges
Refer https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-13.html#mysqld-8-0-13-errors
>>
* Previously, the ER_NO_REFERENCED_ROW_2 and ER_ROW_IS_REFERENCED_2 error messages
for foreign key operations were displayed and revealed information about parent tables,
even when the user had no parent table access privileges. Error handling for this situation has been revised:
* If the user does have table-level privileges for all parent tables,
ER_NO_REFERENCED_ROW_2 and ER_ROW_IS_REFERENCED_2 are displayed, the same as before.
* If the user does not have table-level privileges for all parent tables,
more generic error messages are displayed instead (ER_NO_REFERENCED_ROW and ER_ROW_IS_REFERENCED).
<<
This pull request addresses these 3 failures:
```ruby
$ ARCONN=mysql2 bundle exec ruby -w -Itest test/cases/adapter_test.rb -n /foreign/
Using mysql2
Run options: -n /foreign/ --seed 14251
F
Failure:
ActiveRecord::AdapterForeignKeyTest#test_foreign_key_violations_are_translated_to_specific_exception_with_validate_false [test/cases/adapter_test.rb:348]:
[ActiveRecord::InvalidForeignKey] exception expected, not
Class: <ActiveRecord::StatementInvalid>
Message: <"Mysql2::Error: Cannot add or update a child row: a foreign key constraint fails: INSERT INTO `fk_test_has_fk` (`fk_id`) VALUES (1231231231)">
... snip ...
rails test test/cases/adapter_test.rb:343
F
Failure:
ActiveRecord::AdapterForeignKeyTest#test_foreign_key_violations_on_delete_are_translated_to_specific_exception [test/cases/adapter_test.rb:368]:
[ActiveRecord::InvalidForeignKey] exception expected, not
Class: <ActiveRecord::StatementInvalid>
Message: <"Mysql2::Error: Cannot delete or update a parent row: a foreign key constraint fails: DELETE FROM fk_test_has_pk WHERE pk_id = 1">
... snip ...
rails test test/cases/adapter_test.rb:365
F
Failure:
ActiveRecord::AdapterForeignKeyTest#test_foreign_key_violations_on_insert_are_translated_to_specific_exception [test/cases/adapter_test.rb:358]:
[ActiveRecord::InvalidForeignKey] exception expected, not
Class: <ActiveRecord::StatementInvalid>
Message: <"Mysql2::Error: Cannot add or update a child row: a foreign key constraint fails: INSERT INTO fk_test_has_fk (fk_id) VALUES (0)">
... snip ...
rails test test/cases/adapter_test.rb:357
Finished in 0.087370s, 34.3366 runs/s, 34.3366 assertions/s.
3 runs, 3 assertions, 3 failures, 0 errors, 0 skips
$
```
|
|
|
|
|
|
|
| |
I don't prefer to extract it for one adapter even though all adapters
also does.
Related to #34227.
|
|
|
|
|
|
|
|
|
| |
Will allow sub classes to override the protected
`#check_version` method hook if desired.
For example, this will be most helpful in sub classes that wish
to support lazy initialization because the version check can
be postponed until the connection is ready to be initialized.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In Ruby 2.3 or later, `String#+@` is available and `+@` is faster than `dup`.
```ruby
# frozen_string_literal: true
require "bundler/inline"
gemfile(true) do
source "https://rubygems.org"
gem "benchmark-ips"
end
Benchmark.ips do |x|
x.report('+@') { +"" }
x.report('dup') { "".dup }
x.compare!
end
```
```
$ ruby -v benchmark.rb
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
Warming up --------------------------------------
+@ 282.289k i/100ms
dup 187.638k i/100ms
Calculating -------------------------------------
+@ 6.775M (± 3.6%) i/s - 33.875M in 5.006253s
dup 3.320M (± 2.2%) i/s - 16.700M in 5.032125s
Comparison:
+@: 6775299.3 i/s
dup: 3320400.7 i/s - 2.04x slower
```
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* MySQL 5.1 does not support `utf8mb4` character set
* MySQL 5.1 had been already EOLed on Dec 2013
https://www.mysql.com/support/eol-notice.html
> Per Oracle's Lifetime Support policy, as of December 31, 2013, MySQL 5.1
> is covered under Oracle Sustaining Support.
* MySQL 5.5.8 is the first General Availability of MySQL 5.5
https://dev.mysql.com/doc/relnotes/mysql/5.5/en/news-5-5-8.html
|
|
|
|
| |
are not supported
|
|
|
|
|
|
|
|
|
|
|
| |
Once #33608 merged If users create a new database using MySQL 5.1.x, it will fail to create databases
since MySQL 5.1 does not know `utf8mb4` character set.
This pull request removes `encoding: utf8mb4` from `mysql.yml.tt`
to let create_database method handles default character set by MySQL server version.
`supports_longer_index_key_prefix?` method will need to validate if MySQL 5.5 and 5.6 server configured
correctly to support longer index key prefix, but not yet.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Use utf8mb4 character set by default
`utf8mb4` character set supports supplementary characters including emoji.
`utf8` character set with 3-Byte encoding is not enough to support them.
There was a downside of 4-Byte length character set with MySQL 5.5 and 5.6:
"ERROR 1071 (42000): Specified key was too long; max key length is 767 bytes"
for Rails string data type which is mapped to varchar(255) type.
MySQL 5.7 supports 3072 byte key prefix length by default.
* Remove `DEFAULT COLLATE` from Active Record unit test databases
There should be no "one size fits all" collation in MySQL 5.7.
Let MySQL server choose the default collation for Active Record
unit test databases.
Users can choose their best collation for their databases
by setting `options[:collation]` based on their requirements.
* InnoDB FULLTEXT indexes support since MySQL 5.6
it does not have to use MyISAM storage engine whose maximum key length is 1000 bytes.
Using MyISAM storag engine with utf8mb4 character set would cause
"Specified key was too long; max key length is 1000 bytes"
https://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html
* References
"10.9.1 The utf8mb4 Character Set (4-Byte UTF-8 Unicode Encoding)"
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8mb4.html
"10.9.2 The utf8mb3 Character Set (3-Byte UTF-8 Unicode Encoding)"
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html
"14.8.1.7 Limits on InnoDB Tables"
https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
> If innodb_large_prefix is enabled (the default), the index key prefix limit is 3072 bytes
> for InnoDB tables that use DYNAMIC or COMPRESSED row format.
* CI against MySQL 5.7
Followed this instruction and changed root password to empty string.
https://docs.travis-ci.com/user/database-setup/#MySQL-57
* The recommended minimum version of MySQL is 5.7.9
to support utf8mb4 character set and `innodb_default_row_format`
MySQL 5.7.9 introduces `innodb_default_row_format` to support 3072 byte length index by default.
Users do not have to change MySQL database configuration to support Rails string type.
https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_default_row_format
https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
> If innodb_large_prefix is enabled (the default),
> the index key prefix limit is 3072 bytes for InnoDB tables that use DYNAMIC or COMPRESSED row format.
* The recommended minimum version of MariaDB is 10.2.2
MariaDB 10.2.2 is the first version of MariaDB supporting `innodb_default_row_format`
Also MariaDB says "MySQL 5.7 is compatible with MariaDB 10.2".
- innodb_default_row_format
https://mariadb.com/kb/en/library/xtradbinnodb-server-system-variables/#innodb_default_row_format
- "MariaDB versus MySQL - Compatibility"
https://mariadb.com/kb/en/library/mariadb-vs-mysql-compatibility/
> MySQL 5.7 is compatible with MariaDB 10.2
- "Supported Character Sets and Collations"
https://mariadb.com/kb/en/library/supported-character-sets-and-collations/
|
|
|
|
| |
Rather than a configuration on the connection.
|
|\
| |
| | |
Omit BEGIN/COMMIT statements for empty transactions
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If a transaction is opened and closed without any queries being run, we
can safely omit the `BEGIN` and `COMMIT` statements, as they only exist
to modify the connection's behaviour inside the transaction. This
removes the overhead of those statements when saving a record with no
changes, which makes workarounds like `save if changed?` unnecessary.
This implementation buffers transactions inside the transaction manager
and materializes them the next time the connection is used. For this to
work, the adapter needs to guard all connection use with a call to
`materialize_transactions`. Because of this, adapters must opt in to get
this new behaviour by implementing `supports_lazy_transactions?`.
If `raw_connection` is used to get a reference to the underlying
database connection, the behaviour is disabled and transactions are
opened eagerly, as we can't know how the connection will be used.
However when the connection is checked back into the pool, we can assume
that the application won't use the reference again and reenable lazy
transactions. This prevents a single `raw_connection` call from
disabling lazy transactions for the lifetime of the connection.
|
|/
|
|
| |
https://github.com/rails/rails/issues/31190
|
|
|
|
| |
violations on delete
|
|
|
|
|
|
| |
#33363 has two regressions. First one is that `insert_fixtures_set` is
failed if flags is an array. Second one is that connection flags are not
restored if `set_server_option` is not supported.
|
| |
|
|
|
|
| |
There's no need to wrap the statement in a hash with a single key.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
to support Oracle database support identity data type
Oracle database does not support `INSERT .. DEFAULT VALUES`
then every insert statement needs at least one column name specified.
When `prefetch_primary_key?` returns `true` insert statement
always have the primary key name since the primary key value is selected
from the associated sequence. However, supporting identity data type
will make `prefetch_primary_key?` returns `false`
then no primary key column name added.
As a result, `empty_insert_statement_value` raises `NotImplementedError`
To address this error `empty_insert_statement_value` can take
one argument `primary_key` to generate insert statement like this.
`INSERT INTO "POSTS" ("ID") VALUES(DEFAULT)`
It needs arity change for the public method but no actual behavior changes for the bundled adapters.
Oracle enhanced adapter `empty_insert_statement_value` implementation will be like this:
```
def empty_insert_statement_value(primary_key)
raise NotImplementedError unless primary_key
"(#{quote_column_name(primary_key)}) VALUES(DEFAULT)"
end
```
[Raise NotImplementedError when using empty_insert_statement_value with Oracle](https://github.com/rails/rails/pull/28029)
[Add support for INSERT .. DEFAULT VALUES](https://community.oracle.com/ideas/13845)
|
|
|
|
|
|
|
| |
Prevent `ActiveRecord::FinderMethods#limited_ids_for` from using correct primary
key values even if `ORDER BY` columns include other table's primary key.
Fixes #28364.
|
|
|
|
|
| |
Some places we can't remove because Ruby still don't have a method
equivalent to strip_heredoc to be called in an already existent string.
|
| |
|
|
|
|
|
|
| |
Since #29504, mysql2 adapter lost ability to insert zero value on
primary key due to enforce `NO_AUTO_VALUE_ON_ZERO` disabled.
That is for using `DEFAULT` on auto increment column, but we can use
`NULL` instead in that case.
|
|\
| |
| | |
Build a multi-statement query when inserting fixtures
|
| |
| |
| |
| |
| | |
- mysql will add a 2 bytes margin to the statement, so given a `max_allowed_packet` set to 1024 bytes, a 1024 bytes fixtures will no be inserted (mysql will throw an error)
- Preventing this by decreasing the max_allowed_packet by 2 bytes when doing the comparison with the actual statement size
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- The `insert_fixtures` method can be optimized by making a single multi statement query for all fixtures having the same connection instead of doing a single query per table
- The previous code was bulk inserting fixtures for a single table, making X query for X fixture files
- This patch builds a single **multi statement query** for every tables. Given a set of 3 fixtures (authors, dogs, computers):
```ruby
# before
%w(authors dogs computers).each do |table|
sql = build_sql(table)
connection.query(sql)
end
# after
sql = build_sql(authors, dogs, computers)
connection.query(sql)
```
- `insert_fixtures` is now deprecated, `insert_fixtures_set` is the new way to go with performance improvement
- My tests were done with an app having more than 700 fixtures, the time it takes to insert all of them was around 15s. Using a single multi statement query, it took on average of 8 seconds
- In order for a multi statement to be executed, mysql needs to be connected with the `MULTI_STATEMENTS` [flag](https://dev.mysql.com/doc/refman/5.7/en/c-api-multiple-queries.html), which is done before inserting the fixtures by reconnecting to da the database with the flag declared. Reconnecting to the database creates some caveats:
1. We loose all open transactions; Inside the original code, when inserting fixtures, a transaction is open. Multple delete statements are [executed](https://github.com/rails/rails/blob/a681eaf22955734c142609961a6d71746cfa0583/activerecord/lib/active_record/fixtures.rb#L566) and finally the fixtures are inserted. The problem with this patch is that we need to open the transaction only after we reconnect to the DB otherwise reconnecting drops the open transaction which doesn't commit all delete statements and inserting fixtures doesn't work since we duplicated them (Primary key duplicate exception)...
- In order to fix this problem, the transaction is now open directly inside the `insert_fixtures` method, right after we reconnect to the db
- As an effect, since the transaction is open inside the `insert_fixtures` method, the DELETE statements need to be executed here since the transaction is open later
2. The same problem happens for the `disable_referential_integrity` since we reconnect, the `FOREIGN_KEY_CHECKS` is reset to the original value
- Same solution as 1. , the disable_referential_integrity can be called after we reconnect to the transaction
3. When the multi statement query is executed, no other queries can be performed until we paginate over the set of results, otherwise mysql throws a "Commands out of sync" [Ref](https://dev.mysql.com/doc/refman/5.7/en/commands-out-of-sync.html)
- Iterating over the set of results until `mysql_client.next_result` is false. [Ref](https://github.com/brianmario/mysql2#multiple-result-sets)
- Removed the `active_record.sql "Fixture delete"` notification, the delete statements are now inside the INSERT's one
- On mysql the `max_allowed_packet` is looked up:
1. Before executing the multi-statements query, we check the packet length of each statements, if the packet is bigger than the max_allowed_packet config, an `ActiveRecordError` is raised
2. Otherwise we concatenate the current sql statement into the previous and so on until the packet is `< max_allowed_packet`
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If `collation` is given without `charset`, it may generate invalid SQL.
For example `create_database(:matt_aimonetti, collation: "utf8mb4_bin")`:
```
> CREATE DATABASE `matt_aimonetti` DEFAULT CHARACTER SET `utf8` COLLATE `utf8mb4_bin`;
ERROR 1253 (42000): COLLATION 'utf8mb4_bin' is not valid for CHARACTER SET 'utf8'
```
In MySQL, charset is used to find the default collation. If `collation`
is given explicitly, it is not necessary to give extra charset.
|
| | |
|
| |
| |
| |
| | |
Follow up of #31177.
|
|\ \
| | |
| | |
| | |
| | | |
albertoalmagro/remove-default-mysql-engine-from-ar-5-2
Remove default ENGINE=InnoDB for Mysql2 adapter
|
| |/
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Before this commit ENGINE=InnoDB was added by default to Mysql2 adapter
+create_table+ if no +options+ option was provided. This default ENGINE
was lost as soon as something was passed in at +options+ option, making
its goal and propagation inconsistent, as the programmer needed to
remember including ENGINE=InnoDB when something was passed in.
This commit removes default ENGINE as its use isn't needed anymore for
current MySQL and MariaDB versions. It adds compatibility support and
tests to ensure that default ENGINE is still present for migrations
with version 5.1 and before. It also ensures we still dump the ENGINE
option to +schema.rb+ in order to avoid inconsistencies.
|
| |
| |
| |
| | |
Otherwise it will occur syntax error if primary key is a reserved word.
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use CONSTRAINT_SCHEMA key for information_schema.referential_constraints.
See https://dev.mysql.com/doc/refman/5.7/en/information-schema-optimization.html.
```
> EXPLAIN SELECT fk.referenced_table_name AS 'to_table', fk.referenced_column_name AS 'primary_key', fk.column_name AS 'column', fk.constraint_name AS 'name', rc.update_rule AS 'on_update', rc.delete_rule AS 'on_delete' FROM information_schema.referential_constraints rc JOIN information_schema.key_column_usage fk USING (constraint_schema, constraint_name) WHERE fk.referenced_column_name IS NOT NULL AND fk.table_schema = 'activerecord_unittest' AND fk.table_name = 'fk_test_has_pk' AND rc.constraint_schema = 'activerecord_unittest' AND rc.table_name = 'fk_test_has_pk'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: rc
partitions: NULL
type: ALL
possible_keys: NULL
key: CONSTRAINT_SCHEMA,TABLE_NAME
key_len: NULL
ref: NULL
rows: NULL
filtered: NULL
Extra: Using where; Open_full_table; Scanned 0 databases
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: fk
partitions: NULL
type: ALL
possible_keys: NULL
key: TABLE_SCHEMA,TABLE_NAME
key_len: NULL
ref: NULL
rows: NULL
filtered: NULL
Extra: Using where; Open_full_table; Scanned 0 databases; Using join buffer (Block Nested Loop)
2 rows in set, 1 warning (0.00 sec)
```
|
|
|
|
|
|
|
| |
`options_for_index_columns`
And placed `add_options_for_index_columns` in `schema_statements.rb`
consistently to ease to find related code.
|
| |
|
|\
| |
| | |
Extract sql fragment generators from PostgreSQL adapter
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
statement due to user request (#31235)
This changes `StatementTimeout` to `QueryCanceled` for PostgreSQL.
In MySQL, errno 1317 (`ER_QUERY_INTERRUPTED`) is only used when the
query is manually cancelled.
But in PostgreSQL, `QUERY_CANCELED` error code (57014) which is used
`StatementTimeout` is also used when the both case. And, we can not tell
which reason happened.
So I decided to introduce new error class `QueryCanceled` closer to the
error code name.
|
|/
|
|
|
|
| |
Since #31129, new error class `StatementTimeout` has been added.
`TransactionTimeout` is caused by the timeout shorter than
`StatementTimeout`, but its name is too generic. I think that it should
be a name that understands the difference with `StatementTimeout`.
|
|\
| |
| | |
Add :nodoc: to activerecord [ci skip]
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
timeout exceeded (#31129)
We are sometimes using The MAX_EXECUTION_TIME hint for MySQL depending
on the situation. It will prevent catastrophic performance down by wrong
performing queries.
The new error class `StatementTimeout` will make to be easier to handle
that case.
https://dev.mysql.com/doc/refman/5.7/en/optimizer-hints.html#optimizer-hints-execution-time
|
| |
| |
| |
| | |
This basically reverts 9d4f79d3d394edb74fa2192e5d9ad7b09ce50c6d
|
| | |
|
| |
| |
| |
| |
| |
| |
| | |
`change_column_{default,null,comment}` in mysql2 adapter are passing
`column.sql_type` as `type` to `change_column` to intend keeping
previous type. But `column_for` requires extra query, so use passing
`nil` to `type` explicitly in the internal for the purpose.
|
| | |
|