| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
| |
|
|\ |
|
| |
| |
| |
| |
| | |
It is for agnostic test case, since quoted table name may include `.`
for all adapters, and `[` / `]` for sqlserver adapter.
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
`enum` and `set` are typed cast as `:string`, but currently the
`:string` type is incorrectly reused for schema dumping.
A cast type on columns is not always the same with `sql_type`, this
fixes schema dumping `enum` and `set` columns to use `sql_type` instead
of `type` correctly.
|
|/
|
|
|
|
|
|
| |
When SQLite connects it will silently create a database if the database does not
exist. This behaviour causes different issues because of inconsistent behaviour
between adapters: #36383, #32914. This commit adds a `database_exists?` method
as a way to check the database without creating it. This is a stepping stone to
fully resolving the above issues.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If we put the `while_preventing_writes` on the connection then the
middleware that sends reads to the primary and ensures they can't write
will not work. The `while_preventing_writes` will only be applied to the
connection which it's called on - which in the case of the middleware is
Ar::Base.
This worked fine if you called it directly like
`OtherDbConn.connection.while_preventing_writes` but Rails didn't have a
way of knowing you wanted to call it on all the connections.
The change here moves the `while_preventing_writes` method from the
connection to the handler so that it can block writes to all queries for
that handler. This will apply to all the connections associated with
that handler.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR moves the `schema_migration` to `migration_context` so that we
can access the `schema_migration` per connection.
This does not change behavior of the SchemaMigration if you are using
one database. This also does not change behavior of any public APIs.
`Migrator` is private as is `MigrationContext` so we can change these as
needed.
We now need to pass a `schema_migration` to `Migrator` so that we can
run migrations on the right connection outside the context of a rake
task.
The bugs this fixes were discovered while debugging the issues around
the SchemaCache on initialization with multiple database. It was clear
that `get_all_versions` wouldn't work without these changes outside the
context of a rake task (because in the rake task we establish a
connection and change AR::Base.connection to the db we're running on).
Because the `SchemaCache` relies on the `SchemaMigration` information we
need to make sure we store it per-connection rather than on
ActiveRecord::Base.
[Eileen M. Uchitelle & Aaron Patterson]
|
|
|
|
|
|
|
|
|
|
|
| |
We sometimes say "✂️ newline after `private`" in a code review (e.g.
https://github.com/rails/rails/pull/18546#discussion_r23188776,
https://github.com/rails/rails/pull/34832#discussion_r244847195).
Now `Layout/EmptyLinesAroundAccessModifier` cop have new enforced style
`EnforcedStyle: only_before` (https://github.com/rubocop-hq/rubocop/pull/7059).
That cop and enforced style will reduce the our code review cost.
|
|
|
|
| |
And no longer need to except SCHEMA SQLs manually since 0810c07.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Running this migration on mysql at current master fails
because `add_references_for_alter` is missing.
```
change_table :users, bulk: true do |t|
t.references :article
end
```
This is also true for postgresql adapter,
but its `bulk_alter_table` implementation can fallback in such case.
postgresql's implementation is desirable to prevent unknown failure like this.
|
| |
|
|
|
|
|
| |
I've experienced this issue in our app, some hints only works on Top
level query (e.g. `MAX_EXECUTION_TIME`).
|
|\
| |
| | |
Cache database version in schema cache
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* The database version will get cached in the schema cache file during the
schema cache dump. When the database version check happens, the version will
be pulled from the schema cache and thus avoid querying the database for
the version.
* If the schema cache file doesn't exist, we'll query the database for the
version and cache it on the schema cache object.
* To facilitate this change, all connection adapters now implement
#get_database_version and #database_version. #database_version returns the
value from the schema cache.
* To take advantage of the cached database version, the database version check
will now happen after the schema cache is set on the connection in the
connection pool.
|
|/
|
|
|
|
|
|
| |
* s/Postgres/PostgreSQL/
* s/MYSQL/MySQL/, s/Mysql/MySQL/
* s/Sqlite/SQLite/
Replaced all newly added them after 6089b31.
|
|
|
|
|
|
|
| |
We have a test case for `collation_connection` session variable, so it
should not be changed in any other test.
Fixes #35458.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch has two main portions:
1. Add SQL comment support to Arel via Arel::Nodes::Comment.
2. Implement a Relation#annotate method on top of that.
== Adding SQL comment support
Adds a new Arel::Nodes::Comment node that represents an optional SQL
comment and teachers the relevant visitors how to handle it.
Comment nodes may be added to the basic CRUD statement nodes and set
through any of the four (Select|Insert|Update|Delete)Manager objects.
For example:
manager = Arel::UpdateManager.new
manager.table table
manager.comment("annotation")
manager.to_sql # UPDATE "users" /* annotation */
This new node type will be used by ActiveRecord::Relation to enable
query annotation via SQL comments.
== Implementing the Relation#annotate method
Implements `ActiveRecord::Relation#annotate`, which accepts a comment
string that will be appeneded to any queries generated by the relation.
Some examples:
relation = Post.where(id: 123).annotate("metadata string")
relation.first
# SELECT "posts".* FROM "posts" WHERE "posts"."id" = 123
# LIMIT 1 /* metadata string */
class Tag < ActiveRecord::Base
scope :foo_annotated, -> { annotate("foo") }
end
Tag.foo_annotated.annotate("bar").first
# SELECT "tags".* FROM "tags" LIMIT 1 /* foo */ /* bar */
Also wires up the plumbing so this works with `#update_all` and
`#delete_all` as well.
This feature is useful for instrumentation and general analysis of
queries generated at runtime.
|
| |
|
| |
|
|
|
|
| |
This is to easier make `truncate_tables` to bulk statements.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We as Arm Treasure Data are using Optimizer Hints with a monkey patch
(https://gist.github.com/kamipo/4c8539f0ce4acf85075cf5a6b0d9712e),
especially in order to use `MAX_EXECUTION_TIME` (refer #31129).
Example:
```ruby
class Job < ApplicationRecord
default_scope { optimizer_hints("MAX_EXECUTION_TIME(50000) NO_INDEX_MERGE(jobs)") }
end
```
Optimizer Hints is supported not only for MySQL but also for most
databases (PostgreSQL on RDS, Oracle, SQL Server, etc), it is really
helpful to turn heavy queries for large scale applications.
|
|
|
|
|
|
|
| |
* The READ_QUERY regex would consider reads to be writes if they started with
spaces or parens. For example, a UNION query might have parens around each
SELECT - (SELECT ...) UNION (SELECT ...).
* It will now correctly treat these queries as reads.
|
|\
| |
| |
| | |
Fix the regex that extract mismatched foreign key information
|
|/
|
|
|
|
|
|
|
|
|
|
|
| |
The CI failure for `test_errors_for_bigint_fks_on_integer_pk_table` is
due to the poor regex that extract all ``` `(\w+)` ``` like parts from
the message (`:foreign_key` should be `"old_car_id"`, but `"engines"`):
https://travis-ci.org/rails/rails/jobs/494123455#L1703
I've improved the regex more strictly and have more exercised mismatched
foreign key tests.
Fixes #35294
|
|
|
|
|
|
|
|
|
|
|
| |
Currently `conn.column_exists?("testings", "created_at", "datetime")`
returns false even if the table has the `created_at` column.
That reason is that `column.type` is a symbol but passed `type` is not
normalized to symbol unlike `column_name`, it is surprising behavior to
me.
I've improved that to normalize a value before comparison.
|
|\
| |
| | |
MySQL: `ROW_FORMAT=DYNAMIC` create table option by default
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Since MySQL 5.7.9, the `innodb_default_row_format` option defines the
default row format for InnoDB tables. The default setting is `DYNAMIC`.
The row format is required for indexing on `varchar(255)` with `utf8mb4`
columns.
As long as using MySQL 5.6, CI won't be passed even if MySQL server
setting is properly configured the same as MySQL 5.7
(`innodb_file_per_table = 1`, `innodb_file_format = 'Barracuda'`, and
`innodb_large_prefix = 1`) since InnoDB table is created as the row
format `COMPACT` by default on MySQL 5.6, therefore indexing on string
with `utf8mb4` columns aren't succeeded.
Making `ROW_FORMAT=DYNAMIC` create table option by default for legacy
MySQL version would mitigate the indexing issue on the user side, and it
makes CI would be passed on MySQL 5.6 which is configured properly.
|
|/
|
|
|
|
|
|
|
|
| |
Currently we sometimes find a redundant begin block in code review
(e.g. https://github.com/rails/rails/pull/33604#discussion_r209784205).
I'd like to enable `Style/RedundantBegin` cop to avoid that, since
rescue/else/ensure are allowed inside do/end blocks in Ruby 2.5
(https://bugs.ruby-lang.org/issues/12906), so we'd probably meets with
that situation than before.
|
|
|
|
|
|
|
| |
I originally named this `StatementInvalid` because that's what we do in
GitHub, but `@tenderlove` pointed out that this means apps can't test
for or explitly rescue this error. `StatementInvalid` is pretty broad so
I've renamed this to `ReadOnlyError`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR adds the ability to prevent writes to a database even if the
database user is able to write (ie the database is a primary and not a
replica).
This is useful for a few reasons: 1) when converting your database from
a single db to a primary/replica setup - you can fix all the writes on
reads early on, 2) when we implement automatic database switching or
when an app is manually switching connections this feature can be used
to ensure reads are reading and writes are writing. We want to make sure
we raise if we ever try to write in read mode, regardless of database
type and 3) for local development if you don't want to set up multiple
databases but do want to support rw/ro queries.
This should be used in conjunction with `connected_to` in write mode.
For example:
```
ActiveRecord::Base.connected_to(role: :writing) do
Dog.connection.while_preventing_writes do
Dog.create! # will raise because we're preventing writes
end
end
ActiveRecord::Base.connected_to(role: :reading) do
Dog.connection.while_preventing_writes do
Dog.first # will not raise because we're not writing
end
end
```
|
|
|
|
|
| |
Since #33875, Rails dropped supporting MySQL 5.1 which does not support
utf8mb4. We no longer need to use legacy utf8 (utf8mb3) conservatively.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Use utf8mb4 character set by default
`utf8mb4` character set supports supplementary characters including emoji.
`utf8` character set with 3-Byte encoding is not enough to support them.
There was a downside of 4-Byte length character set with MySQL 5.5 and 5.6:
"ERROR 1071 (42000): Specified key was too long; max key length is 767 bytes"
for Rails string data type which is mapped to varchar(255) type.
MySQL 5.7 supports 3072 byte key prefix length by default.
* Remove `DEFAULT COLLATE` from Active Record unit test databases
There should be no "one size fits all" collation in MySQL 5.7.
Let MySQL server choose the default collation for Active Record
unit test databases.
Users can choose their best collation for their databases
by setting `options[:collation]` based on their requirements.
* InnoDB FULLTEXT indexes support since MySQL 5.6
it does not have to use MyISAM storage engine whose maximum key length is 1000 bytes.
Using MyISAM storag engine with utf8mb4 character set would cause
"Specified key was too long; max key length is 1000 bytes"
https://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html
* References
"10.9.1 The utf8mb4 Character Set (4-Byte UTF-8 Unicode Encoding)"
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8mb4.html
"10.9.2 The utf8mb3 Character Set (3-Byte UTF-8 Unicode Encoding)"
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html
"14.8.1.7 Limits on InnoDB Tables"
https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
> If innodb_large_prefix is enabled (the default), the index key prefix limit is 3072 bytes
> for InnoDB tables that use DYNAMIC or COMPRESSED row format.
* CI against MySQL 5.7
Followed this instruction and changed root password to empty string.
https://docs.travis-ci.com/user/database-setup/#MySQL-57
* The recommended minimum version of MySQL is 5.7.9
to support utf8mb4 character set and `innodb_default_row_format`
MySQL 5.7.9 introduces `innodb_default_row_format` to support 3072 byte length index by default.
Users do not have to change MySQL database configuration to support Rails string type.
https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_default_row_format
https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
> If innodb_large_prefix is enabled (the default),
> the index key prefix limit is 3072 bytes for InnoDB tables that use DYNAMIC or COMPRESSED row format.
* The recommended minimum version of MariaDB is 10.2.2
MariaDB 10.2.2 is the first version of MariaDB supporting `innodb_default_row_format`
Also MariaDB says "MySQL 5.7 is compatible with MariaDB 10.2".
- innodb_default_row_format
https://mariadb.com/kb/en/library/xtradbinnodb-server-system-variables/#innodb_default_row_format
- "MariaDB versus MySQL - Compatibility"
https://mariadb.com/kb/en/library/mariadb-vs-mysql-compatibility/
> MySQL 5.7 is compatible with MariaDB 10.2
- "Supported Character Sets and Collations"
https://mariadb.com/kb/en/library/supported-character-sets-and-collations/
|
|
|
|
| |
Rather than a configuration on the connection.
|
|\
| |
| | |
Omit BEGIN/COMMIT statements for empty transactions
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If a transaction is opened and closed without any queries being run, we
can safely omit the `BEGIN` and `COMMIT` statements, as they only exist
to modify the connection's behaviour inside the transaction. This
removes the overhead of those statements when saving a record with no
changes, which makes workarounds like `save if changed?` unnecessary.
This implementation buffers transactions inside the transaction manager
and materializes them the next time the connection is used. For this to
work, the adapter needs to guard all connection use with a call to
`materialize_transactions`. Because of this, adapters must opt in to get
this new behaviour by implementing `supports_lazy_transactions?`.
If `raw_connection` is used to get a reference to the underlying
database connection, the behaviour is disabled and transactions are
opened eagerly, as we can't know how the connection will be used.
However when the connection is checked back into the pool, we can assume
that the application won't use the reference again and reenable lazy
transactions. This prevents a single `raw_connection` call from
disabling lazy transactions for the lifetime of the connection.
|
|/
|
|
| |
https://github.com/rails/rails/issues/31190
|
|
|
|
| |
A correct, but not obvious use of `ActiveSupport::Testing::MethodCallAssertions`, which might also have been part of #33337 or #33391.
|
|
|
|
| |
Missed these in preparing #33337
|
|
|
|
|
|
| |
Step 1 in #33162
[utilum + bogdanvlviv]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- MariaDB 10.3.7 is the first GA release
https://mariadb.com/kb/en/library/mariadb-1037-release-notes/
- MariaDB 10.3 translates `LENGTH()` to `OCTET_LENGTH()` function
https://mariadb.com/kb/en/library/sql_modeoracle-from-mariadb-103/
> MariaDB translates LENGTH() to OCTET_LENGTH()
- MySQL does NOT translate `LENGTH()` to `OCTET_LENGTH()`
However, it translates `OCTET_LENGTH()` to `LENGTH()`
Here are generated schema dumps of this test to show the differences
between MySQL and MariaDB:
* MySQL 8.0 (Server version: 8.0.11 MySQL Community Server - GPL)
```ruby
create_table \"virtual_columns\", options: \"ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci\", force: :cascade do |t|
t.string \"name\"
t.virtual \"upper_name\", type: :string, as: \"upper(`name`)\"
t.virtual \"name_length\", type: :integer, as: \"length(`name`)\", stored: true
t.virtual \"name_octet_length\", type: :integer, as: \"length(`name`)\", stored: true
end
```
* Maria DB 10.3 (Server version: 10.3.7-MariaDB-1:10.3.7+maria~bionic-log mariadb.org binary distribution)
```ruby
create_table \"virtual_columns\", options: \"ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci\", force: :cascade do |t|
t.string \"name\"
t.virtual \"upper_name\", type: :string, as: \"ucase(`name`)\"
t.virtual \"name_length\", type: :integer, as: \"octet_length(`name`)\", stored: true
t.virtual \"name_octet_length\", type: :integer, as: \"octet_length(`name`)\", stored: true
end
```
|
|
|
|
| |
Follow up of #32605.
|
|
|
|
|
| |
This autocorrects the violations after adding a custom cop in
3305c78dcd.
|
|
|
|
| |
Since Rails 6 requires Ruby 2.4.1+.
|
|
|
|
|
|
|
| |
Prevent `ActiveRecord::FinderMethods#limited_ids_for` from using correct primary
key values even if `ORDER BY` columns include other table's primary key.
Fixes #28364.
|
|
|
|
| |
Closes #31998
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rails has some support for multiple databases but it can be hard to
handle migrations with those. The easiest way to implement multiple
databases is to contain migrations into their own folder ("db/migrate"
for the primary db and "db/seconddb_migrate" for the second db). Without
this you would need to write code that allowed you to switch connections
in migrations. I can tell you from experience that is not a fun way to
implement multiple databases.
This refactoring is a pre-requisite for implementing other features
related to parallel testing and improved handling for multiple
databases.
The refactoring here moves the class methods from the `Migrator` class
into it's own new class `MigrationContext`. The goal was to move the
`migrations_paths` method off of the `Migrator` class and onto the
connection. This allows users to do the following in their
`database.yml`:
```
development:
adapter: mysql2
username: root
password:
development_seconddb:
adapter: mysql2
username: root
password:
migrations_paths: "db/second_db_migrate"
```
Migrations for the `seconddb` can now be store in the
`db/second_db_migrate` directory. Migrations for the primary database
are stored in `db/migrate`".
The refactoring here drastically reduces the internal API for migrations
since we don't need to pass `migrations_paths` around to every single
method. Additionally this change does not require any Rails applications
to make changes unless they want to use the new public API. All of the
class methods from the `Migrator` class were `nodoc`'d except for the
`migrations_paths` and `migrations_path` getter/setters respectively.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If `collation` is given without `charset`, it may generate invalid SQL.
For example `create_database(:matt_aimonetti, collation: "utf8mb4_bin")`:
```
> CREATE DATABASE `matt_aimonetti` DEFAULT CHARACTER SET `utf8` COLLATE `utf8mb4_bin`;
ERROR 1253 (42000): COLLATION 'utf8mb4_bin' is not valid for CHARACTER SET 'utf8'
```
In MySQL, charset is used to find the default collation. If `collation`
is given explicitly, it is not necessary to give extra charset.
|
|
|
|
| |
Follow up of #31177.
|