| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
https://www.postgresql.org/support/versioning/
- 9.1 EOLed on September 2016.
- 9.2 EOLed on September 2017.
9.3 is also not supported since Nov 8, 2018. https://www.postgresql.org/about/news/1905/
I think it may be a little bit early to drop PostgreSQL 9.3 yet.
* Deprecated `supports_ranges?` since no other databases support range data type
* Add `supports_materialized_views?` to abstract adapter
Materialized views itself is supported by other databases, other connection adapters may support them
* Remove `with_manual_interventions`
It was only necessary for PostgreSQL 9.1 or earlier
* Drop CI against PostgreSQL 9.2
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before:
```
LOG: execute <unnamed>: SELECT t.oid, t.typname
FROM pg_type as t
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool')
LOG: execute <unnamed>: SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype
FROM pg_type as t
LEFT JOIN pg_range as r ON oid = rngtypid
WHERE
t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'interval', 'path', 'line', 'polygon', 'circle', 'lseg', 'box', 'time', 'timestamp', 'numeric')
OR t.typtype IN ('r', 'e', 'd')
OR t.typinput::varchar = 'array_in'
OR t.typelem != 0
LOG: statement: SHOW TIME ZONE
LOG: statement: SELECT 1
LOG: execute <unnamed>: SELECT COUNT(*)
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r','v','m') -- (r)elation/table, (v)iew, (m)aterialized view
AND c.relname = 'accounts'
AND n.nspname = ANY (current_schemas(false))
```
After:
```
LOG: execute <unnamed>: SELECT t.oid, t.typname
FROM pg_type as t
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool')
LOG: execute <unnamed>: SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype
FROM pg_type as t
LEFT JOIN pg_range as r ON oid = rngtypid
WHERE
t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'interval', 'path', 'line', 'polygon', 'circle', 'lseg', 'box', 'time', 'timestamp', 'numeric')
OR t.typtype IN ('r', 'e', 'd')
OR t.typinput::varchar = 'array_in'
OR t.typelem != 0
LOG: statement: SHOW TIME ZONE
LOG: statement: SELECT 1
LOG: execute <unnamed>: SELECT COUNT(*)
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r','v','m') -- (r)elation/table, (v)iew, (m)aterialized view
AND c.relname = 'accounts'
AND n.nspname = ANY (current_schemas(false))
```
|
|
|
|
|
|
|
|
| |
in indexdef to be wrapped up by double quotes
Fixes #34493.
*Thomas Bianchini*
|
|\
| |
| | |
Adjust bind length of SQLite to default (999)
|
| |
| |
| |
| |
| | |
Change `#bind_params_length` in SQLite adapter to return the default
maximum amount (999). See https://www.sqlite.org/limits.html
|
|/
|
|
|
|
|
|
|
|
|
| |
This commit adds support for the
`ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.create_unlogged_tables`
setting, which turns `CREATE TABLE` SQL statements into
`CREATE UNLOGGED TABLE` statements.
This can improve PostgreSQL performance but at the
cost of data durability, and thus it is highly recommended
that you *DO NOT* enable this in a production environment.
|
|
|
|
|
|
|
|
| |
Follow up a741208f80dd33420a56486bd9ed2b0b9862234a.
Since a741208, `Decimal#serialize` which is superclass of `Money` type
is no longer no-op, so it consistently serialize/deserialize a value as
a decimal even if schema default.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In Ruby 2.3 or later, `String#+@` is available and `+@` is faster than `dup`.
```ruby
# frozen_string_literal: true
require "bundler/inline"
gemfile(true) do
source "https://rubygems.org"
gem "benchmark-ips"
end
Benchmark.ips do |x|
x.report('+@') { +"" }
x.report('dup') { "".dup }
x.compare!
end
```
```
$ ruby -v benchmark.rb
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
Warming up --------------------------------------
+@ 282.289k i/100ms
dup 187.638k i/100ms
Calculating -------------------------------------
+@ 6.775M (± 3.6%) i/s - 33.875M in 5.006253s
dup 3.320M (± 2.2%) i/s - 16.700M in 5.032125s
Comparison:
+@: 6775299.3 i/s
dup: 3320400.7 i/s - 2.04x slower
```
|
|
|
|
|
| |
Since #33875, Rails dropped supporting MySQL 5.1 which does not support
utf8mb4. We no longer need to use legacy utf8 (utf8mb3) conservatively.
|
|
|
|
|
| |
Follow up #33874.
Related #23393.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Use utf8mb4 character set by default
`utf8mb4` character set supports supplementary characters including emoji.
`utf8` character set with 3-Byte encoding is not enough to support them.
There was a downside of 4-Byte length character set with MySQL 5.5 and 5.6:
"ERROR 1071 (42000): Specified key was too long; max key length is 767 bytes"
for Rails string data type which is mapped to varchar(255) type.
MySQL 5.7 supports 3072 byte key prefix length by default.
* Remove `DEFAULT COLLATE` from Active Record unit test databases
There should be no "one size fits all" collation in MySQL 5.7.
Let MySQL server choose the default collation for Active Record
unit test databases.
Users can choose their best collation for their databases
by setting `options[:collation]` based on their requirements.
* InnoDB FULLTEXT indexes support since MySQL 5.6
it does not have to use MyISAM storage engine whose maximum key length is 1000 bytes.
Using MyISAM storag engine with utf8mb4 character set would cause
"Specified key was too long; max key length is 1000 bytes"
https://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html
* References
"10.9.1 The utf8mb4 Character Set (4-Byte UTF-8 Unicode Encoding)"
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8mb4.html
"10.9.2 The utf8mb3 Character Set (3-Byte UTF-8 Unicode Encoding)"
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html
"14.8.1.7 Limits on InnoDB Tables"
https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
> If innodb_large_prefix is enabled (the default), the index key prefix limit is 3072 bytes
> for InnoDB tables that use DYNAMIC or COMPRESSED row format.
* CI against MySQL 5.7
Followed this instruction and changed root password to empty string.
https://docs.travis-ci.com/user/database-setup/#MySQL-57
* The recommended minimum version of MySQL is 5.7.9
to support utf8mb4 character set and `innodb_default_row_format`
MySQL 5.7.9 introduces `innodb_default_row_format` to support 3072 byte length index by default.
Users do not have to change MySQL database configuration to support Rails string type.
https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_default_row_format
https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
> If innodb_large_prefix is enabled (the default),
> the index key prefix limit is 3072 bytes for InnoDB tables that use DYNAMIC or COMPRESSED row format.
* The recommended minimum version of MariaDB is 10.2.2
MariaDB 10.2.2 is the first version of MariaDB supporting `innodb_default_row_format`
Also MariaDB says "MySQL 5.7 is compatible with MariaDB 10.2".
- innodb_default_row_format
https://mariadb.com/kb/en/library/xtradbinnodb-server-system-variables/#innodb_default_row_format
- "MariaDB versus MySQL - Compatibility"
https://mariadb.com/kb/en/library/mariadb-vs-mysql-compatibility/
> MySQL 5.7 is compatible with MariaDB 10.2
- "Supported Character Sets and Collations"
https://mariadb.com/kb/en/library/supported-character-sets-and-collations/
|
|
|
|
| |
Rather than a configuration on the connection.
|
|\
| |
| | |
Omit BEGIN/COMMIT statements for empty transactions
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If a transaction is opened and closed without any queries being run, we
can safely omit the `BEGIN` and `COMMIT` statements, as they only exist
to modify the connection's behaviour inside the transaction. This
removes the overhead of those statements when saving a record with no
changes, which makes workarounds like `save if changed?` unnecessary.
This implementation buffers transactions inside the transaction manager
and materializes them the next time the connection is used. For this to
work, the adapter needs to guard all connection use with a call to
`materialize_transactions`. Because of this, adapters must opt in to get
this new behaviour by implementing `supports_lazy_transactions?`.
If `raw_connection` is used to get a reference to the underlying
database connection, the behaviour is disabled and transactions are
opened eagerly, as we can't know how the connection will be used.
However when the connection is checked back into the pool, we can assume
that the application won't use the reference again and reenable lazy
transactions. This prevents a single `raw_connection` call from
disabling lazy transactions for the lifetime of the connection.
|
|/
|
|
| |
https://github.com/rails/rails/issues/31190
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* PostgreSQL 10 new relkind for partitioned tables
Starting with PostgreSQL 10, we can now have partitioned tables natively
* Add comment
* Remove extra space
* Add test for partition table in postgreSQL10
* Select 'p' for "BASE TABLE" and add a test case
to support PostgreSQL 10 partition tables
* Address RuboCop offense
* Addressed incorrect `postgresql_version`
Fixes #33008.
[Yannick Schutz & Yasuo Honda & Ryuta Kamizono]
|
|
|
|
| |
Follow up of #33358 for SQLite3.
|
|
|
|
| |
A correct, but not obvious use of `ActiveSupport::Testing::MethodCallAssertions`, which might also have been part of #33337 or #33391.
|
|
|
|
| |
Missed these in preparing #33337
|
|
|
|
|
|
| |
Step 1 in #33162
[utilum + bogdanvlviv]
|
|\
| |
| |
| | |
Support readonly option in SQLite3Adapter
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Readonly sqlite database files are very useful as a data format for
storing configuration/lookup data that is too complicated for YAML
files. But since such files would typically be committed to a source
control repository, it's important to ensure that they are truly safe
from being inadvertently modified. Unfortunately using unix permissions
isn't enough, as sqlite will "helpfully" add the write bit to a database
file whenever it's written to.
sqlite3-ruby has supported a `:readonly` option since version 1.3.2 (see
https://github.com/sparklemotion/sqlite3-ruby/commit/c20c9f5dd2990042)
This simply passes that option through to the adapter if present in the
config hash. I think this is best considered an adapter-specific option
since no other supported database has an identical concept.
|
|
|
|
| |
[Jon Moss & Xavier Noria]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- MariaDB 10.3.7 is the first GA release
https://mariadb.com/kb/en/library/mariadb-1037-release-notes/
- MariaDB 10.3 translates `LENGTH()` to `OCTET_LENGTH()` function
https://mariadb.com/kb/en/library/sql_modeoracle-from-mariadb-103/
> MariaDB translates LENGTH() to OCTET_LENGTH()
- MySQL does NOT translate `LENGTH()` to `OCTET_LENGTH()`
However, it translates `OCTET_LENGTH()` to `LENGTH()`
Here are generated schema dumps of this test to show the differences
between MySQL and MariaDB:
* MySQL 8.0 (Server version: 8.0.11 MySQL Community Server - GPL)
```ruby
create_table \"virtual_columns\", options: \"ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci\", force: :cascade do |t|
t.string \"name\"
t.virtual \"upper_name\", type: :string, as: \"upper(`name`)\"
t.virtual \"name_length\", type: :integer, as: \"length(`name`)\", stored: true
t.virtual \"name_octet_length\", type: :integer, as: \"length(`name`)\", stored: true
end
```
* Maria DB 10.3 (Server version: 10.3.7-MariaDB-1:10.3.7+maria~bionic-log mariadb.org binary distribution)
```ruby
create_table \"virtual_columns\", options: \"ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci\", force: :cascade do |t|
t.string \"name\"
t.virtual \"upper_name\", type: :string, as: \"ucase(`name`)\"
t.virtual \"name_length\", type: :integer, as: \"octet_length(`name`)\", stored: true
t.virtual \"name_octet_length\", type: :integer, as: \"octet_length(`name`)\", stored: true
end
```
|
|
|
|
|
|
|
|
| |
Since `parse_raw_value_as_a_number` may not always parse raw value from
database as a number without type casting (e.g. "$150.55" as money
format).
Fixes #32531.
|
|
|
|
|
|
|
|
|
|
|
| |
Since #26074, introduced force equality checking to build a predicate
consistently for both `find` and `create` (fixes #27313).
But the assumption that only array/range attribute have subtype was
wrong. We need to make force equality checking more strictly not to
allow serialized attribute.
Fixes #32761.
|
|
|
|
| |
Follow up of #32605.
|
|
|
|
|
| |
This autocorrects the violations after adding a custom cop in
3305c78dcd.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For legacy reasons Rails stores time columns on sqlite as full
timestamp strings. However because the date component wasn't being
normalized this meant that when they were read back they were being
prefixed with 2001-01-01 by ActiveModel::Type::Time. This had a
twofold result - first it meant that the fast code path wasn't being
used because the string was invalid and second it was corrupting the
second fractional component being read by the Date._parse code path.
Fix this by a combination of normalizing the timestamps on writing
and also changing Active Model to be more lenient when detecting
whether a string starts with a date component before creating the
dummy time value for parsing.
|
|
|
|
| |
Since Rails 6 requires Ruby 2.4.1+.
|
|
|
|
|
|
|
| |
Prevent `ActiveRecord::FinderMethods#limited_ids_for` from using correct primary
key values even if `ORDER BY` columns include other table's primary key.
Fixes #28364.
|
|
|
|
|
|
|
|
|
| |
BC dates are supported by both date and datetime types.
https://www.postgresql.org/docs/current/static/datatype-datetime.html
Since #1097, new datetime allows year zero as 1 BC, but new date does
not. It should be allowed even in new date consistently.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The values infinity and -infinity are supported by both date and
timestamp types.
https://www.postgresql.org/docs/current/static/datatype-datetime.html#DATATYPE-DATETIME-SPECIAL-TABLE
And also, it can not be known whether a value is infinity correctly
unless cast a value.
I've added `QueryAttribute#infinity?` to handle that case.
Closes #27585.
|
|
|
|
| |
Closes #31998
|
| |
|
| |
|
| |
|
| |
|
|\
| |
| | |
Build a multi-statement query when inserting fixtures
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- The `insert_fixtures` method can be optimized by making a single multi statement query for all fixtures having the same connection instead of doing a single query per table
- The previous code was bulk inserting fixtures for a single table, making X query for X fixture files
- This patch builds a single **multi statement query** for every tables. Given a set of 3 fixtures (authors, dogs, computers):
```ruby
# before
%w(authors dogs computers).each do |table|
sql = build_sql(table)
connection.query(sql)
end
# after
sql = build_sql(authors, dogs, computers)
connection.query(sql)
```
- `insert_fixtures` is now deprecated, `insert_fixtures_set` is the new way to go with performance improvement
- My tests were done with an app having more than 700 fixtures, the time it takes to insert all of them was around 15s. Using a single multi statement query, it took on average of 8 seconds
- In order for a multi statement to be executed, mysql needs to be connected with the `MULTI_STATEMENTS` [flag](https://dev.mysql.com/doc/refman/5.7/en/c-api-multiple-queries.html), which is done before inserting the fixtures by reconnecting to da the database with the flag declared. Reconnecting to the database creates some caveats:
1. We loose all open transactions; Inside the original code, when inserting fixtures, a transaction is open. Multple delete statements are [executed](https://github.com/rails/rails/blob/a681eaf22955734c142609961a6d71746cfa0583/activerecord/lib/active_record/fixtures.rb#L566) and finally the fixtures are inserted. The problem with this patch is that we need to open the transaction only after we reconnect to the DB otherwise reconnecting drops the open transaction which doesn't commit all delete statements and inserting fixtures doesn't work since we duplicated them (Primary key duplicate exception)...
- In order to fix this problem, the transaction is now open directly inside the `insert_fixtures` method, right after we reconnect to the db
- As an effect, since the transaction is open inside the `insert_fixtures` method, the DELETE statements need to be executed here since the transaction is open later
2. The same problem happens for the `disable_referential_integrity` since we reconnect, the `FOREIGN_KEY_CHECKS` is reset to the original value
- Same solution as 1. , the disable_referential_integrity can be called after we reconnect to the transaction
3. When the multi statement query is executed, no other queries can be performed until we paginate over the set of results, otherwise mysql throws a "Commands out of sync" [Ref](https://dev.mysql.com/doc/refman/5.7/en/commands-out-of-sync.html)
- Iterating over the set of results until `mysql_client.next_result` is false. [Ref](https://github.com/brianmario/mysql2#multiple-result-sets)
- Removed the `active_record.sql "Fixture delete"` notification, the delete statements are now inside the INSERT's one
- On mysql the `max_allowed_packet` is looked up:
1. Before executing the multi-statements query, we check the packet length of each statements, if the packet is bigger than the max_allowed_packet config, an `ActiveRecordError` is raised
2. Otherwise we concatenate the current sql statement into the previous and so on until the packet is `< max_allowed_packet`
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Rails has some support for multiple databases but it can be hard to
handle migrations with those. The easiest way to implement multiple
databases is to contain migrations into their own folder ("db/migrate"
for the primary db and "db/seconddb_migrate" for the second db). Without
this you would need to write code that allowed you to switch connections
in migrations. I can tell you from experience that is not a fun way to
implement multiple databases.
This refactoring is a pre-requisite for implementing other features
related to parallel testing and improved handling for multiple
databases.
The refactoring here moves the class methods from the `Migrator` class
into it's own new class `MigrationContext`. The goal was to move the
`migrations_paths` method off of the `Migrator` class and onto the
connection. This allows users to do the following in their
`database.yml`:
```
development:
adapter: mysql2
username: root
password:
development_seconddb:
adapter: mysql2
username: root
password:
migrations_paths: "db/second_db_migrate"
```
Migrations for the `seconddb` can now be store in the
`db/second_db_migrate` directory. Migrations for the primary database
are stored in `db/migrate`".
The refactoring here drastically reduces the internal API for migrations
since we don't need to pass `migrations_paths` around to every single
method. Additionally this change does not require any Rails applications
to make changes unless they want to use the new public API. All of the
class methods from the `Migrator` class were `nodoc`'d except for the
`migrations_paths` and `migrations_path` getter/setters respectively.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If `collation` is given without `charset`, it may generate invalid SQL.
For example `create_database(:matt_aimonetti, collation: "utf8mb4_bin")`:
```
> CREATE DATABASE `matt_aimonetti` DEFAULT CHARACTER SET `utf8` COLLATE `utf8mb4_bin`;
ERROR 1253 (42000): COLLATION 'utf8mb4_bin' is not valid for CHARACTER SET 'utf8'
```
In MySQL, charset is used to find the default collation. If `collation`
is given explicitly, it is not necessary to give extra charset.
|
| |
| |
| |
| |
| | |
This method which is used only in the internal was introduced in
ac384820 and was renamed in #17579. It does not need to be exposed.
|
| |
| |
| |
| |
| |
| |
| |
| | |
An empty string is an invalid value in Ruby's range class.
So need to handle `Float::INFINITY` as it is and cast it in
`encode_range`.
Fixes #31612
|
| | |
|
| | |
|
| |
| |
| |
| | |
Follow up of #31177.
|