| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR proposes moving the schema cache from the connection to the pool
so the connection can ask the pool for the cache. In a future PR our
goal is to be able to read the yaml file from the pool so we can get
rid of the `active_record.check_schema_cache_dump` initializer. This
will fix the issues surrounding dumping the schema cache and mulitple
databases.
Why do we want to get rid of the initializer you ask?
Well I was looking at #34449 and trying to make it work for our usecase
and it revealed A LOT of problems. There are a few issues that I will
fix in remaining PRs with SchemaMigration, but there's a big glaring
issue with this initializer.
When you have an application with multiple databases we'll need to loop
through all the configurations and set the schema cache on those
connections. The problem is on initialization we only have one
connection - the one for Ar::Base. This is fine in a single db
application but not fine in multi-db. If we follow the pattern in #34449
and establish a connection to those other dbs we will end up setting the
cache on the _connection object_ rather than on all connections that
connect for that config.
So even though we looped through the configs and assigned the cache the
cache will not be set (or will be set wrong) once the app is booted
because the connection objects after boot are _different_ than the
connection objects we assigned the cache to.
After trying many different ways to set the schema cache `@tenderlove`
and I came to the conclusion that the initializer is problematic, as is
setting the schema cache twice.
This is part 1 to move the cache to the pool so the cache can read from
the schema cache yaml file instead of setting it when initializing the
app.
To do this we have created a `NullPool` that initializes an empty cache. I
put the `get_schema_cache` and `set_schema_cache` in an `AbstractPool`
so we can share code between `ConnectionPool` and `NullPool` instead of
duplicating code.
Now we only need to set the schema_cache on the pool rather than the
connection. In `discard!` we need to unset the connection from the
schema_cache - we still want the cache just not the connection.
|
|\
| |
| | |
Bring back postgresql_version as an alias.
|
| | |
|
|/ |
|
|
|
|
| |
Follow up of c9e4c848eeeb8999b778fa1ae52185ca5537fffe.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* The database version will get cached in the schema cache file during the
schema cache dump. When the database version check happens, the version will
be pulled from the schema cache and thus avoid querying the database for
the version.
* If the schema cache file doesn't exist, we'll query the database for the
version and cache it on the schema cache object.
* To facilitate this change, all connection adapters now implement
#get_database_version and #database_version. #database_version returns the
value from the schema cache.
* To take advantage of the cached database version, the database version check
will now happen after the schema cache is set on the connection in the
connection pool.
|
|
|
|
| |
This is to easier make `truncate_tables` to bulk statements.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We as Arm Treasure Data are using Optimizer Hints with a monkey patch
(https://gist.github.com/kamipo/4c8539f0ce4acf85075cf5a6b0d9712e),
especially in order to use `MAX_EXECUTION_TIME` (refer #31129).
Example:
```ruby
class Job < ApplicationRecord
default_scope { optimizer_hints("MAX_EXECUTION_TIME(50000) NO_INDEX_MERGE(jobs)") }
end
```
Optimizer Hints is supported not only for MySQL but also for most
databases (PostgreSQL on RDS, Oracle, SQL Server, etc), it is really
helpful to turn heavy queries for large scale applications.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Staled `@default_timezone` would cause an error on `reconnect!` after
`disconnect!`.
https://buildkite.com/rails/rails/builds/59495#23be8079-3a4f-4375-9991-0a6f874554f2
Steps to reproduce:
```
% ARCONN=postgresql bin/test test/cases/adapter_test.rb test/cases/base_test.rb -n "/(?:test_attributes_on_dummy_time|test_reconnect_after_a_disconnect)$/" --seed 15849
Using postgresql
Run options: -n "/(?:test_attributes_on_dummy_time|test_reconnect_after_a_disconnect)$/" --seed 15849
# Running:
.
E
Error:
ActiveRecord::AdapterTestWithoutTransaction#test_reconnect_after_a_disconnect:
NoMethodError: undefined method `add_coder' for #<PG::TypeMapAllStrings:0x00007f85ab9dd5b8>
/Users/kamipo/src/github.com/rails/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:866:in `update_typemap_for_default_timezone'
/Users/kamipo/src/github.com/rails/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:652:in `exec_no_cache'
/Users/kamipo/src/github.com/rails/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:636:in `execute_and_clear'
/Users/kamipo/src/github.com/rails/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:894:in `add_pg_decoders'
/Users/kamipo/src/github.com/rails/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:744:in `connect'
/Users/kamipo/src/github.com/rails/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:285:in `rescue in block in reconnect!'
/Users/kamipo/src/github.com/rails/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:281:in `block in reconnect!'
/Users/kamipo/.rbenv/versions/2.6.1/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/Users/kamipo/src/github.com/rails/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:280:in `reconnect!'
/Users/kamipo/src/github.com/rails/rails/activerecord/test/cases/adapter_test.rb:465:in `block in <class:AdapterTestWithoutTransaction>'
```
|
|
|
|
|
|
| |
It is to work that on `reconnect!` after `disconnect!`.
https://buildkite.com/rails/rails/builds/59378#1efea538-cfca-4d43-8b7e-ae78e97227c8
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Since #23461, all adapters supports prepared statements, so that clears
the prepared statements cache is no longer database specific.
Actually, I struggled to identify the cause of random CI failure in
#23461, that was missing `@statements.clear` in `clear_cache!`.
This extracts `clear_cache!` to ensure the common concerns in the
abstract adapter.
|
|
|
|
|
| |
Adds a method to ActiveRecord allowing records to be inserted in bulk without instantiating ActiveRecord models. This method supports options for handling uniqueness violations by skipping duplicate records or overwriting them in an UPSERT operation.
ActiveRecord already supports bulk-update and bulk-destroy actions that execute SQL UPDATE and DELETE commands directly. It also supports bulk-read actions through `pluck`. It makes sense for it also to support bulk-creation.
|
|
|
|
|
| |
This improves performance of timestamp conversion and avoids
additional string allocations.
|
|
|
|
| |
Follow up #34505.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
https://www.postgresql.org/support/versioning/
- 9.1 EOLed on September 2016.
- 9.2 EOLed on September 2017.
9.3 is also not supported since Nov 8, 2018. https://www.postgresql.org/about/news/1905/
I think it may be a little bit early to drop PostgreSQL 9.3 yet.
* Deprecated `supports_ranges?` since no other databases support range data type
* Add `supports_materialized_views?` to abstract adapter
Materialized views itself is supported by other databases, other connection adapters may support them
* Remove `with_manual_interventions`
It was only necessary for PostgreSQL 9.1 or earlier
* Drop CI against PostgreSQL 9.2
|
|\
| |
| | |
Redact SQL in errors
|
| |
| |
| |
| |
| | |
Move `ActiveRecord::StatementInvalid` SQL to error property.
Also add bindings as an error property.
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before:
```
LOG: execute <unnamed>: SELECT t.oid, t.typname
FROM pg_type as t
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool')
LOG: execute <unnamed>: SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype
FROM pg_type as t
LEFT JOIN pg_range as r ON oid = rngtypid
WHERE
t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'interval', 'path', 'line', 'polygon', 'circle', 'lseg', 'box', 'time', 'timestamp', 'numeric')
OR t.typtype IN ('r', 'e', 'd')
OR t.typinput::varchar = 'array_in'
OR t.typelem != 0
LOG: statement: SHOW TIME ZONE
LOG: statement: SELECT 1
LOG: execute <unnamed>: SELECT COUNT(*)
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r','v','m') -- (r)elation/table, (v)iew, (m)aterialized view
AND c.relname = 'accounts'
AND n.nspname = ANY (current_schemas(false))
```
After:
```
LOG: execute <unnamed>: SELECT t.oid, t.typname
FROM pg_type as t
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool')
LOG: execute <unnamed>: SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype
FROM pg_type as t
LEFT JOIN pg_range as r ON oid = rngtypid
WHERE
t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'interval', 'path', 'line', 'polygon', 'circle', 'lseg', 'box', 'time', 'timestamp', 'numeric')
OR t.typtype IN ('r', 'e', 'd')
OR t.typinput::varchar = 'array_in'
OR t.typelem != 0
LOG: statement: SHOW TIME ZONE
LOG: statement: SELECT 1
LOG: execute <unnamed>: SELECT COUNT(*)
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r','v','m') -- (r)elation/table, (v)iew, (m)aterialized view
AND c.relname = 'accounts'
AND n.nspname = ANY (current_schemas(false))
```
|
|
|
|
|
|
| |
This commit fixes a small typo in documentation of the
"UNLOGGED" table option for PostgreSQL databases, and
clarifies the documentation slightly.
|
|
|
|
|
|
|
|
|
|
|
| |
This commit adds support for the
`ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.create_unlogged_tables`
setting, which turns `CREATE TABLE` SQL statements into
`CREATE UNLOGGED TABLE` statements.
This can improve PostgreSQL performance but at the
cost of data durability, and thus it is highly recommended
that you *DO NOT* enable this in a production environment.
|
| |
|
|
|
|
|
|
|
| |
I don't prefer to extract it for one adapter even though all adapters
also does.
Related to #34227.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since Rails 6.0 will support Ruby 2.4.1 or higher
`# frozen_string_literal: true` magic comment is enough to make string object frozen.
This magic comment is enabled by `Style/FrozenStringLiteralComment` cop.
* Exclude these files not to auto correct false positive `Regexp#freeze`
- 'actionpack/lib/action_dispatch/journey/router/utils.rb'
- 'activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb'
It has been fixed by https://github.com/rubocop-hq/rubocop/pull/6333
Once the newer version of RuboCop released and available at Code Climate these exclude entries should be removed.
* Replace `String#freeze` with `String#-@` manually if explicit frozen string objects are required
- 'actionpack/test/controller/test_case_test.rb'
- 'activemodel/test/cases/type/string_test.rb'
- 'activesupport/lib/active_support/core_ext/string/strip.rb'
- 'activesupport/test/core_ext/string_ext_test.rb'
- 'railties/test/generators/actions_test.rb'
|
|\
| |
| | |
PostgreSQL: prepare for pg-1.1
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Version 1.1.0 deprecates exec and async_exec with a params array due to
distinct semantics between calls with and without params array.
Instead exec_params or async_exec_params shall be used.
Moreover in pg-1.1.0 exec_* and prepare methods are aliases for async_exec_*
and async_prepare. async_* methods don't need to be called explicit -
they are the default now, when calling exec_* and prepare.
For pg versions before 1.1, keep using async_exec.
|
| |
| |
| |
| | |
Rather than a configuration on the connection.
|
|\ \
| | |
| | | |
Omit BEGIN/COMMIT statements for empty transactions
|
| |/
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If a transaction is opened and closed without any queries being run, we
can safely omit the `BEGIN` and `COMMIT` statements, as they only exist
to modify the connection's behaviour inside the transaction. This
removes the overhead of those statements when saving a record with no
changes, which makes workarounds like `save if changed?` unnecessary.
This implementation buffers transactions inside the transaction manager
and materializes them the next time the connection is used. For this to
work, the adapter needs to guard all connection use with a call to
`materialize_transactions`. Because of this, adapters must opt in to get
this new behaviour by implementing `supports_lazy_transactions?`.
If `raw_connection` is used to get a reference to the underlying
database connection, the behaviour is disabled and transactions are
opened eagerly, as we can't know how the connection will be used.
However when the connection is checked back into the pool, we can assume
that the application won't use the reference again and reenable lazy
transactions. This prevents a single `raw_connection` call from
disabling lazy transactions for the lifetime of the connection.
|
|/
|
|
| |
https://github.com/rails/rails/issues/31190
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The values infinity and -infinity are supported by both date and
timestamp types.
https://www.postgresql.org/docs/current/static/datatype-datetime.html#DATATYPE-DATETIME-SPECIAL-TABLE
And also, it can not be known whether a value is infinity correctly
unless cast a value.
I've added `QueryAttribute#infinity?` to handle that case.
Closes #27585.
|
|
|
|
|
|
| |
It doesn't have to do anything, but it shouldn't fail.
Fixes #31766.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pg-1.0.0 is just released and most Gemfiles don't restrict
it's version. But the version is checked when connecting to
the database, which leads to the following error:
Gem::LoadError: can't activate pg (~> 0.18), already activated pg-1.0.0
See also this pg issue:
https://bitbucket.org/ged/ruby-pg/issues/270/pg-100-x64-mingw32-rails-server-not-start
Preparation for pg-1.0 was done in commit f28a331023fab,
but the pg version constraint was not yet relaxed.
|
| |
|
|
|
|
|
|
|
| |
`options_for_index_columns`
And placed `add_options_for_index_columns` in `schema_statements.rb`
consistently to ease to find related code.
|
|
|
|
| |
Add validate_constraint and update naming
|
|\
| |
| |
| |
| | |
gregnavis/support-postgresql-operator-classes-in-indexes
Add support for PostgreSQL operator classes to add_index
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Add support for specifying non-default operator classes in PostgreSQL
indexes. An example CREATE INDEX query that becomes possible is:
CREATE INDEX users_name ON users USING gist (name gist_trgm_ops);
Previously it was possible to specify the `gist` index but not the
custom operator class. The `add_index` call for the above query is:
add_index :users, :name, using: :gist, opclasses: {name: :gist_trgm_ops}
|
|/
|
|
|
|
| |
In #30510, `StatementPool` in `AbstractMysqlAdapter` was hidden in the
doc. But that class is also had in sqlite3 and postgresql adapters and
the base class is :nodoc: class.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
statement due to user request (#31235)
This changes `StatementTimeout` to `QueryCanceled` for PostgreSQL.
In MySQL, errno 1317 (`ER_QUERY_INTERRUPTED`) is only used when the
query is manually cancelled.
But in PostgreSQL, `QUERY_CANCELED` error code (57014) which is used
`StatementTimeout` is also used when the both case. And, we can not tell
which reason happened.
So I decided to introduce new error class `QueryCanceled` closer to the
error code name.
|
|
|
|
|
|
| |
Since #31129, new error class `StatementTimeout` has been added.
`TransactionTimeout` is caused by the timeout shorter than
`StatementTimeout`, but its name is too generic. I think that it should
be a name that understands the difference with `StatementTimeout`.
|
|
|
|
|
|
| |
Use whatever adapter-provided means we have available to ensure forked
children don't send quit/shutdown/goodbye messages to the server on
connections that belonged to their parent.
|
|
|
|
|
|
|
|
|
|
|
|
| |
timeout exceeded (#31129)
We are sometimes using The MAX_EXECUTION_TIME hint for MySQL depending
on the situation. It will prevent catastrophic performance down by wrong
performing queries.
The new error class `StatementTimeout` will make to be easier to handle
that case.
https://dev.mysql.com/doc/refman/5.7/en/optimizer-hints.html#optimizer-hints-execution-time
|
|
|
|
| |
Follow up of #30360.
|
|
|
|
|
|
|
|
| |
since the minimum version of PostgreSQL currently Rails supports is 9.1,
there is no need to handle if `supports_extensions?`
Refer https://www.postgresql.org/docs/9.1/static/sql-createextension.html
"CREATE EXTENSION"
|
|
|
|
| |
This basically reverts 9d4f79d3d394edb74fa2192e5d9ad7b09ce50c6d
|
|
|
|
|
|
|
|
| |
We already found the longer sequence name, but we could not consider
whether it was the sequence name created by serial type due to missed a
max identifier length limitation. I've addressed the sequence name
consideration to respect the max identifier length.
Fixes #28332.
|
| |
|
|
|
|
|
|
|
| |
Currently `SchemaDumper` is only customizable for column options. But
3rd party connection adapters (oracle-enhanced etc) need to customizable
for table or index dumping also. To make it possible, I introduced
adapter specific `SchemaDumper` classes for that.
|