| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| |\
| |
| |
| |
| | |
samphilipd/sam/manual_locking_on_schema_migrations
Make migrations concurrent safe (using advisory locks)
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- Addresses issue #22092
- Works on Postgres and MySQL
- Uses advisory locks because of two important properties:
1. The can be obtained outside of the context of a transaction
2. They are automatically released when the session ends, so if a
migration process crashed for whatever reason the lock is not left
open perpetually
- Adds get_advisory_lock and release_advisory_lock methods to database
adapters
- Attempting to run a migration while another one is in process will
raise a ConcurrentMigrationError instead of attempting to run in
parallel with undefined behavior. This could be rescued and
the migration could exit cleanly instead. Perhaps as a configuration
option?
Technical Notes
==============
The Migrator uses generate_migrator_advisory_lock_key to build the key
for the lock. In order to be compatible across multiple adapters there
are some constraints on this key.
- Postgres limits us to 64 bit signed integers
- MySQL advisory locks are server-wide so we have to scope to the
database
- To fulfil these requirements we use a Migrator salt (a randomly
chosen signed integer with max length of 31 bits) that identifies
the Rails migration process as the owner of the lock. We multiply
this salt with a CRC32 unsigned integer hash of the database name to
get a signed 64 bit integer that can also be converted to a string
to act as a lock key in MySQL databases.
- It is important for subsequent versions of the Migrator to use the
same salt, otherwise different versions of the Migrator will not see
each other's locks.
|
| |\ \
| | |
| | | |
PostgreSQL, Replace static connection param list by libpq's dynamic list
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
This makes the connection adapter future-proof regarding to new parameters.
To maintain backward compatibility, :requiressl is added by hand. It is
deprecated by PostgreSQL since 2003, but still accepted by libpq.
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
In Postgres 8.1 the standard_conforming_strings setting was read-only,
meaning you got an error if you tried to update it. By filtering on
`context = 'user'` we only try to update the setting if it's
user-writable[1].
[1]: http://www.postgresql.org/docs/9.4/static/view-pg-settings.html
|
| | |/
|/|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The standard_conforming_strings setting doesn't exist on all versions of
Postgres, but if it does exist, Rails turns it on. Previously this was done by
effectively disabling errors on the Postgres connection, issuing a SET to turn
the setting on, then re-enabling errors on the connection. However, if you're
running pgbouncer in transaction-pooling mode, you can't guarantee that
successive calls to `#execute` will be sent to the same pgbouncer-postgres
connection, so you can end up disabling errors on a different postgres
connection, and never re-enabling them. Future queries on that connection that
result in errors (e.g. violating unique constraints) will leave the connection
in a bad state where successive queries will fail.
This commit sets standard_conforming_strings by issuing an UPDATE to
pg_settings, which will update the setting if it exists, and do nothing if it
doesn't (rather than erroring out like SET would), which means we can remove
the error-disabling code.
It's also worth noting that Postgres has allowed standard_conforming_strings to
be updated since 8.2 (which is the oldest version Rails supports), so
technically we probably don't even need to be defensive here.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Prior to this commit, Rails makes no differentiation between whether a
query uses bind parameters, and whether or not we cache that query as a
prepared statement. This leads to the cache populating extremely fast in
some cases, with the statements never being reused.
In particular, the two problematic cases are `where(foo: [1, 2, 3])` and
`where("foo = ?", 1)`. In both cases we'll end up quoting the values
rather than using a bind param, causing a cache entry for every value
ever used in that query.
It was noted that we can probably eventually change `where("foo = ?",
1)` to use a bind param, which would resolve that case. Additionally, on
PG we can change our generated query to be `WHERE foo = ANY($1)`, and
pass an array for the bind param. I hope to accomplish both in the
future.
For SQLite and MySQL, we still end up preparing the statements anyway,
we just don't cache it. The statement will be cleaned up after it is
executed. On postgres, we skip the prepare step entirely, as an API is
provided to execute with bind params without preparing the statement.
I'm not 100% happy on the way this ended up being structured. I was
hoping to use a decorator on the visitor, rather than mixing a module
into the object, but the way Arel has it's visitor pattern set up makes
it very difficult to extend without inheritance. I'd like to remove the
duplication from the various places that are extending it, but that'll
require a larger restructuring of that initialization logic. I'm going
to take another look at the structure of it soon.
This changes the signature of one of the adapter's internals, and will
require downstream changes from third party adapters. I'm not too
worried about this, as worst case they can simply add the parameter and
always ignore it, and just keep their previous behavior.
Fixes #21992.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The focus of this change is to make the API more accessible.
References to method and classes should be linked to make it easy to
navigate around.
This patch makes exzessiv use of `rdoc-ref:` to provide more readable
docs. This makes it possible to document `ActiveRecord::Base#save` even
though the method is within a separate module
`ActiveRecord::Persistence`. The goal here is to bring the API closer to
the actual code that you would write.
This commit only deals with Active Record. The other gems will be
updated accordingly but in different commits. The pass through Active
Record is not completely finished yet. A follow up commit will change
the spots I haven't yet had the time to update.
/cc @fxn
|
| | |
| |
| |
| |
| | |
Current master branch includes many schema dumping improvements.
It extract these features to the appropriate files.
|
| | |
| |
| |
| | |
The returning_disabled configuration option is required to make postgresql partitioning triggers work. This commit fixes a bug where an invalid query would be made in cases where returning_disabled was true and objects were created with no attributes defined.
|
| | |
| |
| |
| |
| | |
`@connection` in `StatementPool` is only used for PG adapter.
No need for abstract `StatementPool` class.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Several changes were made in #21110 which I am strongly opposed to.
(this is what I get for going on vacation. :trollface:) No type should
be introduced into the generic `ActiveRecord::Type` namespace, and
*certainly* should not be registered into the registry unconstrained
unless it is supported by *all* adapters (which basically means that it
was specified in the ANSI SQL standard).
I do not think `# :nodoc:` ing the type is sufficient, as it still makes
the code of Rails itself very unclear as to what the role of that class
is. While I would argue that this shouldn't even be a super class, and
that MySql and PG's JSON types are only superficially duplicated (they
might look the same but will change for different reasons in the
future).
However, I don't feel strongly enough about it as a point of contention
(and the biggest cost of harming the blameability has already occured),
so I simply moved the superclass into a namespace where its role is
absolutely clear.
After this change, `attribute :foo, :json` will once again work with
MySQL and PG, but not with Sqlite3 or any third party adapters.
Unresolved questions
--------------------
The types that and adapter publishes (at least those are unique to that
adapter, and not adding additional behavior like `MysqlString` should
probably be part of the adapter's public API. Should we standardize the
namespace for these, and document them?
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
As of MySQL 5.7.8, MySQL supports a native JSON data type.
Example:
create_table :json_data_type do |t|
t.json :settings
end
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This introduces a deprecation cycle to change the behavior of the
default point type in the PostgreSQL adapter. The old behavior will
continue to be available for the immediate future as `:legacy_point`.
The current behavior of returning an `Array` causes several problems,
the most significant of which is that we cannot differentiate between an
array of points, and a point itself in the case of a column with the
`point[]` type.
The attributes API gives us a reasonable way to have a proper
deprecation cycle for this change, so let's take advantage of it. If we
like this change, we can also add proper support for the other geometric
types (line, lseg, box, path, polygon, and circle), all of which are
just aliases for string today.
Fixes #20441
|
| | | |
|
| | |
| |
| |
| |
| | |
This line introduced by the commit fd398475 for using
`Arel::Visitors::BindVisitor`. Currently it is not used.
|
| | | |
|
| | |
| |
| |
| |
| | |
`exec_query` create `ActiveRecord::Result` instance. It is better to use
`select_value` instead of `exec_query` for performance.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
Example:
create_table :foos do |t|
t.string :string_en, collation: 'en_US.UTF-8'
t.text :text_ja, collation: 'ja_JP.UTF-8'
end
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
https://github.com/ruby/ruby/pull/579 - there is a new optimization
since ruby 2.2
Previously regexp patterns were faster (since a string was converted to
regexp underneath anyway). But now string patterns are faster and
better reflect the purpose.
Benchmark.ips do |bm|
bm.report('regexp') { 'this is ::a random string'.gsub(/::/, '/') }
bm.report('string') { 'this is ::a random string'.gsub('::', '/') }
bm.compare!
end
# string: 753724.4 i/s
# regexp: 501443.1 i/s - 1.50x slower
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
We were never clearing the `PG::Result` object used to query the types
when the connection is first established. This would lead to a
potentially large amount of memory being retained for the life of the
connection.
Investigating this issue also revealed several low hanging fruit on the
performance of these methods, and the number of allocations has been
reduced by ~90%.
Fixes #19578
|
| | |
| |
| |
| | |
This obsoletes the ruby based implementations.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
Ruby-pg's default way to serialize values for transmission to the database is to
call #to_s . This however creates a temporary String object for each value.
Setting a class based type map avoids the allocation of this additional String.
The performance benefit is measurable in microbenchmarks, but not with
the overhead of activerecord. However it's free to use and has no drawback.
|
| |/
|
|
|
|
|
|
|
|
| |
The type map was introduced in aafee23, but wasn't properly filled.
This mainly adjusts many locations, that expected strings instead of
integers or boolean.
add_pg_decoders is moved after setup of the StatementPool, because
execute_and_clear could potentially make use of it.
|
| |
|
|
|
|
|
| |
Versions of the pg gem earlier than 0.18.0 cannot be used safely with Ruby 2.2.
Specifically, pg 0.17 when used with Ruby 2.2 has a known bug that causes
random bits to be added to the end of strings. Further explanation here:
https://bitbucket.org/ged/ruby-pg/issue/210/crazy-bytes-being-added-to-record
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
The cause by which the test suite for the mysql adapter broke in 1502cae
(reverted 89ba5bb) is because the precision was not extracted.
The rounding problem in mysql adapter has not been fixed, but `mysql_56`
helper tested only mysql2 adapter, its behavior was not apparent.
|
| |
|
|
|
|
|
|
|
|
| |
This reverts commit 1502caefd30b137fd1a0865be34c5bbf85ba64c1.
The test suite for the mysql adapter broke when this commit was used
with MySQL 5.6.
Conflicts:
activerecord/CHANGELOG.md
|
| |
|
|
|
|
| |
We do this in the adapter classes specifically, so the types aren't
registered if we don't use that adapter. Constants under the PostgreSQL
namespace for example are never loaded if we're using mysql.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
The latest version of the PG gem can actually convert the primitives for
us in C code, which gives a pretty substantial speed up. A few cases
were only there to add the `infinity` method, which I just put on the
range type (which is the only place it was used). Floats also needed to
parse `Infinity` and `NaN`, but it felt reasonable enough to put that on
the generic form.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This predicate was only to figure out if it's safe to do case
insensitive comparison, which is only a problem on PG. Turns out, PG can
just tell us whether we are able to do it or not. If the query turns out
to be a problem, let's just replace that method with checking the SQL
type for `text` or `character`. I'd rather not burden the type objects
with adapter specific knowledge.
The *real* solution, is to deprecate this behavior entirely. The only
reason we need it is because the `:case_sensitive` option for
`validates_uniqueness_of` is documented as "this option is ignored for
non-strings". It makes no sense for us to do that. If the type can't be
compared in a case insensitive way, the user shouldn't tell us to do
case insensitive comparison.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The type from the column is never used, except when being passed to the
attributes API. While leaving the type on the column wasn't necessarily
a bad thing, I worry that it's existence there implies that it is
something which should be used.
During the design and implementation process of the attributes API,
there have been plenty of cases where getting the "right" type object
was hard, but I had easy access to the column objects. For any
contributor who isn't intimately familiar with the intents behind the
type casting system, grabbing the type from the column might easily seem
like the "correct" thing to do.
As such, the goal of this change is to express that the column is not
something that should be used for type casting. The only places that are
"valid" (at the time of this commit) uses of acquiring a type object
from the column are fixtures (as the YAML file is going to mirror the
database more closely than the AR object), and looking up the type
during schema detection to pass to the attributes API
Many of the failing tests were removed, as they've been made obsolete
over the last year. All of the PG column tests were testing nothing
beyond polymorphism. The Mysql2 tests were duplicating the mysql tests,
since they now share a column class.
The implementation is a little hairy, and slightly verbose, but it felt
preferable to going back to 20 constructor options for the columns. If
you are git blaming to figure out wtf I was thinking with them, and have
a better idea, go for it. Just don't use a type object for this.
|
| |
|
|
|
|
|
|
| |
`bound_attributes` is now used universally across the board, removing
the need for the conversion layer. These changes are mostly mechanical,
with the exception of the log subscriber. Additional, we had to
implement `hash` on the attribute objects, so they could be used as a
key for query caching.
|
| | |
|
| |
|
|
|
|
| |
Slightly refactoring `PostgreSQLColumn`. `array` should be readonly.
`default_function` should be initialized by `super`. `sql_type` has been
removed `[]`. Since we already choose to remove it we should not change.
|
| |
|
|
|
| |
In most cases, `create_table_definition` called by table_name (the first
argument) only.
|
| |
|
|
| |
If it is not a default primary key, correctly dump the type and options.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
PG doesn't register it's types using the `int(4)` format that others do.
As such, if we alias `int8` to the other integer types, the range
information is lost. This is fixed by simply registering it separately.
The other option (which I specifically chose to avoid) is to pass the
information of the original type that was being aliased as an argument.
I'd rather avoid that, since an alias should truly be treated the same.
If we need different behavior for a different type, we should explicitly
register it with that, and not have a conditional based on aliasing.
Fixes #18144
[Sean Griffin & ysbaddaden]
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The type registration was simply looking for the OID, and eagerly
fetching/constructing the sub type when it was registered. However,
numeric types have additional parameters which are extracted from the
actual SQL string of the type during lookup, and can have their behavior
change based on the result.
We simply need to use the block form of registration, and look up the
subtype lazily instead.
Fixes #17935
|
| | |
|
| |
|
|
| |
Fixes: https://github.com/rails/rails/issues/17856.
|
| |
|
|
|
|
|
|
| |
I grepped the source code for code snippets wrapped in backticks in the comments
and replaced the backticks with plus signs so they are correctly displayed in
the Rails documentation.
[ci skip]
|
| |
|
|
| |
This reverts commit da99a2a2982d35f670ad9647463e09bfe9032b70.
|
| | |
|
| |
|
|
|
| |
also increase the version of pg required so that people will get the
GVL friendly version
|
