| Commit message (Collapse) | Author | Age | Files | Lines |
... | |
| |
| |
| |
| |
| | |
This makes more sense, as the foreign key ignore pattern is only used by
the schema dumper.
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When dumping the database schema, Rails will dump foreign key names only
if those names were not generate by Rails. Currently this is determined
by checking if the foreign key name is `fk_rails_` followed by
a 10-character hash.
At [Cookpad](https://github.com/cookpad), we use
[Departure](https://github.com/departurerb/departure) (Percona's
pt-online-schema-change runner for ActiveRecord migrations) to run migrations.
Often, `pt-osc` will make a copy of a table in order to run a long migration
without blocking it. In this copy process, foreign keys are copied too,
but [their name is prefixed with an underscore to prevent name collision
](https://www.percona.com/doc/percona-toolkit/LATEST/pt-online-schema-change.html#cmdoption-pt-online-schema-change-alter-foreign-keys-method).
In the process described above, we often end up with a development
database that contains foreign keys which name starts with `_fk_rails_`.
That name does not match the ignore pattern, so next time Rails dumps
the database schema (eg. when running `rake db:migrate`), our
`db/schema.rb` file ends up containing those unwanted foreign key names.
This also produces an unwanted git diff that we'd prefer not to commit.
In this PR, I'd like to suggest a way to expose the foreign key name
ignore pattern to the Rails configuration, so that individual projects
can decide on a different pattern of foreign keys that will not get
their names dumped in `schema.rb`.
|
|
|
|
|
|
|
|
| |
In #24542, quoted_time was introduced to strip the leading date
component for time columns because it was having a significant
effect in mariadb. However, it assumed that the date component
was always 2000-01-01 which isn't the case, especially if the
source wasn't another time column.
|
|
|
|
|
| |
Ruby 2.4+ provides `Hash#compact` and `Hash#compact!` natively,
so `active_support/core_ext/hash/compact` is no longer necessary.
|
| |
|
|
|
|
| |
Duplicated method name list is no longer needed.
|
| |
|
|
|
|
| |
References 89bcca59e91fa9da941de890012872e8288e77b0
|
|
|
|
|
| |
Some places we can't remove because Ruby still don't have a method
equivalent to strip_heredoc to be called in an already existent string.
|
|
|
|
|
|
| |
Since #29504, mysql2 adapter lost ability to insert zero value on
primary key due to enforce `NO_AUTO_VALUE_ON_ZERO` disabled.
That is for using `DEFAULT` on auto increment column, but we can use
`NULL` instead in that case.
|
|
|
|
|
| |
MySQL supports descending indexes from 8.0.1 onwards:
https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-1.html
|
|\
| |
| | |
Build a multi-statement query when inserting fixtures
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- The `insert_fixtures` method can be optimized by making a single multi statement query for all fixtures having the same connection instead of doing a single query per table
- The previous code was bulk inserting fixtures for a single table, making X query for X fixture files
- This patch builds a single **multi statement query** for every tables. Given a set of 3 fixtures (authors, dogs, computers):
```ruby
# before
%w(authors dogs computers).each do |table|
sql = build_sql(table)
connection.query(sql)
end
# after
sql = build_sql(authors, dogs, computers)
connection.query(sql)
```
- `insert_fixtures` is now deprecated, `insert_fixtures_set` is the new way to go with performance improvement
- My tests were done with an app having more than 700 fixtures, the time it takes to insert all of them was around 15s. Using a single multi statement query, it took on average of 8 seconds
- In order for a multi statement to be executed, mysql needs to be connected with the `MULTI_STATEMENTS` [flag](https://dev.mysql.com/doc/refman/5.7/en/c-api-multiple-queries.html), which is done before inserting the fixtures by reconnecting to da the database with the flag declared. Reconnecting to the database creates some caveats:
1. We loose all open transactions; Inside the original code, when inserting fixtures, a transaction is open. Multple delete statements are [executed](https://github.com/rails/rails/blob/a681eaf22955734c142609961a6d71746cfa0583/activerecord/lib/active_record/fixtures.rb#L566) and finally the fixtures are inserted. The problem with this patch is that we need to open the transaction only after we reconnect to the DB otherwise reconnecting drops the open transaction which doesn't commit all delete statements and inserting fixtures doesn't work since we duplicated them (Primary key duplicate exception)...
- In order to fix this problem, the transaction is now open directly inside the `insert_fixtures` method, right after we reconnect to the db
- As an effect, since the transaction is open inside the `insert_fixtures` method, the DELETE statements need to be executed here since the transaction is open later
2. The same problem happens for the `disable_referential_integrity` since we reconnect, the `FOREIGN_KEY_CHECKS` is reset to the original value
- Same solution as 1. , the disable_referential_integrity can be called after we reconnect to the transaction
3. When the multi statement query is executed, no other queries can be performed until we paginate over the set of results, otherwise mysql throws a "Commands out of sync" [Ref](https://dev.mysql.com/doc/refman/5.7/en/commands-out-of-sync.html)
- Iterating over the set of results until `mysql_client.next_result` is false. [Ref](https://github.com/brianmario/mysql2#multiple-result-sets)
- Removed the `active_record.sql "Fixture delete"` notification, the delete statements are now inside the INSERT's one
- On mysql the `max_allowed_packet` is looked up:
1. Before executing the multi-statements query, we check the packet length of each statements, if the packet is bigger than the max_allowed_packet config, an `ActiveRecordError` is raised
2. Otherwise we concatenate the current sql statement into the previous and so on until the packet is `< max_allowed_packet`
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Rails has some support for multiple databases but it can be hard to
handle migrations with those. The easiest way to implement multiple
databases is to contain migrations into their own folder ("db/migrate"
for the primary db and "db/seconddb_migrate" for the second db). Without
this you would need to write code that allowed you to switch connections
in migrations. I can tell you from experience that is not a fun way to
implement multiple databases.
This refactoring is a pre-requisite for implementing other features
related to parallel testing and improved handling for multiple
databases.
The refactoring here moves the class methods from the `Migrator` class
into it's own new class `MigrationContext`. The goal was to move the
`migrations_paths` method off of the `Migrator` class and onto the
connection. This allows users to do the following in their
`database.yml`:
```
development:
adapter: mysql2
username: root
password:
development_seconddb:
adapter: mysql2
username: root
password:
migrations_paths: "db/second_db_migrate"
```
Migrations for the `seconddb` can now be store in the
`db/second_db_migrate` directory. Migrations for the primary database
are stored in `db/migrate`".
The refactoring here drastically reduces the internal API for migrations
since we don't need to pass `migrations_paths` around to every single
method. Additionally this change does not require any Rails applications
to make changes unless they want to use the new public API. All of the
class methods from the `Migrator` class were `nodoc`'d except for the
`migrations_paths` and `migrations_path` getter/setters respectively.
|
|\ \
| | |
| | |
| | | |
Remove dormant check
|
| | |
| | |
| | |
| | |
| | | |
This check was introduced in 6edaa26 and moved through multiple refactorings.
No test are broken after removal and AFAICS there is no way to trigger it.
|
| |/
|/|
| |
| | |
Follow up of #31177.
|
| |
| |
| |
| |
| | |
Actually SQLite3 doesn't have JSON storage class (so it is stored as a
TEXT like Date and Time). But emulating JSON types is convinient for
making database agnostic migrations.
|
| |
| |
| |
| |
| |
| |
| | |
`options_for_index_columns`
And placed `add_options_for_index_columns` in `schema_statements.rb`
consistently to ease to find related code.
|
| | |
|
|\ \
| | |
| | | |
Extract sql fragment generators from PostgreSQL adapter
|
| | | |
|
| | |
| | |
| | |
| | | |
Add validate_constraint and update naming
|
| | | |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Add support for specifying non-default operator classes in PostgreSQL
indexes. An example CREATE INDEX query that becomes possible is:
CREATE INDEX users_name ON users USING gist (name gist_trgm_ops);
Previously it was possible to specify the `gist` index but not the
custom operator class. The `add_index` call for the above query is:
add_index :users, :name, using: :gist, opclasses: {name: :gist_trgm_ops}
|
|\ \ \
| |/ /
|/| | |
Flush idle database connections
|
| | | |
|
|\ \ \
| |/ /
|/| | |
Improve AR connection fork safety
|
| | |
| | |
| | |
| | |
| | |
| | | |
Use whatever adapter-provided means we have available to ensure forked
children don't send quit/shutdown/goodbye messages to the server on
connections that belonged to their parent.
|
|\ \ \
| |/ /
|/| | |
Switch to LIFO for the connection pool
|
| | | |
|
| | | |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Using a FIFO for the connection pool can lead to issues when there are
upstream components (pgbouncer, haproxy, etc.) that terminate
connections that are idle after a period of time. Switching to a LIFO
reduces the probability that a thread will checkout a connection that is
about to be closed by an idle timeout in an upstream component.
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
```
[NoMethodError]: undefined method `state' for nil:NilClass Method:[rescue in block in refresh]
```
In `within_new_transaction`, there is the possibility that
`begin_transaction` returns a `nil`. (i.e.: so `transaction = nil`)
So this method is checking `transaction` for nil in 2 spots.
Unfortunately, there is one line that is not checking `transaction` for `nil`
That line, `commit_transaction`, throws an exception for us in AR 5.0.0.1
The problem with the method is finally realized in the error checking itself.
it calls `transaction.state` (i.e.: nil.state) and that is the final exception
raised.
The actual underlying (user) issue is hidden by this line.
Solution is test transaction for nil.
|
| | |
| | |
| | |
| | |
| | |
| | | |
Replace the primary key type `integer` in docs with `bigint`.
ref #26266
|
| | | |
|
| | |
| | |
| | |
| | | |
`initialize_internal_metadata_table`
|
| | | |
|
| | |
| | |
| | |
| | | |
object
|
| | |
| | |
| | |
| | | |
This basically reverts 9d4f79d3d394edb74fa2192e5d9ad7b09ce50c6d
|
| | |
| | |
| | |
| | |
| | | |
Implemented by #22911
Related to #30677
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
I got this error in production using Puma in multi-threaded mode:
```
RuntimeError: Digest::Base cannot be directly inherited in Ruby
from active_support/security_utils.rb:23:in `variable_size_secure_compare'
from active_support/security_utils.rb:23:in `hexdigest'
from active_support/security_utils.rb:23:in `digest'
```
Looks like Digest uses const_missing to load Digest::SHA256 (https://github.com/ruby/ruby/blob/trunk/ext/digest/lib/digest.rb#L8)
- https://bugs.ruby-lang.org/issues/9494
- https://github.com/ruby/ruby/commit/c02fa39463a0c6bf698b01bc610135604aca2ff4
|
| | | |
|
| | |
| | |
| | |
| | |
| | |
| | | |
Currently the normalization only exists in `primary_key` shorthand. It
should be moved to `new_column_definition` to also affect to
`add_column` with primary key.
|
| | | |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Otherwise using reserved words as composite primary key names will be
failed as an invalid SQL.
Fixes #30518.
|
| | |
| | |
| | |
| | |
| | |
| | | |
type columns
Fixes #30496.
|
| | |
| | |
| | |
| | | |
- Closes #30441
|
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Currently `SchemaDumper` is only customizable for column options. But
3rd party connection adapters (oracle-enhanced etc) need to customizable
for table or index dumping also. To make it possible, I introduced
adapter specific `SchemaDumper` classes for that.
|