| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
| |
Follow up #33874.
Related #23393.
|
| |
|
|\
| |
| | |
Omit BEGIN/COMMIT statements for empty transactions
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If a transaction is opened and closed without any queries being run, we
can safely omit the `BEGIN` and `COMMIT` statements, as they only exist
to modify the connection's behaviour inside the transaction. This
removes the overhead of those statements when saving a record with no
changes, which makes workarounds like `save if changed?` unnecessary.
This implementation buffers transactions inside the transaction manager
and materializes them the next time the connection is used. For this to
work, the adapter needs to guard all connection use with a call to
`materialize_transactions`. Because of this, adapters must opt in to get
this new behaviour by implementing `supports_lazy_transactions?`.
If `raw_connection` is used to get a reference to the underlying
database connection, the behaviour is disabled and transactions are
opened eagerly, as we can't know how the connection will be used.
However when the connection is checked back into the pool, we can assume
that the application won't use the reference again and reenable lazy
transactions. This prevents a single `raw_connection` call from
disabling lazy transactions for the lifetime of the connection.
|
| |
| |
| |
| | |
Fixes #33520.
|
|/
|
|
| |
Related to #33520
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Related #31201.
If creating custom primary key (like a string) in SQLite, it would also
create an internal index implicitly which named begin with "sqlite_".
It need to be hidden since the internal object names are reserved and
prohibited for public use.
See https://www.sqlite.org/fileformat2.html#intschema
Fixes #33320.
|
|\
| |
| |
| | |
Support readonly option in SQLite3Adapter
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Readonly sqlite database files are very useful as a data format for
storing configuration/lookup data that is too complicated for YAML
files. But since such files would typically be committed to a source
control repository, it's important to ensure that they are truly safe
from being inadvertently modified. Unfortunately using unix permissions
isn't enough, as sqlite will "helpfully" add the write bit to a database
file whenever it's written to.
sqlite3-ruby has supported a `:readonly` option since version 1.3.2 (see
https://github.com/sparklemotion/sqlite3-ruby/commit/c20c9f5dd2990042)
This simply passes that option through to the adapter if present in the
config hash. I think this is best considered an adapter-specific option
since no other supported database has an identical concept.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Unlike other databases, changing SQLite3 table definitions need to create a temporary table.
While changing table operations, the original table needs dropped which caused
`SQLite3::ConstraintException: FOREIGN KEY constraint failed` if the table is referenced by foreign keys.
This pull request disables foreign keys by `disable_referential_integrity`.
Also `disable_referential_integrity` method needs to execute `defer_foreign_keys = ON`
to defer re-enabling foreign keys until the transaction is committed.
https://www.sqlite.org/pragma.html#pragma_defer_foreign_keys
Fixes #31988
- This `defer_foreign_keys = ON` has been supported since SQLite 3.8.0
https://www.sqlite.org/releaselog/3_8_0.html and Rails 6 requires SQLite 3.8 #32923 now
- <Models>.reset_column_information added to address `ActiveModel::UnknownAttributeError`
```
Error:
ActiveRecord::Migration::ForeignKeyChangeColumnTest#test_change_column_of_parent_table:
ActiveModel::UnknownAttributeError: unknown attribute 'name' for ActiveRecord::Migration::ForeignKeyChangeColumnTest::Post.
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
These OS versions have SQLite 3.8 or higher by default.
- macOS 10.10 (Yosemite) or higher
- Ubuntu 14.04 LTS or higher
Raising the minimum version of SQLite 3.8 introduces these changes:
- All of bundled adapters support `supports_multi_insert?`
- SQLite 3.8 always satisifies `supports_foreign_keys_in_create?` and `supports_partial_index?`
- sqlite adapter can support `alter_table` method for foreign key referenced tables by #32865
- Deprecated `supports_multi_insert?` method
|
|
|
|
| |
https://github.com/sparklemotion/sqlite3-ruby/blob/v1.3.13/lib/sqlite3/statement.rb#L101-L104
|
|
|
|
| |
Since #31422, `insert_fixtures` is deprecated.
|
|\
| |
| | |
Build a multi-statement query when inserting fixtures
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- The `insert_fixtures` method can be optimized by making a single multi statement query for all fixtures having the same connection instead of doing a single query per table
- The previous code was bulk inserting fixtures for a single table, making X query for X fixture files
- This patch builds a single **multi statement query** for every tables. Given a set of 3 fixtures (authors, dogs, computers):
```ruby
# before
%w(authors dogs computers).each do |table|
sql = build_sql(table)
connection.query(sql)
end
# after
sql = build_sql(authors, dogs, computers)
connection.query(sql)
```
- `insert_fixtures` is now deprecated, `insert_fixtures_set` is the new way to go with performance improvement
- My tests were done with an app having more than 700 fixtures, the time it takes to insert all of them was around 15s. Using a single multi statement query, it took on average of 8 seconds
- In order for a multi statement to be executed, mysql needs to be connected with the `MULTI_STATEMENTS` [flag](https://dev.mysql.com/doc/refman/5.7/en/c-api-multiple-queries.html), which is done before inserting the fixtures by reconnecting to da the database with the flag declared. Reconnecting to the database creates some caveats:
1. We loose all open transactions; Inside the original code, when inserting fixtures, a transaction is open. Multple delete statements are [executed](https://github.com/rails/rails/blob/a681eaf22955734c142609961a6d71746cfa0583/activerecord/lib/active_record/fixtures.rb#L566) and finally the fixtures are inserted. The problem with this patch is that we need to open the transaction only after we reconnect to the DB otherwise reconnecting drops the open transaction which doesn't commit all delete statements and inserting fixtures doesn't work since we duplicated them (Primary key duplicate exception)...
- In order to fix this problem, the transaction is now open directly inside the `insert_fixtures` method, right after we reconnect to the db
- As an effect, since the transaction is open inside the `insert_fixtures` method, the DELETE statements need to be executed here since the transaction is open later
2. The same problem happens for the `disable_referential_integrity` since we reconnect, the `FOREIGN_KEY_CHECKS` is reset to the original value
- Same solution as 1. , the disable_referential_integrity can be called after we reconnect to the transaction
3. When the multi statement query is executed, no other queries can be performed until we paginate over the set of results, otherwise mysql throws a "Commands out of sync" [Ref](https://dev.mysql.com/doc/refman/5.7/en/commands-out-of-sync.html)
- Iterating over the set of results until `mysql_client.next_result` is false. [Ref](https://github.com/brianmario/mysql2#multiple-result-sets)
- Removed the `active_record.sql "Fixture delete"` notification, the delete statements are now inside the INSERT's one
- On mysql the `max_allowed_packet` is looked up:
1. Before executing the multi-statements query, we check the packet length of each statements, if the packet is bigger than the max_allowed_packet config, an `ActiveRecordError` is raised
2. Otherwise we concatenate the current sql statement into the previous and so on until the packet is `< max_allowed_packet`
|
|\ \
| | |
| | | |
Don't perform unnecessary check with false, just use true/false values
|
| | |
| | |
| | |
| | | |
@active on SQLite adapter.
|
| | |
| | |
| | |
| | |
| | | |
This method which is used only in the internal was introduced in
ac384820 and was renamed in #17579. It does not need to be exposed.
|
| | | |
|
| | |
| | |
| | |
| | |
| | | |
Originally I tried to add `assert_equal pk.sql_type, ref.sql_type`. But
the assert failed even though the same type due to `sql_type` of primary
key in SQLite is upper case. Prefer lower case like other types.
|
| |/
|/| |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
`connection.primary_key` also return composite primary keys, so
`from_primary_key_column` may not be found even if `from_primary_key` is
presented.
```
% ARCONN=sqlite3 be ruby -w -Itest
test/cases/adapters/sqlite3/sqlite3_adapter_test.rb -n
test_copy_table_with_composite_primary_keys
Using sqlite3
Run options: -n test_copy_table_with_composite_primary_keys --seed 19041
# Running:
E
Error:
ActiveRecord::ConnectionAdapters::SQLite3AdapterTest#test_copy_table_with_composite_primary_keys:
NoMethodError: undefined method `type' for nil:NilClass
/path/to/rails/activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb:411:in
`block in copy_table'
```
This change fixes `copy_table` to do not lose composite primary keys.
|
|\ \
| | |
| | | |
Fix sqlite migrations with custom primary keys
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Previously, if a record was created with a custom primary key, that
table could not be migrated using sqlite. While attempting to copy the
table, the type of the primary key was ignored.
Once that was corrected, copying the indexes would fail because custom
primary keys are autoindexed by sqlite by default.
To correct that, this skips copying the index if the index name begins
with "sqlite_". This is a reserved word that indicates that the
index is an internal schema object. SQLite prohibits applications from
creating objects whose names begin with "sqlite_", so this string should
be safe to use as a check.
ref https://www.sqlite.org/fileformat2.html#intschema
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This is a regression since Rails 4.2.
SQLite3 integer is stored in 1, 2, 3, 4, 6, or 8 bytes depending on the
magnitude of the value. Assuming default valid value as 4 bytes caused
that actual valid value in INTEGER storage class cannot be stored and
existing value cannot be found.
https://www.sqlite.org/datatype3.html
We should allow valid value in INTEGER storage class in SQLite3 to fix
the regression.
Fixes #22594.
|
|/ /
| |
| |
| |
| | |
Actually SQLite3 doesn't have JSON storage class (so it is stored as a
TEXT like Date and Time). But emulating JSON types is convinient for
making database agnostic migrations.
|
| |
| |
| |
| |
| |
| | |
In #30510, `StatementPool` in `AbstractMysqlAdapter` was hidden in the
doc. But that class is also had in sqlite3 and postgresql adapters and
the base class is :nodoc: class.
|
| |
| |
| |
| | |
This basically reverts 9d4f79d3d394edb74fa2192e5d9ad7b09ce50c6d
|
| | |
|
| |
| |
| |
| |
| |
| |
| | |
Currently `SchemaDumper` is only customizable for column options. But
3rd party connection adapters (oracle-enhanced etc) need to customizable
for table or index dumping also. To make it possible, I introduced
adapter specific `SchemaDumper` classes for that.
|
| | |
|
| | |
|
| |
| |
| |
| | |
Because `to_sql` is public API. I introduced `to_sql_and_binds` internal
API to return SQL and binds.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
A common source of bugs and code bloat within Active Record has been the
need for us to maintain the list of bind values separately from the AST
they're associated with. This makes any sort of AST manipulation
incredibly difficult, as any time we want to potentially insert or
remove an AST node, we need to traverse the entire tree to find where
the associated bind parameters are.
With this change, the bind parameters now live on the AST directly.
Active Record does not need to know or care about them until the final
AST traversal for SQL construction. Rather than returning just the SQL,
the Arel collector will now return both the SQL and the bind parameters.
At this point the connection adapter will have all the values that it
had before.
A bit of this code is janky and something I'd like to refactor later. In
particular, I don't like how we're handling associations in the
predicate builder, the special casing of `StatementCache::Substitute` in
`QueryAttribute`, or generally how we're handling bind value replacement
in the statement cache when prepared statements are disabled.
This also mostly reverts #26378, as it moved all the code into a
location that I wanted to delete.
/cc @metaskills @yahonda, this change will affect the adapters
Fixes #29766.
Fixes #29804.
Fixes #26541.
Close #28539.
Close #24769.
Close #26468.
Close #26202.
There are probably other issues/PRs that can be closed because of this
commit, but that's all I could find on the first few pages.
|
| | |
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Abstract boolean serialization has been using 't' and 'f', with MySQL
overriding that to use 1 and 0.
This has the advantage that SQLite natively recognizes 1 and 0 as true
and false, but does not natively recognize 't' and 'f'.
This change in serialization requires a migration of stored boolean data
for SQLite databases, so it's implemented behind a configuration flag
whose default false value is deprecated. The flag itself can be
deprecated in a future version of Rails. While loaded models will give
the correct result for boolean columns without migrating old data,
where() clauses will interact incorrectly with old data.
While working in this area, also change the abstract adapter to use
`"TRUE"` and `"FALSE"` as quoted values and `true` and `false` for
unquoted. These are supported by PostreSQL, and MySQL remains
overriden.
|
|\ \ |
|
| | |
| | |
| | |
| | |
| | | |
This reverts commit 3420a14590c0e6915d8b6c242887f74adb4120f9, reversing
changes made to afb66a5a598ce4ac74ad84b125a5abf046dcf5aa.
|
| |\ \
| | | |
| | | |
| | | | |
Enforce frozen string in Rubocop
|
| | | | |
|
| |/ /
| | |
| | |
| | |
| | |
| | | |
`test_middleware_caches` is sometimes failed since #29454.
The failure is due to schema statements are affected by query caching.
Bypassing query caching for schema statements to avoid the issue.
|
|/ / |
|
| |
| |
| |
| |
| |
| |
| |
| | |
Improves the performance from O(n) to O(1).
Previously it would require 50 queries to
insert 50 fixtures. Now it takes only one query.
Disabled on sqlite which doesn't support multiple inserts.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
`supports_statement_cache?` was introduced in 3.1.0.beta1 (104d0b2) for
bind parameter substitution, but it is no longer used in 3.1.0.rc1
(73ff679). Originally it should respect `prepared_statements` rather
than `supports_statement_cache?` (fd39847).
One more thing, named `supports_statement_cache?` is pretty misreading.
We have `StatementCache` and `StatementPool`. However,
`supports_statement_cache?` doesn't mean `StatementCache`, but
`StatementPool` unlike its name.
https://github.com/rails/rails/blob/v5.1.0/activerecord/lib/active_record/statement_cache.rb
https://github.com/rails/rails/blob/v5.1.0/activerecord/lib/active_record/connection_adapters/statement_pool.rb
|
| |
| |
| |
| |
| |
| |
| | |
* Use keyword arguments in `IndexDefinition` to ease to ignore unused
options and to avoid to initialize incorrect empty value.
* Place it in `SchemaStatements` for consistency.
* And tiny tweaks.
|
| | |
|
|\ \
| | |
| | | |
Extract `data_source_sql` to refactor data source statements
|
| | | |
|