aboutsummaryrefslogtreecommitdiffstats
path: root/activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb
Commit message (Collapse)AuthorAgeFilesLines
* Prevent write queries for `exec_query`Ryuta Kamizono2018-12-111-1/+5
| | | | Follow up #34505.
* Rename error that occurs when writing on a readEileen Uchitelle2018-12-071-1/+1
| | | | | | | I originally named this `StatementInvalid` because that's what we do in GitHub, but `@tenderlove` pointed out that this means apps can't test for or explitly rescue this error. `StatementInvalid` is pretty broad so I've renamed this to `ReadOnlyError`.
* Address "warning: shadowing outer local variable - parts"Ryuta Kamizono2018-12-031-1/+2
| | | | And hide the `READ_QUERY` internal constant.
* Add ability to prevent writes to a databaseEileen Uchitelle2018-11-301-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR adds the ability to prevent writes to a database even if the database user is able to write (ie the database is a primary and not a replica). This is useful for a few reasons: 1) when converting your database from a single db to a primary/replica setup - you can fix all the writes on reads early on, 2) when we implement automatic database switching or when an app is manually switching connections this feature can be used to ensure reads are reading and writes are writing. We want to make sure we raise if we ever try to write in read mode, regardless of database type and 3) for local development if you don't want to set up multiple databases but do want to support rw/ro queries. This should be used in conjunction with `connected_to` in write mode. For example: ``` ActiveRecord::Base.connected_to(role: :writing) do Dog.connection.while_preventing_writes do Dog.create! # will raise because we're preventing writes end end ActiveRecord::Base.connected_to(role: :reading) do Dog.connection.while_preventing_writes do Dog.first # will not raise because we're not writing end end ```
* Merge pull request #34468 from gmcgibbon/redact_sql_in_errorsRafael França2018-11-231-4/+4
|\ | | | | Redact SQL in errors
| * Redact SQL in errorsGannon McGibbon2018-11-221-4/+4
| | | | | | | | | | Move `ActiveRecord::StatementInvalid` SQL to error property. Also add bindings as an error property.
* | Use squiggly heredoc to strip odd indentation in the executed SQLRyuta Kamizono2018-11-221-1/+1
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before: ``` LOG: execute <unnamed>: SELECT t.oid, t.typname FROM pg_type as t WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool') LOG: execute <unnamed>: SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype FROM pg_type as t LEFT JOIN pg_range as r ON oid = rngtypid WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'interval', 'path', 'line', 'polygon', 'circle', 'lseg', 'box', 'time', 'timestamp', 'numeric') OR t.typtype IN ('r', 'e', 'd') OR t.typinput::varchar = 'array_in' OR t.typelem != 0 LOG: statement: SHOW TIME ZONE LOG: statement: SELECT 1 LOG: execute <unnamed>: SELECT COUNT(*) FROM pg_class c LEFT JOIN pg_namespace n ON n.oid = c.relnamespace WHERE c.relkind IN ('r','v','m') -- (r)elation/table, (v)iew, (m)aterialized view AND c.relname = 'accounts' AND n.nspname = ANY (current_schemas(false)) ``` After: ``` LOG: execute <unnamed>: SELECT t.oid, t.typname FROM pg_type as t WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool') LOG: execute <unnamed>: SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype FROM pg_type as t LEFT JOIN pg_range as r ON oid = rngtypid WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'interval', 'path', 'line', 'polygon', 'circle', 'lseg', 'box', 'time', 'timestamp', 'numeric') OR t.typtype IN ('r', 'e', 'd') OR t.typinput::varchar = 'array_in' OR t.typelem != 0 LOG: statement: SHOW TIME ZONE LOG: statement: SELECT 1 LOG: execute <unnamed>: SELECT COUNT(*) FROM pg_class c LEFT JOIN pg_namespace n ON n.oid = c.relnamespace WHERE c.relkind IN ('r','v','m') -- (r)elation/table, (v)iew, (m)aterialized view AND c.relname = 'accounts' AND n.nspname = ANY (current_schemas(false)) ```
* Adjust bind length of SQLite to default (999)Gannon McGibbon2018-11-131-0/+6
| | | | | Change `#bind_params_length` in SQLite adapter to return the default maximum amount (999). See https://www.sqlite.org/limits.html
* Consistently extract checking version for all adaptersRyuta Kamizono2018-10-171-5/+6
| | | | | | | I don't prefer to extract it for one adapter even though all adapters also does. Related to #34227.
* Add `Style/RedundantFreeze` to remove redudant `.freeze`Yasuo Honda2018-09-291-1/+1
| | | | | | | | | | | | | | | | | | | | | Since Rails 6.0 will support Ruby 2.4.1 or higher `# frozen_string_literal: true` magic comment is enough to make string object frozen. This magic comment is enabled by `Style/FrozenStringLiteralComment` cop. * Exclude these files not to auto correct false positive `Regexp#freeze` - 'actionpack/lib/action_dispatch/journey/router/utils.rb' - 'activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb' It has been fixed by https://github.com/rubocop-hq/rubocop/pull/6333 Once the newer version of RuboCop released and available at Code Climate these exclude entries should be removed. * Replace `String#freeze` with `String#-@` manually if explicit frozen string objects are required - 'actionpack/test/controller/test_case_test.rb' - 'activemodel/test/cases/type/string_test.rb' - 'activesupport/lib/active_support/core_ext/string/strip.rb' - 'activesupport/test/core_ext/string_ext_test.rb' - 'railties/test/generators/actions_test.rb'
* Deprecate ActiveRecord::Result#to_hash in favor of #to_aKevin Cheng2018-09-181-1/+1
| | | | | | | method returns an array of hashes, not a hash e.g. Hash.try_convert(result) calls #to_hash and raises a TypeError [Gannon McGibbon + Kevin Cheng]
* SQLite3: Support multiple args function for expression indexesRyuta Kamizono2018-09-141-3/+6
| | | | | Follow up #33874. Related #23393.
* SQLite3 adapter supports expression indexesgkemmey2018-09-131-0/+4
|
* Merge pull request #32647 from eugeneius/lazy_transactionsMatthew Draper2018-08-231-0/+8
|\ | | | | Omit BEGIN/COMMIT statements for empty transactions
| * Omit BEGIN/COMMIT statements for empty transactionsEugene Kenny2018-08-131-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a transaction is opened and closed without any queries being run, we can safely omit the `BEGIN` and `COMMIT` statements, as they only exist to modify the connection's behaviour inside the transaction. This removes the overhead of those statements when saving a record with no changes, which makes workarounds like `save if changed?` unnecessary. This implementation buffers transactions inside the transaction manager and materializes them the next time the connection is used. For this to work, the adapter needs to guard all connection use with a call to `materialize_transactions`. Because of this, adapters must opt in to get this new behaviour by implementing `supports_lazy_transactions?`. If `raw_connection` is used to get a reference to the underlying database connection, the behaviour is disabled and transactions are opened eagerly, as we can't know how the connection will be used. However when the connection is checked back into the pool, we can assume that the application won't use the reference again and reenable lazy transactions. This prevents a single `raw_connection` call from disabling lazy transactions for the lifetime of the connection.
* | SQLite3: Fix rename reference column not to lose foreign key constraintRyuta Kamizono2018-08-201-11/+15
| | | | | | | | Fixes #33520.
* | SQLite3 adapter `alter_table` method restores foreign keysYasuo Honda2018-08-111-1/+9
|/ | | | Related to #33520
* SQLite: Don't leak internal schema objectsRyuta Kamizono2018-07-091-3/+0
| | | | | | | | | | | | | | Related #31201. If creating custom primary key (like a string) in SQLite, it would also create an internal index implicitly which named begin with "sqlite_". It need to be hidden since the internal object names are reserved and prohibited for public use. See https://www.sqlite.org/fileformat2.html#intschema Fixes #33320.
* Merge pull request #33242 from brasic/sqlite-readonlyRyuta Kamizono2018-07-031-1/+3
|\ | | | | | | Support readonly option in SQLite3Adapter
| * Support readonly option in SQLite3AdapterCarl Brasic2018-07-021-1/+3
|/ | | | | | | | | | | | | | | | | Readonly sqlite database files are very useful as a data format for storing configuration/lookup data that is too complicated for YAML files. But since such files would typically be committed to a source control repository, it's important to ensure that they are truly safe from being inadvertently modified. Unfortunately using unix permissions isn't enough, as sqlite will "helpfully" add the write bit to a database file whenever it's written to. sqlite3-ruby has supported a `:readonly` option since version 1.3.2 (see https://github.com/sparklemotion/sqlite3-ruby/commit/c20c9f5dd2990042) This simply passes that option through to the adapter if present in the config hash. I think this is best considered an adapter-specific option since no other supported database has an identical concept.
* Disable foreign keys during `alter_table` for sqlite3 adapterYasuo Honda2018-05-221-5/+10
| | | | | | | | | | | | | | | | | | | | | | | | | Unlike other databases, changing SQLite3 table definitions need to create a temporary table. While changing table operations, the original table needs dropped which caused `SQLite3::ConstraintException: FOREIGN KEY constraint failed` if the table is referenced by foreign keys. This pull request disables foreign keys by `disable_referential_integrity`. Also `disable_referential_integrity` method needs to execute `defer_foreign_keys = ON` to defer re-enabling foreign keys until the transaction is committed. https://www.sqlite.org/pragma.html#pragma_defer_foreign_keys Fixes #31988 - This `defer_foreign_keys = ON` has been supported since SQLite 3.8.0 https://www.sqlite.org/releaselog/3_8_0.html and Rails 6 requires SQLite 3.8 #32923 now - <Models>.reset_column_information added to address `ActiveModel::UnknownAttributeError` ``` Error: ActiveRecord::Migration::ForeignKeyChangeColumnTest#test_change_column_of_parent_table: ActiveModel::UnknownAttributeError: unknown attribute 'name' for ActiveRecord::Migration::ForeignKeyChangeColumnTest::Post. ```
* Bump minimum SQLite version to 3.8Yasuo Honda2018-05-211-6/+6
| | | | | | | | | | | | | | These OS versions have SQLite 3.8 or higher by default. - macOS 10.10 (Yosemite) or higher - Ubuntu 14.04 LTS or higher Raising the minimum version of SQLite 3.8 introduces these changes: - All of bundled adapters support `supports_multi_insert?` - SQLite 3.8 always satisifies `supports_foreign_keys_in_create?` and `supports_partial_index?` - sqlite adapter can support `alter_table` method for foreign key referenced tables by #32865 - Deprecated `supports_multi_insert?` method
* `columns` is cached in the statementRyuta Kamizono2018-04-241-6/+3
| | | | https://github.com/sparklemotion/sqlite3-ruby/blob/v1.3.13/lib/sqlite3/statement.rb#L101-L104
* Add test case for deprecated `insert_fixtures`Ryuta Kamizono2018-01-261-3/+5
| | | | Since #31422, `insert_fixtures` is deprecated.
* Merge pull request #31422 from Edouard-chin/multistatement-fixturesMatthew Draper2018-01-241-0/+12
|\ | | | | Build a multi-statement query when inserting fixtures
| * Build a multi-statement query when inserting fixtures:Edouard CHIN2018-01-221-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - The `insert_fixtures` method can be optimized by making a single multi statement query for all fixtures having the same connection instead of doing a single query per table - The previous code was bulk inserting fixtures for a single table, making X query for X fixture files - This patch builds a single **multi statement query** for every tables. Given a set of 3 fixtures (authors, dogs, computers): ```ruby # before %w(authors dogs computers).each do |table| sql = build_sql(table) connection.query(sql) end # after sql = build_sql(authors, dogs, computers) connection.query(sql) ``` - `insert_fixtures` is now deprecated, `insert_fixtures_set` is the new way to go with performance improvement - My tests were done with an app having more than 700 fixtures, the time it takes to insert all of them was around 15s. Using a single multi statement query, it took on average of 8 seconds - In order for a multi statement to be executed, mysql needs to be connected with the `MULTI_STATEMENTS` [flag](https://dev.mysql.com/doc/refman/5.7/en/c-api-multiple-queries.html), which is done before inserting the fixtures by reconnecting to da the database with the flag declared. Reconnecting to the database creates some caveats: 1. We loose all open transactions; Inside the original code, when inserting fixtures, a transaction is open. Multple delete statements are [executed](https://github.com/rails/rails/blob/a681eaf22955734c142609961a6d71746cfa0583/activerecord/lib/active_record/fixtures.rb#L566) and finally the fixtures are inserted. The problem with this patch is that we need to open the transaction only after we reconnect to the DB otherwise reconnecting drops the open transaction which doesn't commit all delete statements and inserting fixtures doesn't work since we duplicated them (Primary key duplicate exception)... - In order to fix this problem, the transaction is now open directly inside the `insert_fixtures` method, right after we reconnect to the db - As an effect, since the transaction is open inside the `insert_fixtures` method, the DELETE statements need to be executed here since the transaction is open later 2. The same problem happens for the `disable_referential_integrity` since we reconnect, the `FOREIGN_KEY_CHECKS` is reset to the original value - Same solution as 1. , the disable_referential_integrity can be called after we reconnect to the transaction 3. When the multi statement query is executed, no other queries can be performed until we paginate over the set of results, otherwise mysql throws a "Commands out of sync" [Ref](https://dev.mysql.com/doc/refman/5.7/en/commands-out-of-sync.html) - Iterating over the set of results until `mysql_client.next_result` is false. [Ref](https://github.com/brianmario/mysql2#multiple-result-sets) - Removed the `active_record.sql "Fixture delete"` notification, the delete statements are now inside the INSERT's one - On mysql the `max_allowed_packet` is looked up: 1. Before executing the multi-statements query, we check the packet length of each statements, if the packet is bigger than the max_allowed_packet config, an `ActiveRecordError` is raised 2. Otherwise we concatenate the current sql statement into the previous and so on until the packet is `< max_allowed_packet`
* | Merge pull request #24964 from vipulnsward/true-falseMatthew Draper2018-01-231-2/+2
|\ \ | | | | | | Don't perform unnecessary check with false, just use true/false values
| * | Dont perform unnecessary check with false, just use true/false values for ↵Vipul A M2016-05-111-2/+2
| | | | | | | | | | | | @active on SQLite adapter.
* | | Deprecate `valid_alter_table_type?` in sqlite3 adapterRyuta Kamizono2018-01-041-7/+12
| | | | | | | | | | | | | | | This method which is used only in the internal was introduced in ac384820 and was renamed in #17579. It does not need to be exposed.
* | | Fix recreating partial indexes after alter table for sqlitefatkodima2017-12-311-0/+1
| | |
* | | Make `sql_type` of primary key in SQLite to comparable with an integer (#28008)Ryuta Kamizono2017-12-261-1/+1
| | | | | | | | | | | | | | | Originally I tried to add `assert_equal pk.sql_type, ref.sql_type`. But the assert failed even though the same type due to `sql_type` of primary key in SQLite is upper case. Prefer lower case like other types.
* | | Fix `add_column` with :primary_key type compatibility for SQLitefatkodima2017-12-251-1/+1
| |/ |/|
* | SQLite: Fix `copy_table` with composite primary keysRyuta Kamizono2017-12-081-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `connection.primary_key` also return composite primary keys, so `from_primary_key_column` may not be found even if `from_primary_key` is presented. ``` % ARCONN=sqlite3 be ruby -w -Itest test/cases/adapters/sqlite3/sqlite3_adapter_test.rb -n test_copy_table_with_composite_primary_keys Using sqlite3 Run options: -n test_copy_table_with_composite_primary_keys --seed 19041 # Running: E Error: ActiveRecord::ConnectionAdapters::SQLite3AdapterTest#test_copy_table_with_composite_primary_keys: NoMethodError: undefined method `type' for nil:NilClass /path/to/rails/activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb:411:in `block in copy_table' ``` This change fixes `copy_table` to do not lose composite primary keys.
* | Merge pull request #31327 from aellispierce/custom-id-change-table-sqliteEileen M. Uchitelle2017-12-071-1/+5
|\ \ | | | | | | Fix sqlite migrations with custom primary keys
| * | Fix sqlite migrations with custom primary keysAshley Ellis Pierce2017-12-061-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, if a record was created with a custom primary key, that table could not be migrated using sqlite. While attempting to copy the table, the type of the primary key was ignored. Once that was corrected, copying the indexes would fail because custom primary keys are autoindexed by sqlite by default. To correct that, this skips copying the index if the index name begins with "sqlite_". This is a reserved word that indicates that the index is an internal schema object. SQLite prohibits applications from creating objects whose names begin with "sqlite_", so this string should be safe to use as a check. ref https://www.sqlite.org/fileformat2.html#intschema
* | | SQLite3 valid integer value should be 8 bytes (64-bit signed integer) (#28379)Ryuta Kamizono2017-12-031-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a regression since Rails 4.2. SQLite3 integer is stored in 1, 2, 3, 4, 6, or 8 bytes depending on the magnitude of the value. Assuming default valid value as 4 bytes caused that actual valid value in INTEGER storage class cannot be stored and existing value cannot be found. https://www.sqlite.org/datatype3.html We should allow valid value in INTEGER storage class in SQLite3 to fix the regression. Fixes #22594.
* | | Emulate JSON types for SQLite3 adapter (#29664)Ryuta Kamizono2017-12-031-1/+6
|/ / | | | | | | | | Actually SQLite3 doesn't have JSON storage class (so it is stored as a TEXT like Date and Time). But emulating JSON types is convinient for making database agnostic migrations.
* | Add :nodoc: to `StatementPool` which is internal used [ci skip]Ryuta Kamizono2017-11-301-2/+1
| | | | | | | | | | | | In #30510, `StatementPool` in `AbstractMysqlAdapter` was hidden in the doc. But that class is also had in sqlite3 and postgresql adapters and the base class is :nodoc: class.
* | [Active Record] require => require_relativeAkira Matsuda2017-10-211-8/+8
| | | | | | | | This basically reverts 9d4f79d3d394edb74fa2192e5d9ad7b09ce50c6d
* | Use tt in doc for ActiveRecord [ci skip]Yoshiyuki Hirano2017-08-271-2/+2
| |
* | Refactor `SchemaDumper` to make it possible to adapter specific customizationRyuta Kamizono2017-08-221-1/+0
| | | | | | | | | | | | | | Currently `SchemaDumper` is only customizable for column options. But 3rd party connection adapters (oracle-enhanced etc) need to customizable for table or index dumping also. To make it possible, I introduced adapter specific `SchemaDumper` classes for that.
* | Update links to use https instead of http [ci skip]Yoshiyuki Hirano2017-08-221-1/+1
| |
* | Place `update_table_definition` consistently in `SchemaStatements`Ryuta Kamizono2017-08-211-4/+0
| |
* | Restore `to_sql` to return only SQL (#29945)Ryuta Kamizono2017-08-181-2/+1
| | | | | | | | Because `to_sql` is public API. I introduced `to_sql_and_binds` internal API to return SQL and binds.
* | Refactor Active Record to let Arel manage bind paramsSean Griffin2017-07-241-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A common source of bugs and code bloat within Active Record has been the need for us to maintain the list of bind values separately from the AST they're associated with. This makes any sort of AST manipulation incredibly difficult, as any time we want to potentially insert or remove an AST node, we need to traverse the entire tree to find where the associated bind parameters are. With this change, the bind parameters now live on the AST directly. Active Record does not need to know or care about them until the final AST traversal for SQL construction. Rather than returning just the SQL, the Arel collector will now return both the SQL and the bind parameters. At this point the connection adapter will have all the values that it had before. A bit of this code is janky and something I'd like to refactor later. In particular, I don't like how we're handling associations in the predicate builder, the special casing of `StatementCache::Substitute` in `QueryAttribute`, or generally how we're handling bind value replacement in the statement cache when prepared statements are disabled. This also mostly reverts #26378, as it moved all the code into a location that I wanted to delete. /cc @metaskills @yahonda, this change will affect the adapters Fixes #29766. Fixes #29804. Fixes #26541. Close #28539. Close #24769. Close #26468. Close #26202. There are probably other issues/PRs that can be closed because of this commit, but that's all I could find on the first few pages.
* | Use frozen-string-literal in ActiveRecordKir Shatrov2017-07-191-0/+2
| |
* | Set `represent_boolean_as_integer` via `configuration`yuuji.yaginuma2017-07-161-1/+1
| |
* | Fix boolean column migration scriptyuuji.yaginuma2017-07-131-1/+1
| |
* | Change sqlite3 boolean serialization to use 1 and 0Lisa Ugray2017-07-111-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Abstract boolean serialization has been using 't' and 'f', with MySQL overriding that to use 1 and 0. This has the advantage that SQLite natively recognizes 1 and 0 as true and false, but does not natively recognize 't' and 'f'. This change in serialization requires a migration of stored boolean data for SQLite databases, so it's implemented behind a configuration flag whose default false value is deprecated. The flag itself can be deprecated in a future version of Rails. While loaded models will give the correct result for boolean columns without migrating old data, where() clauses will interact incorrectly with old data. While working in this area, also change the abstract adapter to use `"TRUE"` and `"FALSE"` as quoted values and `true` and `false` for unquoted. These are supported by PostreSQL, and MySQL remains overriden.
* | Merge branch 'master' into require_relative_2017Xavier Noria2017-07-021-3/+3
|\ \
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-05 14:55:21 +0000