| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
| |
[Gannon McGibbon + Max Albrecht]
|
|
|
|
|
| |
This commit follows the path we started at commit #ea4f0e2
and continued at PR #33229.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Provides both a forked process and threaded parallelization options. To
use add `parallelize` to your test suite.
Takes a `workers` argument that controls how many times the process
is forked. For each process a new database will be created suffixed
with the worker number; test-database-0 and test-database-1
respectively.
If `ENV["PARALLEL_WORKERS"]` is set the workers argument will be ignored
and the environment variable will be used instead. This is useful for CI
environments, or other environments where you may need more workers than
you do for local testing.
If the number of workers is set to `1` or fewer, the tests will not be
parallelized.
The default parallelization method is to fork processes. If you'd like to
use threads instead you can pass `with: :threads` to the `parallelize`
method. Note the threaded parallelization does not create multiple
database and will not work with system tests at this time.
parallelize(workers: 2, with: :threads)
The threaded parallelization uses Minitest's parallel exector directly.
The processes paralleliztion uses a Ruby Drb server.
For parallelization via threads a setup hook and cleanup hook are
provided.
```
class ActiveSupport::TestCase
parallelize_setup do |worker|
# setup databases
end
parallelize_teardown do |worker|
# cleanup database
end
parallelize(workers: 2)
end
```
[Eileen M. Uchitelle, Aaron Patterson]
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- The `insert_fixtures` method can be optimized by making a single multi statement query for all fixtures having the same connection instead of doing a single query per table
- The previous code was bulk inserting fixtures for a single table, making X query for X fixture files
- This patch builds a single **multi statement query** for every tables. Given a set of 3 fixtures (authors, dogs, computers):
```ruby
# before
%w(authors dogs computers).each do |table|
sql = build_sql(table)
connection.query(sql)
end
# after
sql = build_sql(authors, dogs, computers)
connection.query(sql)
```
- `insert_fixtures` is now deprecated, `insert_fixtures_set` is the new way to go with performance improvement
- My tests were done with an app having more than 700 fixtures, the time it takes to insert all of them was around 15s. Using a single multi statement query, it took on average of 8 seconds
- In order for a multi statement to be executed, mysql needs to be connected with the `MULTI_STATEMENTS` [flag](https://dev.mysql.com/doc/refman/5.7/en/c-api-multiple-queries.html), which is done before inserting the fixtures by reconnecting to da the database with the flag declared. Reconnecting to the database creates some caveats:
1. We loose all open transactions; Inside the original code, when inserting fixtures, a transaction is open. Multple delete statements are [executed](https://github.com/rails/rails/blob/a681eaf22955734c142609961a6d71746cfa0583/activerecord/lib/active_record/fixtures.rb#L566) and finally the fixtures are inserted. The problem with this patch is that we need to open the transaction only after we reconnect to the DB otherwise reconnecting drops the open transaction which doesn't commit all delete statements and inserting fixtures doesn't work since we duplicated them (Primary key duplicate exception)...
- In order to fix this problem, the transaction is now open directly inside the `insert_fixtures` method, right after we reconnect to the db
- As an effect, since the transaction is open inside the `insert_fixtures` method, the DELETE statements need to be executed here since the transaction is open later
2. The same problem happens for the `disable_referential_integrity` since we reconnect, the `FOREIGN_KEY_CHECKS` is reset to the original value
- Same solution as 1. , the disable_referential_integrity can be called after we reconnect to the transaction
3. When the multi statement query is executed, no other queries can be performed until we paginate over the set of results, otherwise mysql throws a "Commands out of sync" [Ref](https://dev.mysql.com/doc/refman/5.7/en/commands-out-of-sync.html)
- Iterating over the set of results until `mysql_client.next_result` is false. [Ref](https://github.com/brianmario/mysql2#multiple-result-sets)
- Removed the `active_record.sql "Fixture delete"` notification, the delete statements are now inside the INSERT's one
- On mysql the `max_allowed_packet` is looked up:
1. Before executing the multi-statements query, we check the packet length of each statements, if the packet is bigger than the max_allowed_packet config, an `ActiveRecordError` is raised
2. Otherwise we concatenate the current sql statement into the previous and so on until the packet is `< max_allowed_packet`
|
|
|
|
| |
This basically reverts 9d4f79d3d394edb74fa2192e5d9ad7b09ce50c6d
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
Improves the performance from O(n) to O(1).
Previously it would require 50 queries to
insert 50 fixtures. Now it takes only one query.
Disabled on sqlite which doesn't support multiple inserts.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Allow a default value to be declared for class_attribute
* Convert to using class_attribute default rather than explicit setter
* Removed instance_accessor option by mistake
* False is a valid default value
* Documentation
|
|
|
|
| |
fixtures, not an empty array.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This ensures multiple threads inside a transactional test to see consistent
database state.
When a system test starts Puma spins up one thread and Capybara spins up
another thread. Because of this when tests are run the database cannot
see what was inserted into the database on teardown. This is because
there are two threads using two different connections.
This change uses the statement cache to lock the threads to using a
single connection ID instead of each not being able to see each other.
This code only runs in the fixture setup and teardown so it does not
affect real production databases.
When a transaction is opened we set `lock_thread` to `Thread.current` so
we can keep track of which connection the thread is using. When we
rollback the transaction we unlock the thread and then there will be no
left-over data in the database because the transaction will roll back
the correct connections.
[ Eileen M. Uchitelle, Matthew Draper ]
|
|
|
|
| |
Actually, private methods cannot be called with `self.`, so it's not just redundant, it's a bad habit in Ruby
|
| |
|
| |
|
|
|
|
| |
[ci skip]
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
If using namespaced fixtures, get following Ruby warning.
```
activerecord/lib/active_record/fixtures.rb:922: warning: method redefined; discarding old admin_foos
activerecord/lib/active_record/fixtures.rb:922: warning: previous definition of admin_foos was here
```
This is happening because of the multiple set the same path when setting the
fixture name. Fix to remove the duplicate path.
|
|
|
|
|
|
|
| |
`ActiveSupport::TestCase` was replaced `ActiveRecord::TestCase` in #26150.
But this docs is for rails apps per se, it should be `ActiveSupport::TestCase`.
See https://github.com/rails/rails/pull/26150#discussion_r74710989.
|
|
|
|
| |
test cases
|
| |
|
| |
|
| |
|
|
|
|
|
| |
The current code base is not uniform. After some discussion,
we have chosen to go with double quotes by default.
|
|
|
|
|
|
| |
Follow up to #20818.
`retrieve_connection` is passed `spec_name` instead of `klass` since #24844.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
loaded model classes have their connections wrapped in transactions.
See #17776
In Rails 4 config.eager_load was changed to false in the test environment. This
means that model classes that connect to alternate databases with
establish_connection are not loaded at start up. If use_transactional_fixtures
is enabled, transactions are wrapped around the connections that have been
established only at the start of the test suite. So model classes loaded later
don't have transactions causing data created in the alternate database not to
be removed.
This change resolves that by creating a new connection.active_record
notification that gets fired whenever a connection is established. I then added
a subscriber after we set up transactions in the test environment to listen for
additional connections and wrap those in transactions as well.
|
|
|
| |
[ci skip]
|
|
|
|
| |
Still more to do. Please assist!
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Look at `TestFixtures.set_fixture_class`. As documented, it
accepts a mapping of fixture identifiers (string or symbol) to Classes
(the model classes that implement the named fixture).
Look now at the initialization of `TestFixtures.fixture_class_names`.
It defines a Hash, which will return a string by default (where the
string is the estimated class name of the given fixture identifier).
Now look at TestFixtures.load_fixtures. It calls `FixtureSet.create_fixtures`,
passing in the mapping of `fixture_class_names`.
Following this on to `FixtureSet.create_fixtures`, this instantiates a
`FixtureSet::ClassCache`, passing in the map of class names.
`ClassCache`, in turn, calls `insert_class` for each value in the cache.
(Recall that `set_fixture_class` puts Class objects in there, while the
default proc for the mapping puts String objects.)
Look finally at `insert_class`. If the value is present, it checks to
see if the value is a subclass of `AR::Base`. Fair enough...but wait!
What if the value is a String? You get an exception, because a String
instance cannot be compared with a Class.
Judging from the implementation, it seems like the expected behavior
here is for `fixture_class_names` to have no default proc. Look-ups are
supposed to happen via `ClassCache`, with `fixture_class_names` existing
solely as a repository for explicitly-registered class mappings.
That is what this change does.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The focus of this change is to make the API more accessible.
References to method and classes should be linked to make it easy to
navigate around.
This patch makes exzessiv use of `rdoc-ref:` to provide more readable
docs. This makes it possible to document `ActiveRecord::Base#save` even
though the method is within a separate module
`ActiveRecord::Persistence`. The goal here is to bring the API closer to
the actual code that you would write.
This commit only deals with Active Record. The other gems will be
updated accordingly but in different commits. The pass through Active
Record is not completely finished yet. A follow up commit will change
the spots I haven't yet had the time to update.
/cc @fxn
|
|\
| |
| |
| |
| |
| |
| | |
Allow fixtures YAML files to set the model class in the file itself
Conflicts:
activerecord/CHANGELOG.md
|
|/
|
|
|
|
|
|
| |
Currently, `set_fixture_class` is only available using the
`TestFixtures` concern and it is ignored for `rake db:fixtures:load`.
Using the correct model class, it is possible for the fixture load
to also load the associations from the YAML files (e.g., `:belongs_to`
and `:has_many`).
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
After 908cfef was introduced fixtures that did not set an enum would
return nil instead of the default enum value.
The fixtures should assume the default if a different enum is not
defined.
The change checks first if the enum is defined in the fixture before
setting it based on the fixture.
|
|
|
|
|
| |
My impression of the code sample is that the intent is to assign guy_1, guy_2, etc. The code as it stood would have assigned guy_1, guy_1, etc.
This simply replaces the 1 with the iterator variable i.
|
|\
| |
| | |
Fix crash when loading fixture with belongs_to association defined in abstract base class
|
| |
| |
| |
| | |
abstract base class.
|
|/
|
|
| |
Time instance will be casted so we don't have to do `to_s(:db)`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, values for columns backing Active Record enums must be
specified as integers in test fixtures:
awdr:
title: "Agile Web Development with Rails"
status: 2
rfr:
title: "Ruby for Rails"
status: <%= Book.statuses[:proposed] %>
This is potentially confusing, since enum values are typically
specified as symbols or strings in application code. To resolve the
confusion, this change permits the use of symbols or strings to specify
enum values:
awdr:
status: :published
It is compatible with fixtures that specify enum values as integers.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I’m renaming all instances of `use_transcational_fixtures` to
`use_transactional_tests` and “transactional fixtures” to
“transactional tests”.
I’m deprecating `use_transactional_fixtures=`. So anyone who is
explicitly setting this will get a warning telling them to use
`use_transactional_tests=` instead.
I’m maintaining backwards compatibility—both forms will work.
`use_transactional_tests` will check to see if
`use_transactional_fixtures` is set and use that, otherwise it will use
itself. But because `use_transactional_tests` is a class attribute
(created with `class_attribute`) this requires a little bit of hoop
jumping. The writer method that `class_attribute` generates defines a
new reader method that return the value being set. Which means we can’t
set the default of `true` using `use_transactional_tests=` as was done
previously because that won’t take into account anyone using
`use_transactional_fixtures`. Instead I defined the reader method
manually and it checks `use_transactional_fixtures`. If it was set then
it should be used, otherwise it should return the default, which is
`true`. If someone uses `use_transactional_tests=` then it will
overwrite the backwards-compatible method with whatever they set.
|
|\
| |
| | |
Added testcase for #18742
|
| |
| |
| |
| | |
When a fs.model_class.connection uses a different database than connection, connection.reset_pk_sequence will fail with an exception causing fixture load to rollback. This is reproducible for any ActiveRecord::Base class that calls establish_connection with a different database.
|
|\ \
| |/
|/| |
Rely on through table name in has_many fixtures
|
| |
| |
| |
| |
| |
| |
| | |
Rather than using the association's join_table method, which
constructs a table name from conventions, this should rely on the
through reflection's table_name to be resilient to tables that were
not automatically named.
|
| |
| |
| |
| |
| |
| | |
[ci skip]
ref https://bugs.ruby-lang.org/issues/8846
|
| | |
|
|\ \
| | |
| | | |
Fix STI for fixtures from multi-files
|