aboutsummaryrefslogtreecommitdiffstats
path: root/railties/test/application/test_runner_test.rb
Commit message (Collapse)AuthorAgeFilesLines
* Fail parallel tests if workers exit earlyJohn Hawthorn2019-05-301-0/+18
| | | | | | | | | | | | | | | | Previously, if a test worker exited early, the in-flight test it was supposed to run wasn't reported as a failure. If all workers exited immediately, this would be reported as ex. Finished in 1.708349s, 39.2192 runs/s, 79.0237 assertions/s. 67 runs, 135 assertions, 0 failures, 0 errors, 2 skips This commit validates that all workers finish running tests by ensuring that the queue is empty after they exit. This works because we signal the workers to exit by pushing nil onto the queue, so that there should be a number of items left in the queue matching potentially missed tests.
* Make system tests take failed screenshots in `before_teardown` hookRichard Macklin2019-04-201-0/+26
| | | | | | | | | | | | Previously we were calling the `take_failed_screenshot` method in an `after_teardown` hook. However, this means that other teardown hooks have to be executed before we take the screenshot. Since there can be dynamic updates to the page after the assertion fails and before we take a screenshot, it seems desirable to minimize that gap as much as possible. Taking the screenshot in a `before_teardown` rather than an `after_teardown` helps with that, and has a side benefit of allowing us to remove the nested `ensure` commented on here: https://github.com/rails/rails/pull/34411#discussion_r232819478
* Railities typo fixes.alkesh262019-02-011-1/+1
|
* Rename methods and update docsEileen Uchitelle2019-01-291-0/+1
| | | | | | | | | | | | | | This is a minor update to the named methods for the following: - s/desired_capabilities/capabilities - s/driver_options/capabilities Since they are all the same thing we should keep the name the same throughout the feature. Updated docs to match / be a little bit clearer Also updated the Gemfile for selenium-webdriver.
* Implement a way to add browser capabilities:Edouard CHIN2019-01-291-0/+2
| | | | | | | | | | | | | | | | | | * There is currently no way to define specific browser capabilities since our SystemTest driver override the `option` key [Ref](https://github.com/rails/rails/blob/a07d0680787ced3c04b362fa7a238c918211ac70/actionpack/lib/action_dispatch/system_testing/driver.rb#L35) This option key is used internally by selenium to add custom capabilities on the browser. Depending on the Browser, some option are allowed to be passed inside a hash, the driver takes care of setting whatever you passed on the driver option. An example [here](https://github.com/rails/rails/blob/a07d0680787ced3c04b362fa7a238c918211ac70/actionpack/lib/action_dispatch/system_testing/driver.rb#L35) where you are allowed to pass args such as `--no-sandbox` etc However this behavior was only meant for backward compatibility and as you can see it's deprecated. The non-deprecated behavior is to create a `<Driver>::Option` object containing all the capabilities we want. This is what we [currently do](https://github.com/rails/rails/blob/a07d0680787ced3c04b362fa7a238c918211ac70/actionpack/lib/action_dispatch/system_testing/browser.rb#L34-L36) when chrome or firefox are in headless mode. This PR allows to pass a block when calling `driven_by`, the block will be pased a `<Driver>::Option` instance. You can modify this object the way you want by adding any capabilities. The option object will be then passed to selenium. ```ruby driven_by :selenium, using: :chrome do |driver_option| driver_option.add_argument('--no-sandbox') driver_option.add_emulation(device: 'iphone 4') end ```
* Add `rails test:channels`.bogdanvlviv2019-01-161-2/+13
| | | | | Add this rake task to test channels only. We've added `rails test:mailboxes` recently in the same way #34828.
* Add Rake task for testing mailboxesGeorge Claghorn2018-12-301-2/+14
|
* Clarify implicit meaning of 'workers: 2' in parallelization tests.Kasper Timm Hansen2018-12-191-6/+8
|
* Add option to set parallel test worker count to the physical core count of ↵Bogdan2018-12-181-5/+14
| | | | | | | | | | | | | | | | | | | | | | the machine (#34735) * Add option to set parallel test worker count to the physical core count of the machine Also, use the physical core count of the machine as the default number of workers, and generate the `test_helper.rb` file with `parallelize(workers: :number_of_processors)` Closes #34734 * Ensure that we always test parallel testing Since #34734 we decided to use the physical core count of the machine as the default number of workers in the parallel testing, we need to ensure that some tests use at least 2 workers because we could run those tests on VM that has only 1 physical core. It also fixes tests failures on the CI since Travis server we are using has only one physical core. See https://travis-ci.org/rails/rails/jobs/469281088#L2352
* Correctly handle unknown object in parallel testsyuuji.yaginuma2018-11-171-0/+14
| | | | | | | | | | | | | | | | | | | | | DRb wraps in `DRbUnknown` if the data contains a type that can not be resolved locally. This can happen if an error occurs in the test and the error class can not be resolved on the server side. When this happens, an instance of `DRbUnknown` is passed to the `result` of `Server#record`. This causes another error(undefined method assertions for #<DRb::DRbUnknown:> (NoMethodError)) in `reporter.record`. This can confirm by the following steps. ``` $ rails new app -B --dev; cd app $ rails g scaffold user name:string $ edit `config.action_controller.allow_forgery_protection = true` in environments/test.rb $ bin/rails t ``` In the case of `DRbUnknown` occurs, can't resolve error exception. So wrap exception with `DRbRemoteError` in the same way as an unmarshalled object.
* Merge pull request #34411 from N0xFF/masterRafael França2018-11-131-0/+25
|\ | | | | Reset Capybara sessions if failed system test screenshot raising an exception
| * Reset sessions on failed system test screenshotMaxim Perepelitsa2018-11-131-0/+25
| | | | | | | | | | Reset Capybara sessions if `take_failed_screenshot` raise exception in system test `after_teardown`.
* | Add test for parallel tests with unmarshable exceptionyuuji.yaginuma2018-11-091-0/+24
|/ | | | Follow up #34131.
* Ensure that running tests in parallel doesn't display schema load outputbogdanvlviv2018-08-081-0/+18
| | | | | | | | https://github.com/rails/rails/pull/33479 changed `#load_schema` to prevent displaying schema load on running tests in parallel. We should test this in order to prevent any regression in the future. Context https://github.com/rails/rails/pull/33479#discussion_r206870727
* Show rails instead of bin/rails on USAGE instructionsAlberto Almagro2018-07-061-1/+1
| | | | | With this commit, rails commands usage instructions display now +rails+ instead of +bin/rails+ within their recommendations.
* Recommend use of rails over bin/railsAlberto Almagro2018-07-061-1/+1
| | | | | | | | | As discussed in #33203 rails command already looks for, and runs, bin/rails if it is present. We were mixing recommendations within guides and USAGE guidelines, in some files we recommended using rails, in others bin/rails and in some cases we even had both options mixed together.
* Remove needless printyuuji.yaginuma2018-02-171-1/+0
| | | | It seems to debug print.
* Remove usage of strip_heredoc in the framework in favor of <<~Rafael Mendonça França2018-02-161-2/+1
| | | | | Some places we can't remove because Ruby still don't have a method equivalent to strip_heredoc to be called in an already existent string.
* Add test parallelization to Railseileencodes2018-02-151-4/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Provides both a forked process and threaded parallelization options. To use add `parallelize` to your test suite. Takes a `workers` argument that controls how many times the process is forked. For each process a new database will be created suffixed with the worker number; test-database-0 and test-database-1 respectively. If `ENV["PARALLEL_WORKERS"]` is set the workers argument will be ignored and the environment variable will be used instead. This is useful for CI environments, or other environments where you may need more workers than you do for local testing. If the number of workers is set to `1` or fewer, the tests will not be parallelized. The default parallelization method is to fork processes. If you'd like to use threads instead you can pass `with: :threads` to the `parallelize` method. Note the threaded parallelization does not create multiple database and will not work with system tests at this time. parallelize(workers: 2, with: :threads) The threaded parallelization uses Minitest's parallel exector directly. The processes paralleliztion uses a Ruby Drb server. For parallelization via threads a setup hook and cleanup hook are provided. ``` class ActiveSupport::TestCase parallelize_setup do |worker| # setup databases end parallelize_teardown do |worker| # cleanup database end parallelize(workers: 2) end ``` [Eileen M. Uchitelle, Aaron Patterson]
* Set the Rails environment from an environment variableAaron Patterson2017-12-081-1/+1
| | | | | | | | | Option parsing happens too late to have any impact on the Rails environment. Rails accesses the environment name and memoizes it too early in the boot process for a commandline option to have any impact on the database connection, so we'll change this test to set the environment from an environment variable (and ensure it still works when running tests with `ruby`)
* Add failing test for wrong database connectionAaron Patterson2017-12-081-0/+17
| | | | | | When tests are run with just `ruby`, the RAILS_ENV is set to `development` too early, and we connect to the development database rather than the test database.
* Fix Rails environment when running tests with RubyAaron Patterson2017-12-061-0/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | I frequently run tests with `ruby`, not with a runner like `rake` or `rails`. When running the test with just `ruby` the `RAILS_ENV` environment variable did not get set to "test", and this would cause the tests to fail (and even mutate the development database!) This commit adds integration tests for running tests with just `ruby` and ensures the environment gets defaulted to "test". I also added a test to ensure that passing an environment to `-e` actually works (and fixed that case too). An interesting / annoying thing is that Minitest picks up it's plugins by asking RubyGems for a list of files: https://github.com/seattlerb/minitest/blob/ca6a71ca901016db09a5ad466b4adea4b52a504a/lib/minitest.rb#L92-L100 This means that RubyGems needs to somehow know about the file before it can return it to Minitest. Since we are not packaging Rails as a Gem before running the integration tests on it (duh, why would you do that?), RubyGems doesn't know about the file, so it can't tell Minitest, so Minitest doesn't automatically require it. This means I had to manually require and insert the plugin in our integration test. I've left comments about that in the test as well. Ugh.
* Remove unnecessary migration deletionyuuji.yaginuma2017-11-081-5/+0
| | | | | Since isolation application is generated with the `--skip-gemfile` option, so `active_storage:install` is not executed.
* `rails new` runs `rails active_storage:install`bogdanvlviv2017-11-061-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Omit `rails activestorage:install` for jdbcmysql, jdbc and shebang tests AppGeneratorTest#test_config_jdbcmysql_database rails aborted! LoadError: Could not load 'active_record/connection_adapters/mysql_adapter'. Make sure that the adapter in config/database.yml is valid. If you use an adapter other than 'mysql2', 'postgresql' or 'sqlite3' add the necessary adapter gem to the Gemfile. (compressed) bin/rails:4:in `<main>' Tasks: TOP => activestorage:install => environment (See full trace by running task with --trace) AppGeneratorTest#test_config_jdbc_database rails aborted! LoadError: Could not load 'active_record/connection_adapters/jdbc_adapter'. Make sure that the adapter in config/database.yml is valid. If you use an adapter other than 'mysql2', 'postgresql' or 'sqlite3' add the necessary adapter gem to the Gemfile. (compressed) bin/rails:4:in `<main>' Tasks: TOP => activestorage:install => environment (See full trace by running task with --trace) AppGeneratorTest#test_shebang_is_added_to_rails_file /home/ubuntu/.rbenv/versions/2.4.1/bin/ruby: no Ruby script found in input (LoadError) Prevent PendingMigrationError in tests * Run `bin/rails db:migrate RAILS_ENV=test` in test_cases before start tests to prevent PendingMigrationError * FileUtils.rm_r("db/migrate") * --skip-active-storage Fix failed tests in `railties/test/railties/engine_test.rb` Related to #30111 Imporve `SharedGeneratorTests#test_default_frameworks_are_required_when_others_are_removed` - Explicitly skip active_storage - Ensure that skipped frameworks are commented - Ensure that default frameworks are not commented Fix error `Errno::ENOSPC: No space left on device - sendfile` Since `rails new` runs `rails active_storage:install` that boots an app. Since adding Bootsnap 0312a5c67e35b960e33677b5358c539f1047e4e1 during booting an app, it creates the cache: 264K tmp/cache/bootsnap-load-path-cache 27M tmp/cache/bootsnap-compile-cache * teardown_app must remove app
* Run in-app rails commands via fork+load where possibleMatthew Draper2017-09-041-28/+26
| | | | | While this avoids shell argument parsing, we still pass through everything in our stack.
* Adding frozen_string_literal pragma to Railties.Pat Allan2017-08-141-0/+2
|
* Merge remote-tracking branch 'origin/master' into unlock-minitestRafael Mendonça França2017-08-011-2/+28
|\
| * Allow bin/rails test task to take absolute paths as argumentsPawan Dubey2017-07-251-0/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Solves #29923 This regression was caused due to a wrong regex to filter out paths, introduced in commit 796a1cf0e The regex was /^\w+\// which did not accept paths with a leading slash and hence all absolute paths were filtered out. This change introduces a change in regex which allows for a leading slash and acts on the matched term accordingly. While cascading through the case block, the paths are checked for line number specification, existence of a directory at that path and if none of those match, then it is considered to be a path to the file. The regex matchers specified are filtered out via the call to `Array#compact` since they do not match any of these conditions.
| * * Don't eagerly require Rails' minitest plugin.Kasper Timm Hansen2017-07-101-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By making the Rails minitest behave like a standard minitest plugin we're much more likely to not break when people use other minitest plugins. Like minitest-focus and pride. To do this, we need to behave like minitest: require files up front and then perform the plugin behavior via the at_exit hook. This also saves us a fair bit of wrangling with test file loading. Finally, since the environment and warnings options have to be applied as early as possible, and since minitest loads plugins at_exit, they have to be moved to the test command. * Don't expect the root method. It's likely this worked because we eagerly loaded the Rails minitest plugin and that somehow defined a root method on `Rails`. * Assign a backtrace to failed exceptions. Otherwise Minitest pukes when attempting to filter the backtrace (which Rails' backtrace cleaner then removes). Means the exception message test has to be revised too. This is likely caused by the rails minitest plugin now being loaded for these tests and assigning a default backtrace cleaner.
| * Revert "Merge pull request #29540 from kirs/rubocop-frozen-string"Matthew Draper2017-07-021-1/+0
| | | | | | | | | | This reverts commit 3420a14590c0e6915d8b6c242887f74adb4120f9, reversing changes made to afb66a5a598ce4ac74ad84b125a5abf046dcf5aa.
| * Enforce frozen string in RubocopKir Shatrov2017-07-011-0/+1
| |
* | Fix test runner's outputyuuji.yaginuma2017-07-301-1/+1
|/ | | | | | | | Output changed due to specification change of `SummaryReporter#aggregated_results` in minitest 5.10.2. In my opinion, that should fix rails's test runner(proceeding with #29354). However, we still need discussion and the fix itself is minor, so I think that we can fix only the test first.
* Define path with __dir__bogdanvlviv2017-05-231-1/+1
| | | | | | ".. with __dir__ we can restore order in the Universe." - by @fxn Related to 5b8738c2df003a96f0e490c43559747618d10f5f
* Remove duplicated "test" prefixRyuta Kamizono2017-04-071-1/+1
|
* Avoid running system tests by defaultRobin Dupret2017-03-051-0/+46
| | | | | | | | | | These tests may be expansive so let's only allow users to run them through `bin/rails test:system` or by passing a path to the `test` command. The same applies for `bin/rake test`. Refs #28109.
* Fix random failure on system test with ajaxFumiaki MATSUSHIMA2017-03-031-0/+28
| | | | | | | | If application has ajax, browser may begin request after rollback. `teardown_fixtures` will be called after `super` on `after_teardown` so we must call `Capybara.reset_sessions!` before `super` https://github.com/rails/rails/blob/b61a56541aecd7ac685d4f19d943177a3f1b465a/activerecord/lib/active_record/fixtures.rb#L857
* Add back tests for test:units and test:functionals.Kasper Timm Hansen2017-02-231-12/+16
| | | | | | | Would have caught that the invoke changes broke rake delegation behavior. And we do ship the behavior so we should test it.
* Fix run_via[]= backwards compatibility.Kasper Timm Hansen2017-02-211-0/+10
| | | | | | | | | ``` Minitest.run_via[:rails] = true ``` 👆 would break because a simple alias won't catch the second true argument there.
* Collect all file patterns when running multiple rake test tasksDominic Cleal2017-02-041-0/+15
| | | | | | | Replaces the rake_patterns instance variable with simple require, as `autorun` will run tests from all eagerly required test files. Fixes #27801
* add warnings option to test runneryuuji.yaginuma2017-01-241-0/+11
|
* Add more rubocop rules about whitespacesRafael Mendonça França2016-10-291-6/+6
|
* applies new string literal convention in railties/testXavier Noria2016-08-061-124/+124
| | | | | The current code base is not uniform. After some discussion, we have chosen to go with double quotes by default.
* remove unnessary option setting from test runneryuuji.yaginuma2016-04-251-0/+8
| | | | | | | If run the test over the `rake` command, because of the test patterns is passed via `rake_run` method, do not need to be obtained from the argv. This probably fixes #24372.
* reestablish previous connection after creating all databasesAaron Patterson2016-02-231-0/+13
| | | | | | creating all databases mutates the connection pool. This patch restores the connection pool to the connection spec established before creating all databases. Fixes #23279
* make sure `rake test` respects TESTOPTSAaron Patterson2016-02-221-0/+13
| | | | | We should be able to pass options to minitest via TESTOPTS environment variable
* test runner, relay minitest information about the error location.Yves Senn2016-02-151-1/+1
| | | | | This is a follow-up to ea70c29 to bring back the assertion error location information provided by minitest.
* Revert "Prefer Minitest's location for test failures."Yves Senn2016-02-151-1/+1
| | | | | | | | | | | | | | | | | | | | This reverts commit 0db310586ac3e15be95d2ef27ff40cfa24c05c10. Closes #23686. Conflicts: railties/test/application/test_runner_test.rb It's possible that the `result.location` returned by minitest is outside the test file itself. For example in the case of mocha. This resulted in bad rerun snipptets: ``` bin/rails test app/models/deliveries/delivery.rb:103 ``` Let's always use the first line of the failed test-case in our rerun snippet. We can display the line number of the assertion error elsewhere.
* Fix mixing line filters with Minitest's -n filter.Kasper Timm Hansen2016-02-051-0/+24
| | | | | | | | | | | Previous commit accidentally broke mixing line filters with string -n filter. Fix by checking if it is a string and returning it. We also need to ensure the -n filter carry forward into any other composite filters. Fix by letting the named filter be extractable, so we'll keep this for the next runnable's run.
* Fix model test path typo uncovered in previous commit.Kasper Timm Hansen2016-02-031-1/+1
| | | | | | | | | | | | Because of the expanding whitelist for test filters, this test ended up running the tests on lines 4 and 9 in the post test even though the path wasn't right. Happened incidentally because the same line numbers were used in both account and post test. Add the .rb line so the file is required correctly and the filters are applied.
* Fix line filters running tests from multiple runnables.Kasper Timm Hansen2016-02-031-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | `derive_regexp` was written with the assumption that we were run from a blank slate — that if the filter didn't match we might as well return it because it was nil. This isn't the case because minitest calls `run` on every runnable. Which is any subclass of Minitest::Runnable, such as ActiveSupport::TestCase, ActionDispatch::IntegrationTest as well as any inheriting from those. Thus after the first `run` we'd have put in a composite filter in `options[:filter]` making the next `run` create a linked list when it failed to match the regexp and put the composite filter as the head. Every runnable would accumulate more and more of the same filters, which effectively acted like an expanding whitelist and we ran tests from other runnables. Clog the accumulation by returning nil if there's no filter to derive a regexp from. Note: we pass a seed in the tests because Minitest shuffles the runnables to ensure the whitelist is expanded enough that the failure is triggered.
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-01 11:37:34 +0000