diff options
Diffstat (limited to 'activejob')
50 files changed, 2042 insertions, 392 deletions
diff --git a/activejob/CHANGELOG.md b/activejob/CHANGELOG.md index a3d13ad162..138c8a8ef4 100644 --- a/activejob/CHANGELOG.md +++ b/activejob/CHANGELOG.md @@ -1,3 +1,113 @@ +* Make job argument assertions with `Time`, `ActiveSupport::TimeWithZone`, and `DateTime` work by dropping microseconds. Microsecond precision is lost during serialization. + + *Gannon McGibbon* + + +## Rails 6.0.0.beta3 (March 11, 2019) ## + +* No changes. + + +## Rails 6.0.0.beta2 (February 25, 2019) ## + +* No changes. + + +## Rails 6.0.0.beta1 (January 18, 2019) ## + +* Return false instead of the job instance when `enqueue` is aborted. + + This will be the behavior in Rails 6.1 but it can be controlled now with + `config.active_job.return_false_on_aborted_enqueue`. + + *Kir Shatrov* + +* Keep executions for each specific declaration + + Each `retry_on` declaration has now its own specific executions counter. Before it was + shared between all executions of a job. + + *Alberto Almagro* + +* Allow all assertion helpers that have a `only` and `except` keyword to accept + Procs. + + *Edouard Chin* + +* Restore `HashWithIndifferentAccess` support to `ActiveJob::Arguments.deserialize`. + + *Gannon McGibbon* + +* Include deserialized arguments in job instances returned from + `assert_enqueued_with` and `assert_performed_with` + + *Alan Wu* + +* Allow `assert_enqueued_with`/`assert_performed_with` methods to accept + a proc for the `args` argument. This is useful to check if only a subset of arguments + matches your expectations. + + *Edouard Chin* + +* `ActionDispatch::IntegrationTest` includes `ActiveJob::TestHelper` module by default. + + *Ricardo Díaz* + +* Added `enqueue_retry.active_job`, `retry_stopped.active_job`, and `discard.active_job` hooks. + + *steves* + +* Allow `assert_performed_with` to be called without a block. + + *bogdanvlviv* + +* Execution of `assert_performed_jobs`, and `assert_no_performed_jobs` + without a block should respect passed `:except`, `:only`, and `:queue` options. + + *bogdanvlviv* + +* Allow `:queue` option to job assertions and helpers. + + *bogdanvlviv* + +* Allow `perform_enqueued_jobs` to be called without a block. + + Performs all of the jobs that have been enqueued up to this point in the test. + + *Kevin Deisz* + +* Move `enqueue`/`enqueue_at` notifications to an around callback. + + Improves timing accuracy over the old after callback by including + time spent writing to the adapter's IO implementation. + + *Zach Kemp* + +* Allow call `assert_enqueued_with` with no block. + + Example: + ``` + def test_assert_enqueued_with + MyJob.perform_later(1,2,3) + assert_enqueued_with(job: MyJob, args: [1,2,3], queue: 'low') + + MyJob.set(wait_until: Date.tomorrow.noon).perform_later + assert_enqueued_with(job: MyJob, at: Date.tomorrow.noon) + end + ``` + + *bogdanvlviv* + +* Allow passing multiple exceptions to `retry_on`, and `discard_on`. + + *George Claghorn* + +* Pass the error instance as the second parameter of block executed by `discard_on`. + + Fixes #32853. + + *Yuji Yaginuma* + * Remove support for Qu gem. Reasons are that the Qu gem wasn't compatible since Rails 5.1, @@ -14,9 +124,9 @@ *Andrew White* -* Rails 6 requires Ruby 2.4.1 or newer. +* Rails 6 requires Ruby 2.5.0 or newer. - *Jeremy Daer* + *Jeremy Daer*, *Kasper Timm Hansen* * Add support to define custom argument serializers. diff --git a/activejob/MIT-LICENSE b/activejob/MIT-LICENSE index 274211f710..aedc21bca2 100644 --- a/activejob/MIT-LICENSE +++ b/activejob/MIT-LICENSE @@ -1,4 +1,4 @@ -Copyright (c) 2014-2018 David Heinemeier Hansson +Copyright (c) 2014-2019 David Heinemeier Hansson Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the diff --git a/activejob/README.md b/activejob/README.md index d49fcfe3c2..462d319992 100644 --- a/activejob/README.md +++ b/activejob/README.md @@ -1,7 +1,7 @@ -# Active Job -- Make work happen later +# Active Job – Make work happen later Active Job is a framework for declaring jobs and making them run on a variety -of queueing backends. These jobs can be everything from regularly scheduled +of queuing backends. These jobs can be everything from regularly scheduled clean-ups, to billing charges, to mailings. Anything that can be chopped up into small units of work and run in parallel, really. @@ -17,12 +17,13 @@ about API differences between Delayed Job and Resque. Picking your queuing backend becomes more of an operational concern, then. And you'll be able to switch between them without having to rewrite your jobs. +You can read more about Active Job in the [Active Job Basics](https://edgeguides.rubyonrails.org/active_job_basics.html) guide. ## Usage -To learn how to use your preferred queueing backend see its adapter +To learn how to use your preferred queuing backend see its adapter documentation at -[ActiveJob::QueueAdapters](http://api.rubyonrails.org/classes/ActiveJob/QueueAdapters.html). +[ActiveJob::QueueAdapters](https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters.html). Declare a job like so: @@ -39,7 +40,7 @@ end Enqueue a job like so: ```ruby -MyJob.perform_later record # Enqueue a job to be performed as soon as the queueing system is free. +MyJob.perform_later record # Enqueue a job to be performed as soon as the queuing system is free. ``` ```ruby @@ -82,11 +83,11 @@ This works with any class that mixes in GlobalID::Identification, which by default has been mixed into Active Record classes. -## Supported queueing systems +## Supported queuing systems -Active Job has built-in adapters for multiple queueing backends (Sidekiq, +Active Job has built-in adapters for multiple queuing backends (Sidekiq, Resque, Delayed Job and others). To get an up-to-date list of the adapters -see the API Documentation for [ActiveJob::QueueAdapters](http://api.rubyonrails.org/classes/ActiveJob/QueueAdapters.html). +see the API Documentation for [ActiveJob::QueueAdapters](https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters.html). **Please note:** We are not accepting pull requests for new adapters. We encourage library authors to provide an ActiveJob adapter as part of @@ -121,7 +122,7 @@ Active Job is released under the MIT license: API documentation is at: -* http://api.rubyonrails.org +* https://api.rubyonrails.org Bug reports for the Ruby on Rails project can be filed here: diff --git a/activejob/Rakefile b/activejob/Rakefile index b4da75adab..037e84fca9 100644 --- a/activejob/Rakefile +++ b/activejob/Rakefile @@ -28,6 +28,7 @@ namespace :test do task "env:integration" do ENV["AJ_INTEGRATION_TESTS"] = "1" + ENV["SKIP_REQUIRE_WEBPACKER"] = "true" end ACTIVEJOB_ADAPTERS.each do |adapter| @@ -38,7 +39,7 @@ namespace :test do t.libs << "test" t.test_files = FileList["test/cases/**/*_test.rb"] t.verbose = true - t.warning = false + t.warning = true t.ruby_opts = ["--dev"] if defined?(JRUBY_VERSION) end @@ -56,7 +57,7 @@ namespace :test do t.libs << "test" t.test_files = FileList["test/integration/**/*_test.rb"] t.verbose = true - t.warning = false + t.warning = true t.ruby_opts = ["--dev"] if defined?(JRUBY_VERSION) end end @@ -67,11 +68,9 @@ def run_without_aborting(tasks) errors = [] tasks.each do |task| - begin - Rake::Task[task].invoke - rescue Exception - errors << task - end + Rake::Task[task].invoke + rescue Exception + errors << task end abort "Errors running #{errors.join(', ')}" if errors.any? diff --git a/activejob/activejob.gemspec b/activejob/activejob.gemspec index be6292f737..aeffe55af6 100644 --- a/activejob/activejob.gemspec +++ b/activejob/activejob.gemspec @@ -7,15 +7,15 @@ Gem::Specification.new do |s| s.name = "activejob" s.version = version s.summary = "Job framework with pluggable queues." - s.description = "Declare job classes that can be run by a variety of queueing backends." + s.description = "Declare job classes that can be run by a variety of queuing backends." - s.required_ruby_version = ">= 2.4.1" + s.required_ruby_version = ">= 2.5.0" s.license = "MIT" s.author = "David Heinemeier Hansson" s.email = "david@loudthinking.com" - s.homepage = "http://rubyonrails.org" + s.homepage = "https://rubyonrails.org" s.files = Dir["CHANGELOG.md", "MIT-LICENSE", "README.md", "lib/**/*"] s.require_path = "lib" @@ -25,6 +25,9 @@ Gem::Specification.new do |s| "changelog_uri" => "https://github.com/rails/rails/blob/v#{version}/activejob/CHANGELOG.md" } + # NOTE: Please read our dependency guidelines before updating versions: + # https://edgeguides.rubyonrails.org/security.html#dependency-management-and-cves + s.add_dependency "activesupport", version s.add_dependency "globalid", ">= 0.3.6" end diff --git a/activejob/lib/active_job.rb b/activejob/lib/active_job.rb index 01fab4d918..5f20ef9b9f 100644 --- a/activejob/lib/active_job.rb +++ b/activejob/lib/active_job.rb @@ -1,7 +1,7 @@ # frozen_string_literal: true #-- -# Copyright (c) 2014-2018 David Heinemeier Hansson +# Copyright (c) 2014-2019 David Heinemeier Hansson # # Permission is hereby granted, free of charge, to any person obtaining # a copy of this software and associated documentation files (the diff --git a/activejob/lib/active_job/arguments.rb b/activejob/lib/active_job/arguments.rb index 86bb0c5540..92eb58aaaf 100644 --- a/activejob/lib/active_job/arguments.rb +++ b/activejob/lib/active_job/arguments.rb @@ -14,28 +14,30 @@ module ActiveJob end # Raised when an unsupported argument type is set as a job argument. We - # currently support NilClass, Integer, Float, String, TrueClass, FalseClass, - # BigDecimal, and objects that can be represented as GlobalIDs (ex: Active Record). + # currently support String, Integer, Float, NilClass, TrueClass, FalseClass, + # BigDecimal, Symbol, Date, Time, DateTime, ActiveSupport::TimeWithZone, + # ActiveSupport::Duration, Hash, ActiveSupport::HashWithIndifferentAccess, + # Array or GlobalID::Identification instances, although this can be extended + # by adding custom serializers. # Raised if you set the key for a Hash something else than a string or # a symbol. Also raised when trying to serialize an object which can't be - # identified with a Global ID - such as an unpersisted Active Record model. + # identified with a GlobalID - such as an unpersisted Active Record model. class SerializationError < ArgumentError; end module Arguments extend self - # :nodoc: - TYPE_WHITELIST = [ NilClass, String, Integer, Float, BigDecimal, TrueClass, FalseClass ] - - # Serializes a set of arguments. Whitelisted types are returned - # as-is. Arrays/Hashes are serialized element by element. - # All other types are serialized using GlobalID. + # Serializes a set of arguments. Intrinsic types that can safely be + # serialized without mutation are returned as-is. Arrays/Hashes are + # serialized element by element. All other types are serialized using + # GlobalID. def serialize(arguments) arguments.map { |argument| serialize_argument(argument) } end - # Deserializes a set of arguments. Whitelisted types are returned - # as-is. Arrays/Hashes are deserialized element by element. - # All other types are deserialized using GlobalID. + # Deserializes a set of arguments. Intrinsic types that can safely be + # deserialized without mutation are returned as-is. Arrays/Hashes are + # deserialized element by element. All other types are deserialized using + # GlobalID. def deserialize(arguments) arguments.map { |argument| deserialize_argument(argument) } rescue @@ -45,11 +47,13 @@ module ActiveJob private # :nodoc: - GLOBALID_KEY = "_aj_globalid".freeze + PERMITTED_TYPES = [ NilClass, String, Integer, Float, BigDecimal, TrueClass, FalseClass ] + # :nodoc: + GLOBALID_KEY = "_aj_globalid" # :nodoc: - SYMBOL_KEYS_KEY = "_aj_symbol_keys".freeze + SYMBOL_KEYS_KEY = "_aj_symbol_keys" # :nodoc: - WITH_INDIFFERENT_ACCESS_KEY = "_aj_hash_with_indifferent_access".freeze + WITH_INDIFFERENT_ACCESS_KEY = "_aj_hash_with_indifferent_access" # :nodoc: OBJECT_SERIALIZER_KEY = "_aj_serialized" @@ -60,25 +64,25 @@ module ActiveJob OBJECT_SERIALIZER_KEY, OBJECT_SERIALIZER_KEY.to_sym, WITH_INDIFFERENT_ACCESS_KEY, WITH_INDIFFERENT_ACCESS_KEY.to_sym, ] - private_constant :RESERVED_KEYS + private_constant :PERMITTED_TYPES, :RESERVED_KEYS, :GLOBALID_KEY, :SYMBOL_KEYS_KEY, :WITH_INDIFFERENT_ACCESS_KEY def serialize_argument(argument) case argument - when *TYPE_WHITELIST + when *PERMITTED_TYPES argument when GlobalID::Identification convert_to_global_id_hash(argument) when Array argument.map { |arg| serialize_argument(arg) } when ActiveSupport::HashWithIndifferentAccess - result = serialize_hash(argument) - result[WITH_INDIFFERENT_ACCESS_KEY] = serialize_argument(true) - result + serialize_indifferent_hash(argument) when Hash symbol_keys = argument.each_key.grep(Symbol).map(&:to_s) result = serialize_hash(argument) result[SYMBOL_KEYS_KEY] = symbol_keys result + when -> (arg) { arg.respond_to?(:permitted?) } + serialize_indifferent_hash(argument.to_h) else Serializers.serialize(argument) end @@ -87,8 +91,8 @@ module ActiveJob def deserialize_argument(argument) case argument when String - GlobalID::Locator.locate(argument) || argument - when *TYPE_WHITELIST + argument + when *PERMITTED_TYPES argument when Array argument.map { |arg| deserialize_argument(arg) } @@ -144,8 +148,17 @@ module ActiveJob end end + def serialize_indifferent_hash(indifferent_hash) + result = serialize_hash(indifferent_hash) + result[WITH_INDIFFERENT_ACCESS_KEY] = serialize_argument(true) + result + end + def transform_symbol_keys(hash, symbol_keys) - hash.transform_keys do |key| + # NOTE: HashWithIndifferentAccess#transform_keys always + # returns stringified keys with indifferent access + # so we call #to_h here to ensure keys are symbolized. + hash.to_h.transform_keys do |key| if symbol_keys.include?(key) key.to_sym else diff --git a/activejob/lib/active_job/base.rb b/activejob/lib/active_job/base.rb index 2b2a59e969..ed41fac4b8 100644 --- a/activejob/lib/active_job/base.rb +++ b/activejob/lib/active_job/base.rb @@ -40,7 +40,7 @@ module ActiveJob #:nodoc: # Records that are passed in are serialized/deserialized using Global # ID. More information can be found in Arguments. # - # To enqueue a job to be performed as soon as the queueing system is free: + # To enqueue a job to be performed as soon as the queuing system is free: # # ProcessPhotoJob.perform_later(photo) # @@ -60,7 +60,6 @@ module ActiveJob #:nodoc: # * SerializationError - Error class for serialization errors. class Base include Core - include Serializers include QueueAdapter include QueueName include QueuePriority diff --git a/activejob/lib/active_job/callbacks.rb b/activejob/lib/active_job/callbacks.rb index 334b24fb3b..6b82ea6cab 100644 --- a/activejob/lib/active_job/callbacks.rb +++ b/activejob/lib/active_job/callbacks.rb @@ -29,6 +29,9 @@ module ActiveJob included do define_callbacks :perform define_callbacks :enqueue + + class_attribute :return_false_on_aborted_enqueue, instance_accessor: false, instance_predicate: false + self.return_false_on_aborted_enqueue = false end # These methods will be included into any Active Job object, adding @@ -130,7 +133,7 @@ module ActiveJob set_callback(:enqueue, :after, *filters, &blk) end - # Defines a callback that will get called around the enqueueing + # Defines a callback that will get called around the enqueuing # of the job. # # class VideoProcessJob < ActiveJob::Base diff --git a/activejob/lib/active_job/core.rb b/activejob/lib/active_job/core.rb index da841ae45b..283125698d 100644 --- a/activejob/lib/active_job/core.rb +++ b/activejob/lib/active_job/core.rb @@ -6,35 +6,42 @@ module ActiveJob module Core extend ActiveSupport::Concern - included do - # Job arguments - attr_accessor :arguments - attr_writer :serialized_arguments + # Job arguments + attr_accessor :arguments + attr_writer :serialized_arguments - # Timestamp when the job should be performed - attr_accessor :scheduled_at + # Timestamp when the job should be performed + attr_accessor :scheduled_at - # Job Identifier - attr_accessor :job_id + # Job Identifier + attr_accessor :job_id - # Queue in which the job will reside. - attr_writer :queue_name + # Queue in which the job will reside. + attr_writer :queue_name - # Priority that the job will have (lower is more priority). - attr_writer :priority + # Priority that the job will have (lower is more priority). + attr_writer :priority - # ID optionally provided by adapter - attr_accessor :provider_job_id + # ID optionally provided by adapter + attr_accessor :provider_job_id - # Number of times this job has been executed (which increments on every retry, like after an exception). - attr_accessor :executions + # Number of times this job has been executed (which increments on every retry, like after an exception). + attr_accessor :executions - # I18n.locale to be used during the job. - attr_accessor :locale + # Hash that contains the number of times this job handled errors for each specific retry_on declaration. + # Keys are the string representation of the exceptions listed in the retry_on declaration, + # while its associated value holds the number of executions where the corresponding retry_on + # declaration handled one of its listed exceptions. + attr_accessor :exception_executions - # Timezone to be used during the job. - attr_accessor :timezone - end + # I18n.locale to be used during the job. + attr_accessor :locale + + # Timezone to be used during the job. + attr_accessor :timezone + + # Track when a job was enqueued + attr_accessor :enqueued_at # These methods will be included into any Active Job object, adding # helpers for de/serialization and creation of job instances. @@ -77,10 +84,11 @@ module ActiveJob @queue_name = self.class.queue_name @priority = self.class.priority @executions = 0 + @exception_executions = {} end # Returns a hash with the job data that can safely be passed to the - # queueing adapter. + # queuing adapter. def serialize { "job_class" => self.class.name, @@ -88,10 +96,12 @@ module ActiveJob "provider_job_id" => provider_job_id, "queue_name" => queue_name, "priority" => priority, - "arguments" => serialize_arguments(arguments), + "arguments" => serialize_arguments_if_needed(arguments), "executions" => executions, + "exception_executions" => exception_executions, "locale" => I18n.locale.to_s, - "timezone" => Time.zone.try(:name) + "timezone" => Time.zone.try(:name), + "enqueued_at" => Time.now.utc.iso8601 } end @@ -128,24 +138,38 @@ module ActiveJob self.priority = job_data["priority"] self.serialized_arguments = job_data["arguments"] self.executions = job_data["executions"] + self.exception_executions = job_data["exception_executions"] self.locale = job_data["locale"] || I18n.locale.to_s self.timezone = job_data["timezone"] || Time.zone.try(:name) + self.enqueued_at = job_data["enqueued_at"] end private + def serialize_arguments_if_needed(arguments) + if arguments_serialized? + @serialized_arguments + else + serialize_arguments(arguments) + end + end + def deserialize_arguments_if_needed - if defined?(@serialized_arguments) && @serialized_arguments.present? + if arguments_serialized? @arguments = deserialize_arguments(@serialized_arguments) @serialized_arguments = nil end end - def serialize_arguments(serialized_args) - Arguments.serialize(serialized_args) + def serialize_arguments(arguments) + Arguments.serialize(arguments) end def deserialize_arguments(serialized_args) Arguments.deserialize(serialized_args) end + + def arguments_serialized? + defined?(@serialized_arguments) && @serialized_arguments + end end end diff --git a/activejob/lib/active_job/enqueuing.rb b/activejob/lib/active_job/enqueuing.rb index 53cb98fc71..c5ee76a69e 100644 --- a/activejob/lib/active_job/enqueuing.rb +++ b/activejob/lib/active_job/enqueuing.rb @@ -9,10 +9,12 @@ module ActiveJob # Includes the +perform_later+ method for job initialization. module ClassMethods - # Push a job onto the queue. The arguments must be legal JSON types - # (+string+, +int+, +float+, +nil+, +true+, +false+, +hash+ or +array+) or - # GlobalID::Identification instances. Arbitrary Ruby objects - # are not supported. + # Push a job onto the queue. By default the arguments must be either String, + # Integer, Float, NilClass, TrueClass, FalseClass, BigDecimal, Symbol, Date, + # Time, DateTime, ActiveSupport::TimeWithZone, ActiveSupport::Duration, + # Hash, ActiveSupport::HashWithIndifferentAccess, Array or + # GlobalID::Identification instances, although this can be extended by adding + # custom serializers. # # Returns an instance of the job class queued with arguments available in # Job#arguments. @@ -46,14 +48,33 @@ module ActiveJob self.scheduled_at = options[:wait_until].to_f if options[:wait_until] self.queue_name = self.class.queue_name_from_part(options[:queue]) if options[:queue] self.priority = options[:priority].to_i if options[:priority] + successfully_enqueued = false + run_callbacks :enqueue do if scheduled_at self.class.queue_adapter.enqueue_at self, scheduled_at else self.class.queue_adapter.enqueue self end + + successfully_enqueued = true + end + + if successfully_enqueued + self + else + if self.class.return_false_on_aborted_enqueue + false + else + ActiveSupport::Deprecation.warn( + "Rails 6.1 will return false when the enqueuing is aborted. Make sure your code doesn't depend on it" \ + " returning the instance of the job and set `config.active_job.return_false_on_aborted_enqueue = true`" \ + " to remove the deprecations." + ) + + self + end end - self end end end diff --git a/activejob/lib/active_job/exceptions.rb b/activejob/lib/active_job/exceptions.rb index ae700848d0..9e00942a1c 100644 --- a/activejob/lib/active_job/exceptions.rb +++ b/activejob/lib/active_job/exceptions.rb @@ -30,28 +30,38 @@ module ActiveJob # class RemoteServiceJob < ActiveJob::Base # retry_on CustomAppException # defaults to 3s wait, 5 attempts # retry_on AnotherCustomAppException, wait: ->(executions) { executions * 2 } - # retry_on(YetAnotherCustomAppException) do |job, exception| - # ExceptionNotifier.caught(exception) - # end + # # retry_on ActiveRecord::Deadlocked, wait: 5.seconds, attempts: 3 - # retry_on Net::OpenTimeout, wait: :exponentially_longer, attempts: 10 + # retry_on Net::OpenTimeout, Timeout::Error, wait: :exponentially_longer, attempts: 10 # retries at most 10 times for Net::OpenTimeout and Timeout::Error combined + # # To retry at most 10 times for each individual exception: + # # retry_on Net::OpenTimeout, wait: :exponentially_longer, attempts: 10 + # # retry_on Timeout::Error, wait: :exponentially_longer, attempts: 10 + # + # retry_on(YetAnotherCustomAppException) do |job, error| + # ExceptionNotifier.caught(error) + # end # # def perform(*args) # # Might raise CustomAppException, AnotherCustomAppException, or YetAnotherCustomAppException for something domain specific # # Might raise ActiveRecord::Deadlocked when a local db deadlock is detected - # # Might raise Net::OpenTimeout when the remote service is down + # # Might raise Net::OpenTimeout or Timeout::Error when the remote service is down # end # end - def retry_on(exception, wait: 3.seconds, attempts: 5, queue: nil, priority: nil) - rescue_from exception do |error| - if executions < attempts - logger.error "Retrying #{self.class} in #{wait} seconds, due to a #{exception}. The original exception was #{error.cause.inspect}." - retry_job wait: determine_delay(wait), queue: queue, priority: priority + def retry_on(*exceptions, wait: 3.seconds, attempts: 5, queue: nil, priority: nil) + rescue_from(*exceptions) do |error| + # Guard against jobs that were persisted before we started having individual executions counters per retry_on + self.exception_executions ||= {} + self.exception_executions[exceptions.to_s] = (exception_executions[exceptions.to_s] || 0) + 1 + + if exception_executions[exceptions.to_s] < attempts + retry_job wait: determine_delay(wait), queue: queue, priority: priority, error: error else if block_given? - yield self, error + instrument :retry_stopped, error: error do + yield self, error + end else - logger.error "Stopped retrying #{self.class} due to a #{exception}, which reoccurred on #{executions} attempts. The original exception was #{error.cause.inspect}." + instrument :retry_stopped, error: error raise error end end @@ -67,8 +77,8 @@ module ActiveJob # # class SearchIndexingJob < ActiveJob::Base # discard_on ActiveJob::DeserializationError - # discard_on(CustomAppException) do |job, exception| - # ExceptionNotifier.caught(exception) + # discard_on(CustomAppException) do |job, error| + # ExceptionNotifier.caught(error) # end # # def perform(record) @@ -76,12 +86,10 @@ module ActiveJob # # Might raise CustomAppException for something domain specific # end # end - def discard_on(exception) - rescue_from exception do |error| - if block_given? - yield self, exception - else - logger.error "Discarded #{self.class} due to a #{exception}. The original exception was #{error.cause.inspect}." + def discard_on(*exceptions) + rescue_from(*exceptions) do |error| + instrument :discard, error: error do + yield self, error if block_given? end end end @@ -109,7 +117,9 @@ module ActiveJob # end # end def retry_job(options = {}) - enqueue options + instrument :enqueue_retry, options.slice(:error, :wait) do + enqueue options + end end private @@ -130,5 +140,11 @@ module ActiveJob raise "Couldn't determine a delay based on #{seconds_or_duration_or_algorithm.inspect}" end end + + def instrument(name, error: nil, wait: nil, &block) + payload = { job: self, adapter: self.class.queue_adapter, error: error, wait: wait } + + ActiveSupport::Notifications.instrument("#{name}.active_job", payload, &block) + end end end diff --git a/activejob/lib/active_job/execution.rb b/activejob/lib/active_job/execution.rb index d75be376ec..e96dbcd4c9 100644 --- a/activejob/lib/active_job/execution.rb +++ b/activejob/lib/active_job/execution.rb @@ -26,16 +26,16 @@ module ActiveJob end end - # Performs the job immediately. The job is not sent to the queueing adapter + # Performs the job immediately. The job is not sent to the queuing adapter # but directly executed by blocking the execution of others until it's finished. # # MyJob.new(*args).perform_now def perform_now + # Guard against jobs that were persisted before we started counting executions by zeroing out nil counters + self.executions = (executions || 0) + 1 + deserialize_arguments_if_needed run_callbacks :perform do - # Guard against jobs that were persisted before we started counting executions by zeroing out nil counters - self.executions = (executions || 0) + 1 - perform(*arguments) end rescue => exception diff --git a/activejob/lib/active_job/gem_version.rb b/activejob/lib/active_job/gem_version.rb index 770f70dc5e..5313a4ab9e 100644 --- a/activejob/lib/active_job/gem_version.rb +++ b/activejob/lib/active_job/gem_version.rb @@ -10,7 +10,7 @@ module ActiveJob MAJOR = 6 MINOR = 0 TINY = 0 - PRE = "alpha" + PRE = "beta3" STRING = [MAJOR, MINOR, TINY, PRE].compact.join(".") end diff --git a/activejob/lib/active_job/logging.rb b/activejob/lib/active_job/logging.rb index 3312857ac7..1134e718a8 100644 --- a/activejob/lib/active_job/logging.rb +++ b/activejob/lib/active_job/logging.rb @@ -27,13 +27,13 @@ module ActiveJob end end - after_enqueue do |job| + around_enqueue do |job, block| if job.scheduled_at - ActiveSupport::Notifications.instrument "enqueue_at.active_job", - adapter: job.class.queue_adapter, job: job + ActiveSupport::Notifications.instrument("enqueue_at.active_job", + adapter: job.class.queue_adapter, job: job, &block) else - ActiveSupport::Notifications.instrument "enqueue.active_job", - adapter: job.class.queue_adapter, job: job + ActiveSupport::Notifications.instrument("enqueue.active_job", + adapter: job.class.queue_adapter, job: job, &block) end end end @@ -70,7 +70,7 @@ module ActiveJob def perform_start(event) info do job = event.payload[:job] - "Performing #{job.class.name} (Job ID: #{job.job_id}) from #{queue_name(event)}" + args_info(job) + "Performing #{job.class.name} (Job ID: #{job.job_id}) from #{queue_name(event)} enqueued at #{job.enqueued_at}" + args_info(job) end end @@ -88,6 +88,38 @@ module ActiveJob end end + def enqueue_retry(event) + job = event.payload[:job] + ex = event.payload[:error] + wait = event.payload[:wait] + + info do + if ex + "Retrying #{job.class} in #{wait.to_i} seconds, due to a #{ex.class}." + else + "Retrying #{job.class} in #{wait.to_i} seconds." + end + end + end + + def retry_stopped(event) + job = event.payload[:job] + ex = event.payload[:error] + + error do + "Stopped retrying #{job.class} due to a #{ex.class}, which reoccurred on #{job.executions} attempts." + end + end + + def discard(event) + job = event.payload[:job] + ex = event.payload[:error] + + error do + "Discarded #{job.class} due to a #{ex.class}." + end + end + private def queue_name(event) event.payload[:adapter].class.name.demodulize.remove("Adapter") + "(#{event.payload[:job].queue_name})" diff --git a/activejob/lib/active_job/queue_adapter.rb b/activejob/lib/active_job/queue_adapter.rb index 006a683b85..954bfd1dd1 100644 --- a/activejob/lib/active_job/queue_adapter.rb +++ b/activejob/lib/active_job/queue_adapter.rb @@ -22,6 +22,8 @@ module ActiveJob _queue_adapter end + # Returns string denoting the name of the configured queue adapter. + # By default returns +"async"+. def queue_adapter_name _queue_adapter_name end diff --git a/activejob/lib/active_job/queue_adapters.rb b/activejob/lib/active_job/queue_adapters.rb index 7854467cc1..a4b7eb86f1 100644 --- a/activejob/lib/active_job/queue_adapters.rb +++ b/activejob/lib/active_job/queue_adapters.rb @@ -3,19 +3,19 @@ module ActiveJob # == Active Job adapters # - # Active Job has adapters for the following queueing backends: + # Active Job has adapters for the following queuing backends: # # * {Backburner}[https://github.com/nesquena/backburner] # * {Delayed Job}[https://github.com/collectiveidea/delayed_job] # * {Que}[https://github.com/chanks/que] # * {queue_classic}[https://github.com/QueueClassic/queue_classic] # * {Resque}[https://github.com/resque/resque] - # * {Sidekiq}[http://sidekiq.org] + # * {Sidekiq}[https://sidekiq.org] # * {Sneakers}[https://github.com/jondot/sneakers] # * {Sucker Punch}[https://github.com/brandonhilkert/sucker_punch] - # * {Active Job Async Job}[http://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/AsyncAdapter.html] - # * {Active Job Inline}[http://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/InlineAdapter.html] - # * Please Note: We are not accepting pull requests for new adapters. See the README for more details. + # * {Active Job Async Job}[https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/AsyncAdapter.html] + # * {Active Job Inline}[https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/InlineAdapter.html] + # * Please Note: We are not accepting pull requests for new adapters. See the {README}[link:files/activejob/README_md.html] for more details. # # === Backends Features # @@ -52,7 +52,7 @@ module ActiveJob # # No: The adapter will run jobs at the next opportunity and cannot use perform_later. # - # N/A: The adapter does not support queueing. + # N/A: The adapter does not support queuing. # # NOTE: # queue_classic supports job scheduling since version 3.1. @@ -74,7 +74,7 @@ module ActiveJob # # No: Does not allow the priority of jobs to be configured. # - # N/A: The adapter does not support queueing, and therefore sorting them. + # N/A: The adapter does not support queuing, and therefore sorting them. # # ==== Timeout # @@ -121,7 +121,7 @@ module ActiveJob autoload :SuckerPunchAdapter autoload :TestAdapter - ADAPTER = "Adapter".freeze + ADAPTER = "Adapter" private_constant :ADAPTER class << self diff --git a/activejob/lib/active_job/queue_adapters/async_adapter.rb b/activejob/lib/active_job/queue_adapters/async_adapter.rb index ebf6f384e3..53a7e3d53e 100644 --- a/activejob/lib/active_job/queue_adapters/async_adapter.rb +++ b/activejob/lib/active_job/queue_adapters/async_adapter.rb @@ -31,7 +31,7 @@ module ActiveJob # jobs. Since jobs share a single thread pool, long-running jobs will block # short-lived jobs. Fine for dev/test; bad for production. class AsyncAdapter - # See {Concurrent::ThreadPoolExecutor}[https://ruby-concurrency.github.io/concurrent-ruby/Concurrent/ThreadPoolExecutor.html] for executor options. + # See {Concurrent::ThreadPoolExecutor}[https://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/ThreadPoolExecutor.html] for executor options. def initialize(**executor_options) @scheduler = Scheduler.new(**executor_options) end diff --git a/activejob/lib/active_job/queue_adapters/backburner_adapter.rb b/activejob/lib/active_job/queue_adapters/backburner_adapter.rb index 0ba93c6e0b..7dc49310ac 100644 --- a/activejob/lib/active_job/queue_adapters/backburner_adapter.rb +++ b/activejob/lib/active_job/queue_adapters/backburner_adapter.rb @@ -16,12 +16,12 @@ module ActiveJob # Rails.application.config.active_job.queue_adapter = :backburner class BackburnerAdapter def enqueue(job) #:nodoc: - Backburner::Worker.enqueue JobWrapper, [ job.serialize ], queue: job.queue_name + Backburner::Worker.enqueue(JobWrapper, [job.serialize], queue: job.queue_name, pri: job.priority) end def enqueue_at(job, timestamp) #:nodoc: delay = timestamp - Time.current.to_f - Backburner::Worker.enqueue JobWrapper, [ job.serialize ], queue: job.queue_name, delay: delay + Backburner::Worker.enqueue(JobWrapper, [job.serialize], queue: job.queue_name, pri: job.priority, delay: delay) end class JobWrapper #:nodoc: diff --git a/activejob/lib/active_job/queue_adapters/inline_adapter.rb b/activejob/lib/active_job/queue_adapters/inline_adapter.rb index 3d0b590212..ca04dc943c 100644 --- a/activejob/lib/active_job/queue_adapters/inline_adapter.rb +++ b/activejob/lib/active_job/queue_adapters/inline_adapter.rb @@ -16,7 +16,7 @@ module ActiveJob end def enqueue_at(*) #:nodoc: - raise NotImplementedError, "Use a queueing backend to enqueue jobs in the future. Read more at http://guides.rubyonrails.org/active_job_basics.html" + raise NotImplementedError, "Use a queueing backend to enqueue jobs in the future. Read more at https://guides.rubyonrails.org/active_job_basics.html" end end end diff --git a/activejob/lib/active_job/queue_adapters/test_adapter.rb b/activejob/lib/active_job/queue_adapters/test_adapter.rb index 885f9ff01c..c134257ebc 100644 --- a/activejob/lib/active_job/queue_adapters/test_adapter.rb +++ b/activejob/lib/active_job/queue_adapters/test_adapter.rb @@ -12,7 +12,7 @@ module ActiveJob # # Rails.application.config.active_job.queue_adapter = :test class TestAdapter - attr_accessor(:perform_enqueued_jobs, :perform_enqueued_at_jobs, :filter, :reject) + attr_accessor(:perform_enqueued_jobs, :perform_enqueued_at_jobs, :filter, :reject, :queue) attr_writer(:enqueued_jobs, :performed_jobs) # Provides a store of all the enqueued jobs with the TestAdapter so you can check them. @@ -29,14 +29,14 @@ module ActiveJob return if filtered?(job) job_data = job_to_hash(job) - enqueue_or_perform(perform_enqueued_jobs, job, job_data) + perform_or_enqueue(perform_enqueued_jobs, job, job_data) end def enqueue_at(job, timestamp) #:nodoc: return if filtered?(job) job_data = job_to_hash(job, at: timestamp) - enqueue_or_perform(perform_enqueued_at_jobs, job, job_data) + perform_or_enqueue(perform_enqueued_at_jobs, job, job_data) end private @@ -44,7 +44,7 @@ module ActiveJob { job: job.class, args: job.serialize.fetch("arguments"), queue: job.queue_name }.merge!(extras) end - def enqueue_or_perform(perform, job, job_data) + def perform_or_enqueue(perform, job, job_data) if perform performed_jobs << job_data Base.execute job.serialize @@ -54,14 +54,28 @@ module ActiveJob end def filtered?(job) + filtered_queue?(job) || filtered_job_class?(job) + end + + def filtered_queue?(job) + if queue + job.queue_name != queue.to_s + end + end + + def filtered_job_class?(job) if filter - !Array(filter).include?(job.class) + !filter_as_proc(filter).call(job) elsif reject - Array(reject).include?(job.class) - else - false + filter_as_proc(reject).call(job) end end + + def filter_as_proc(filter) + return filter if filter.is_a?(Proc) + + ->(job) { Array(filter).include?(job.class) } + end end end end diff --git a/activejob/lib/active_job/queue_name.rb b/activejob/lib/active_job/queue_name.rb index 9dc6bc7f2e..de259261de 100644 --- a/activejob/lib/active_job/queue_name.rb +++ b/activejob/lib/active_job/queue_name.rb @@ -18,6 +18,26 @@ module ActiveJob # post.to_feed! # end # end + # + # Can be given a block that will evaluate in the context of the job + # allowing +self.arguments+ to be accessed so that a dynamic queue name + # can be applied: + # + # class PublishToFeedJob < ApplicationJob + # queue_as do + # post = self.arguments.first + # + # if post.paid? + # :paid_feeds + # else + # :feeds + # end + # end + # + # def perform(post) + # post.to_feed! + # end + # end def queue_as(part_name = nil, &block) if block_given? self.queue_name = block @@ -34,7 +54,7 @@ module ActiveJob end included do - class_attribute :queue_name, instance_accessor: false, default: default_queue_name + class_attribute :queue_name, instance_accessor: false, default: -> { self.class.default_queue_name } class_attribute :queue_name_delimiter, instance_accessor: false, default: "_" end diff --git a/activejob/lib/active_job/railtie.rb b/activejob/lib/active_job/railtie.rb index d0294854d3..ecc0908d5f 100644 --- a/activejob/lib/active_job/railtie.rb +++ b/activejob/lib/active_job/railtie.rb @@ -30,6 +30,10 @@ module ActiveJob send(k, v) if respond_to? k end end + + ActiveSupport.on_load(:action_dispatch_integration_test) do + include ActiveJob::TestHelper + end end initializer "active_job.set_reloader_hook" do |app| diff --git a/activejob/lib/active_job/serializers.rb b/activejob/lib/active_job/serializers.rb index df66e66659..a5d90f48b8 100644 --- a/activejob/lib/active_job/serializers.rb +++ b/activejob/lib/active_job/serializers.rb @@ -7,7 +7,6 @@ module ActiveJob # and to add new ones. It also has helpers to serialize/deserialize objects. module Serializers # :nodoc: extend ActiveSupport::Autoload - extend ActiveSupport::Concern autoload :ObjectSerializer autoload :SymbolSerializer diff --git a/activejob/lib/active_job/serializers/object_serializer.rb b/activejob/lib/active_job/serializers/object_serializer.rb index 1dfd1e44be..6d280969be 100644 --- a/activejob/lib/active_job/serializers/object_serializer.rb +++ b/activejob/lib/active_job/serializers/object_serializer.rb @@ -38,7 +38,7 @@ module ActiveJob { Arguments::OBJECT_SERIALIZER_KEY => self.class.name }.merge!(hash) end - # Deserilizes an argument form a JSON primiteve type. + # Deserializes an argument from a JSON primitive type. def deserialize(_argument) raise NotImplementedError end diff --git a/activejob/lib/active_job/test_helper.rb b/activejob/lib/active_job/test_helper.rb index 1cd2c40c15..463020a332 100644 --- a/activejob/lib/active_job/test_helper.rb +++ b/activejob/lib/active_job/test_helper.rb @@ -1,7 +1,6 @@ # frozen_string_literal: true require "active_support/core_ext/class/subclasses" -require "active_support/core_ext/hash/keys" module ActiveJob # Provides helper methods for testing Active Job @@ -52,7 +51,7 @@ module ActiveJob queue_adapter_changed_jobs.each { |klass| klass.disable_test_adapter } end - # Specifies the queue adapter to use with all active job test helpers. + # Specifies the queue adapter to use with all Active Job test helpers. # # Returns an instance of the queue adapter and defaults to # <tt>ActiveJob::QueueAdapters::TestAdapter</tt>. @@ -75,7 +74,7 @@ module ActiveJob # assert_enqueued_jobs 2 # end # - # If a block is passed, that block will cause the specified number of + # If a block is passed, asserts that the block will cause the specified number of # jobs to be enqueued. # # def test_jobs_again @@ -89,7 +88,7 @@ module ActiveJob # end # end # - # The number of times a specific job was enqueued can be asserted. + # Asserts the number of times a specific job was enqueued by passing +:only+ option. # # def test_logging_job # assert_enqueued_jobs 1, only: LoggingJob do @@ -98,7 +97,7 @@ module ActiveJob # end # end # - # The number of times a job except specific class was enqueued can be asserted. + # Asserts the number of times a job except specific class was enqueued by passing +:except+ option. # # def test_logging_job # assert_enqueued_jobs 1, except: HelloJob do @@ -107,7 +106,10 @@ module ActiveJob # end # end # - # The number of times a job is enqueued to a specific queue can also be asserted. + # +:only+ and +:except+ options accepts Class, Array of Class or Proc. When passed a Proc, + # a hash containing the job's class and it's argument are passed as argument. + # + # Asserts the number of times a job is enqueued to a specific queue by passing +:queue+ option. # # def test_logging_job # assert_enqueued_jobs 2, queue: 'default' do @@ -117,14 +119,18 @@ module ActiveJob # end def assert_enqueued_jobs(number, only: nil, except: nil, queue: nil) if block_given? - original_count = enqueued_jobs_size(only: only, except: except, queue: queue) + original_count = enqueued_jobs_with(only: only, except: except, queue: queue) + yield - new_count = enqueued_jobs_size(only: only, except: except, queue: queue) - assert_equal number, new_count - original_count, "#{number} jobs expected, but #{new_count - original_count} were enqueued" + + new_count = enqueued_jobs_with(only: only, except: except, queue: queue) + + actual_count = new_count - original_count else - actual_count = enqueued_jobs_size(only: only, except: except, queue: queue) - assert_equal number, actual_count, "#{number} jobs expected, but #{actual_count} were enqueued" + actual_count = enqueued_jobs_with(only: only, except: except, queue: queue) end + + assert_equal number, actual_count, "#{number} jobs expected, but #{actual_count} were enqueued" end # Asserts that no jobs have been enqueued. @@ -135,7 +141,7 @@ module ActiveJob # assert_enqueued_jobs 1 # end # - # If a block is passed, that block should not cause any job to be enqueued. + # If a block is passed, asserts that the block will not cause any job to be enqueued. # # def test_jobs_again # assert_no_enqueued_jobs do @@ -143,7 +149,7 @@ module ActiveJob # end # end # - # It can be asserted that no jobs of a specific kind are enqueued: + # Asserts that no jobs of a specific kind are enqueued by passing +:only+ option. # # def test_no_logging # assert_no_enqueued_jobs only: LoggingJob do @@ -151,7 +157,7 @@ module ActiveJob # end # end # - # It can be asserted that no jobs except specific class are enqueued: + # Asserts that no jobs except specific class are enqueued by passing +:except+ option. # # def test_no_logging # assert_no_enqueued_jobs except: HelloJob do @@ -159,16 +165,27 @@ module ActiveJob # end # end # + # +:only+ and +:except+ options accepts Class, Array of Class or Proc. When passed a Proc, + # a hash containing the job's class and it's argument are passed as argument. + # + # Asserts that no jobs are enqueued to a specific queue by passing +:queue+ option + # + # def test_no_logging + # assert_no_enqueued_jobs queue: 'default' do + # LoggingJob.set(queue: :some_queue).perform_later + # end + # end + # # Note: This assertion is simply a shortcut for: # # assert_enqueued_jobs 0, &block - def assert_no_enqueued_jobs(only: nil, except: nil, &block) - assert_enqueued_jobs 0, only: only, except: except, &block + def assert_no_enqueued_jobs(only: nil, except: nil, queue: nil, &block) + assert_enqueued_jobs 0, only: only, except: except, queue: queue, &block end # Asserts that the number of performed jobs matches the given number. # If no block is passed, <tt>perform_enqueued_jobs</tt> - # must be called around the job call. + # must be called around or after the job call. # # def test_jobs # assert_performed_jobs 0 @@ -178,13 +195,14 @@ module ActiveJob # end # assert_performed_jobs 1 # - # perform_enqueued_jobs do - # HelloJob.perform_later('yves') - # assert_performed_jobs 2 - # end + # HelloJob.perform_later('yves') + # + # perform_enqueued_jobs + # + # assert_performed_jobs 2 # end # - # If a block is passed, that block should cause the specified number of + # If a block is passed, asserts that the block will cause the specified number of # jobs to be performed. # # def test_jobs_again @@ -198,7 +216,7 @@ module ActiveJob # end # end # - # The block form supports filtering. If the :only option is specified, + # This method also supports filtering. If the +:only+ option is specified, # then only the listed job(s) will be performed. # # def test_hello_job @@ -208,7 +226,7 @@ module ActiveJob # end # end # - # Also if the :except option is specified, + # Also if the +:except+ option is specified, # then the job(s) except specific class will be performed. # # def test_hello_job @@ -229,17 +247,42 @@ module ActiveJob # end # end # end - def assert_performed_jobs(number, only: nil, except: nil) + # + # A proc may also be specified. When passed a Proc, the job's instance will be passed as argument. + # + # def test_hello_and_logging_jobs + # assert_nothing_raised do + # assert_performed_jobs(1, only: ->(job) { job.is_a?(HelloJob) }) do + # HelloJob.perform_later('jeremy') + # LoggingJob.perform_later('stewie') + # RescueJob.perform_later('david') + # end + # end + # end + # + # If the +:queue+ option is specified, + # then only the job(s) enqueued to a specific queue will be performed. + # + # def test_assert_performed_jobs_with_queue_option + # assert_performed_jobs 1, queue: :some_queue do + # HelloJob.set(queue: :some_queue).perform_later("jeremy") + # HelloJob.set(queue: :other_queue).perform_later("bogdan") + # end + # end + def assert_performed_jobs(number, only: nil, except: nil, queue: nil, &block) if block_given? original_count = performed_jobs.size - perform_enqueued_jobs(only: only, except: except) { yield } + + perform_enqueued_jobs(only: only, except: except, queue: queue, &block) + new_count = performed_jobs.size - assert_equal number, new_count - original_count, - "#{number} jobs expected, but #{new_count - original_count} were performed" + + performed_jobs_size = new_count - original_count else - performed_jobs_size = performed_jobs.size - assert_equal number, performed_jobs_size, "#{number} jobs expected, but #{performed_jobs_size} were performed" + performed_jobs_size = performed_jobs_with(only: only, except: except, queue: queue) end + + assert_equal number, performed_jobs_size, "#{number} jobs expected, but #{performed_jobs_size} were performed" end # Asserts that no jobs have been performed. @@ -253,7 +296,7 @@ module ActiveJob # end # end # - # If a block is passed, that block should not cause any job to be performed. + # If a block is passed, asserts that the block will not cause any job to be performed. # # def test_jobs_again # assert_no_performed_jobs do @@ -261,7 +304,7 @@ module ActiveJob # end # end # - # The block form supports filtering. If the :only option is specified, + # The block form supports filtering. If the +:only+ option is specified, # then only the listed job(s) will not be performed. # # def test_no_logging @@ -270,7 +313,7 @@ module ActiveJob # end # end # - # Also if the :except option is specified, + # Also if the +:except+ option is specified, # then the job(s) except specific class will not be performed. # # def test_no_logging @@ -279,14 +322,53 @@ module ActiveJob # end # end # + # +:only+ and +:except+ options accepts Class, Array of Class or Proc. When passed a Proc, + # an instance of the job will be passed as argument. + # + # If the +:queue+ option is specified, + # then only the job(s) enqueued to a specific queue will not be performed. + # + # def test_assert_no_performed_jobs_with_queue_option + # assert_no_performed_jobs queue: :some_queue do + # HelloJob.set(queue: :other_queue).perform_later("jeremy") + # end + # end + # # Note: This assertion is simply a shortcut for: # # assert_performed_jobs 0, &block - def assert_no_performed_jobs(only: nil, except: nil, &block) - assert_performed_jobs 0, only: only, except: except, &block + def assert_no_performed_jobs(only: nil, except: nil, queue: nil, &block) + assert_performed_jobs 0, only: only, except: except, queue: queue, &block end - # Asserts that the job passed in the block has been enqueued with the given arguments. + # Asserts that the job has been enqueued with the given arguments. + # + # def test_assert_enqueued_with + # MyJob.perform_later(1,2,3) + # assert_enqueued_with(job: MyJob, args: [1,2,3], queue: 'low') + # + # MyJob.set(wait_until: Date.tomorrow.noon).perform_later + # assert_enqueued_with(job: MyJob, at: Date.tomorrow.noon) + # end + # + # + # The +args+ argument also accepts a proc which will get passed the actual + # job's arguments. Your proc needs to return a boolean value determining if + # the job's arguments matches your expectation. This is useful to check only + # for a subset of arguments. + # + # def test_assert_enqueued_with + # expected_args = ->(job_args) do + # assert job_args.first.key?(:foo) + # end + # + # MyJob.perform_later(foo: 'bar', other_arg: 'No need to check in the test') + # assert_enqueued_with(job: MyJob, args: expected_args, queue: 'low') + # end + # + # + # If a block is passed, asserts that the block will cause the job to be + # enqueued with the given arguments. # # def test_assert_enqueued_with # assert_enqueued_with(job: MyJob, args: [1,2,3], queue: 'low') do @@ -298,19 +380,70 @@ module ActiveJob # end # end def assert_enqueued_with(job: nil, args: nil, at: nil, queue: nil) - original_enqueued_jobs_count = enqueued_jobs.count expected = { job: job, args: args, at: at, queue: queue }.compact - serialized_args = serialize_args_for_assertion(expected) - yield - in_block_jobs = enqueued_jobs.drop(original_enqueued_jobs_count) - matching_job = in_block_jobs.find do |in_block_job| - serialized_args.all? { |key, value| value == in_block_job[key] } + expected_args = prepare_args_for_assertion(expected) + + if block_given? + original_enqueued_jobs_count = enqueued_jobs.count + + yield + + jobs = enqueued_jobs.drop(original_enqueued_jobs_count) + else + jobs = enqueued_jobs + end + + matching_job = jobs.find do |enqueued_job| + deserialized_job = deserialize_args_for_assertion(enqueued_job) + + expected_args.all? do |key, value| + if value.respond_to?(:call) + value.call(deserialized_job[key]) + else + value == deserialized_job[key] + end + end end + assert matching_job, "No enqueued job found with #{expected}" instantiate_job(matching_job) end - # Asserts that the job passed in the block has been performed with the given arguments. + # Asserts that the job has been performed with the given arguments. + # + # def test_assert_performed_with + # MyJob.perform_later(1,2,3) + # + # perform_enqueued_jobs + # + # assert_performed_with(job: MyJob, args: [1,2,3], queue: 'high') + # + # MyJob.set(wait_until: Date.tomorrow.noon).perform_later + # + # perform_enqueued_jobs + # + # assert_performed_with(job: MyJob, at: Date.tomorrow.noon) + # end + # + # The +args+ argument also accepts a proc which will get passed the actual + # job's arguments. Your proc needs to return a boolean value determining if + # the job's arguments matches your expectation. This is useful to check only + # for a subset of arguments. + # + # def test_assert_performed_with + # expected_args = ->(job_args) do + # assert job_args.first.key?(:foo) + # end + # MyJob.perform_later(foo: 'bar', other_arg: 'No need to check in the test') + # + # perform_enqueued_jobs + # + # assert_performed_with(job: MyJob, args: expected_args, queue: 'high') + # end + # + # If a block is passed, that block performs all of the jobs that were + # enqueued throughout the duration of the block and asserts that + # the job has been performed with the given arguments in the block. # # def test_assert_performed_with # assert_performed_with(job: MyJob, args: [1,2,3], queue: 'high') do @@ -321,20 +454,39 @@ module ActiveJob # MyJob.set(wait_until: Date.tomorrow.noon).perform_later # end # end - def assert_performed_with(job: nil, args: nil, at: nil, queue: nil) - original_performed_jobs_count = performed_jobs.count + def assert_performed_with(job: nil, args: nil, at: nil, queue: nil, &block) expected = { job: job, args: args, at: at, queue: queue }.compact - serialized_args = serialize_args_for_assertion(expected) - perform_enqueued_jobs { yield } - in_block_jobs = performed_jobs.drop(original_performed_jobs_count) - matching_job = in_block_jobs.find do |in_block_job| - serialized_args.all? { |key, value| value == in_block_job[key] } + expected_args = prepare_args_for_assertion(expected) + + if block_given? + original_performed_jobs_count = performed_jobs.count + + perform_enqueued_jobs(&block) + + jobs = performed_jobs.drop(original_performed_jobs_count) + else + jobs = performed_jobs + end + + matching_job = jobs.find do |enqueued_job| + deserialized_job = deserialize_args_for_assertion(enqueued_job) + + expected_args.all? do |key, value| + if value.respond_to?(:call) + value.call(deserialized_job[key]) + else + value == deserialized_job[key] + end + end end + assert matching_job, "No performed job found with #{expected}" instantiate_job(matching_job) end - # Performs all enqueued jobs in the duration of the block. + # Performs all enqueued jobs. If a block is given, performs all of the jobs + # that were enqueued throughout the duration of the block. If a block is + # not given, performs all of the enqueued jobs up to this point in the test. # # def test_perform_enqueued_jobs # perform_enqueued_jobs do @@ -343,6 +495,14 @@ module ActiveJob # assert_performed_jobs 1 # end # + # def test_perform_enqueued_jobs_without_block + # MyJob.perform_later(1, 2, 3) + # + # perform_enqueued_jobs + # + # assert_performed_jobs 1 + # end + # # This method also supports filtering. If the +:only+ option is specified, # then only the listed job(s) will be performed. # @@ -365,24 +525,45 @@ module ActiveJob # assert_performed_jobs 1 # end # - def perform_enqueued_jobs(only: nil, except: nil) + # +:only+ and +:except+ options accepts Class, Array of Class or Proc. When passed a Proc, + # an instance of the job will be passed as argument. + # + # If the +:queue+ option is specified, + # then only the job(s) enqueued to a specific queue will be performed. + # + # def test_perform_enqueued_jobs_with_queue + # perform_enqueued_jobs queue: :some_queue do + # MyJob.set(queue: :some_queue).perform_later(1, 2, 3) # will be performed + # HelloJob.set(queue: :other_queue).perform_later(1, 2, 3) # will not be performed + # end + # assert_performed_jobs 1 + # end + # + def perform_enqueued_jobs(only: nil, except: nil, queue: nil) + return flush_enqueued_jobs(only: only, except: except, queue: queue) unless block_given? + validate_option(only: only, except: except) + old_perform_enqueued_jobs = queue_adapter.perform_enqueued_jobs old_perform_enqueued_at_jobs = queue_adapter.perform_enqueued_at_jobs old_filter = queue_adapter.filter old_reject = queue_adapter.reject + old_queue = queue_adapter.queue begin queue_adapter.perform_enqueued_jobs = true queue_adapter.perform_enqueued_at_jobs = true queue_adapter.filter = only queue_adapter.reject = except + queue_adapter.queue = queue + yield ensure queue_adapter.perform_enqueued_jobs = old_perform_enqueued_jobs queue_adapter.perform_enqueued_at_jobs = old_perform_enqueued_at_jobs queue_adapter.filter = old_filter queue_adapter.reject = old_reject + queue_adapter.queue = old_queue end end @@ -404,31 +585,78 @@ module ActiveJob performed_jobs.clear end - def enqueued_jobs_size(only: nil, except: nil, queue: nil) + def jobs_with(jobs, only: nil, except: nil, queue: nil) validate_option(only: only, except: except) - enqueued_jobs.count do |job| + + jobs.count do |job| job_class = job.fetch(:job) + if only - next false unless Array(only).include?(job_class) + next false unless filter_as_proc(only).call(job) elsif except - next false if Array(except).include?(job_class) + next false if filter_as_proc(except).call(job) end + if queue next false unless queue.to_s == job.fetch(:queue, job_class.queue_name) end + + yield job if block_given? + true end end - def serialize_args_for_assertion(args) - args.dup.tap do |serialized_args| - serialized_args[:args] = ActiveJob::Arguments.serialize(serialized_args[:args]) if serialized_args[:args] - serialized_args[:at] = serialized_args[:at].to_f if serialized_args[:at] + def filter_as_proc(filter) + return filter if filter.is_a?(Proc) + + ->(job) { Array(filter).include?(job.fetch(:job)) } + end + + def enqueued_jobs_with(only: nil, except: nil, queue: nil, &block) + jobs_with(enqueued_jobs, only: only, except: except, queue: queue, &block) + end + + def performed_jobs_with(only: nil, except: nil, queue: nil, &block) + jobs_with(performed_jobs, only: only, except: except, queue: queue, &block) + end + + def flush_enqueued_jobs(only: nil, except: nil, queue: nil) + enqueued_jobs_with(only: only, except: except, queue: queue) do |payload| + instantiate_job(payload).perform_now + queue_adapter.performed_jobs << payload + end + end + + def prepare_args_for_assertion(args) + args.dup.tap do |arguments| + arguments[:at] = arguments[:at].to_f if arguments[:at] + arguments[:args] = round_time_arguments(arguments[:args]) if arguments[:args] + end + end + + def round_time_arguments(argument) + case argument + when Time, ActiveSupport::TimeWithZone, DateTime + argument.change(usec: 0) + when Hash + argument.transform_values { |value| round_time_arguments(value) } + when Array + argument.map { |element| round_time_arguments(element) } + else + argument + end + end + + def deserialize_args_for_assertion(job) + job.dup.tap do |new_job| + new_job[:args] = ActiveJob::Arguments.deserialize(new_job[:args]) if new_job[:args] end end def instantiate_job(payload) - job = payload[:job].new(*payload[:args]) + args = ActiveJob::Arguments.deserialize(payload[:args]) + job = payload[:job].new(*args) job.scheduled_at = Time.at(payload[:at]) if payload.key?(:at) job.queue_name = payload[:queue] job diff --git a/activejob/lib/rails/generators/job/job_generator.rb b/activejob/lib/rails/generators/job/job_generator.rb index 69b4fe7d26..03346a7f12 100644 --- a/activejob/lib/rails/generators/job/job_generator.rb +++ b/activejob/lib/rails/generators/job/job_generator.rb @@ -28,6 +28,10 @@ module Rails # :nodoc: end private + def file_name + @_file_name ||= super.sub(/_job\z/i, "") + end + def application_job_file_name @application_job_file_name ||= if mountable_engine? "app/jobs/#{namespaced_path}/application_job.rb" diff --git a/activejob/test/cases/argument_serialization_test.rb b/activejob/test/cases/argument_serialization_test.rb index e5f1f087fe..da198abc0b 100644 --- a/activejob/test/cases/argument_serialization_test.rb +++ b/activejob/test/cases/argument_serialization_test.rb @@ -5,6 +5,7 @@ require "active_job/arguments" require "models/person" require "active_support/core_ext/hash/indifferent_access" require "jobs/kwargs_job" +require "support/stubs/strong_parameters" class ArgumentSerializationTest < ActiveSupport::TestCase setup do @@ -40,6 +41,10 @@ class ArgumentSerializationTest < ActiveSupport::TestCase assert_arguments_roundtrip [@person] end + test "should keep Global IDs strings as they are" do + assert_arguments_roundtrip [@person.to_gid.to_s] + end + test "should dive deep into arrays and hashes" do assert_arguments_roundtrip [3, [@person]] assert_arguments_roundtrip [{ "a" => @person }] @@ -49,6 +54,15 @@ class ArgumentSerializationTest < ActiveSupport::TestCase assert_arguments_roundtrip([a: 1, "b" => 2]) end + test "serialize a ActionController::Parameters" do + parameters = Parameters.new(a: 1) + + assert_equal( + { "a" => 1, "_aj_hash_with_indifferent_access" => true }, + ActiveJob::Arguments.serialize([parameters]).first + ) + end + test "serialize a hash" do symbol_key = { a: 1 } string_key = { "a" => 1 } @@ -73,6 +87,7 @@ class ArgumentSerializationTest < ActiveSupport::TestCase string_key = { "a" => 1, "_aj_symbol_keys" => [] } another_string_key = { "a" => 1 } indifferent_access = { "a" => 1, "_aj_hash_with_indifferent_access" => true } + indifferent_access_symbol_key = symbol_key.with_indifferent_access assert_equal( { a: 1 }, @@ -90,6 +105,10 @@ class ArgumentSerializationTest < ActiveSupport::TestCase { "a" => 1 }, ActiveJob::Arguments.deserialize([indifferent_access]).first ) + assert_equal( + { a: 1 }, + ActiveJob::Arguments.deserialize([indifferent_access_symbol_key]).first + ) end test "should maintain hash with indifferent access" do @@ -121,8 +140,10 @@ class ArgumentSerializationTest < ActiveSupport::TestCase end test "should not allow reserved hash keys" do - ["_aj_globalid", :_aj_globalid, "_aj_symbol_keys", :_aj_symbol_keys, - "_aj_hash_with_indifferent_access", :_aj_hash_with_indifferent_access].each do |key| + ["_aj_globalid", :_aj_globalid, + "_aj_symbol_keys", :_aj_symbol_keys, + "_aj_hash_with_indifferent_access", :_aj_hash_with_indifferent_access, + "_aj_serialized", :_aj_serialized].each do |key| assert_raises ActiveJob::SerializationError do ActiveJob::Arguments.serialize [key => 1] end diff --git a/activejob/test/cases/callbacks_test.rb b/activejob/test/cases/callbacks_test.rb index df6ce16858..895edb34a5 100644 --- a/activejob/test/cases/callbacks_test.rb +++ b/activejob/test/cases/callbacks_test.rb @@ -2,6 +2,7 @@ require "helper" require "jobs/callback_job" +require "jobs/abort_before_enqueue_job" require "active_support/core_ext/object/inclusion" @@ -22,4 +23,28 @@ class CallbacksTest < ActiveSupport::TestCase assert "CallbackJob ran around_enqueue_start".in? enqueued_callback_job.history assert "CallbackJob ran around_enqueue_stop".in? enqueued_callback_job.history end + + test "#enqueue returns false when before_enqueue aborts callback chain and return_false_on_aborted_enqueue = true" do + prev = ActiveJob::Base.return_false_on_aborted_enqueue + ActiveJob::Base.return_false_on_aborted_enqueue = true + assert_equal false, AbortBeforeEnqueueJob.new.enqueue + ensure + ActiveJob::Base.return_false_on_aborted_enqueue = prev + end + + test "#enqueue returns self when before_enqueue aborts callback chain and return_false_on_aborted_enqueue = false" do + prev = ActiveJob::Base.return_false_on_aborted_enqueue + ActiveJob::Base.return_false_on_aborted_enqueue = false + job = AbortBeforeEnqueueJob.new + assert_deprecated do + assert_equal job, job.enqueue + end + ensure + ActiveJob::Base.return_false_on_aborted_enqueue = prev + end + + test "#enqueue returns self when the job was enqueued" do + job = CallbackJob.new + assert_equal job, job.enqueue + end end diff --git a/activejob/test/cases/exceptions_test.rb b/activejob/test/cases/exceptions_test.rb index bc33d79f61..c88162bf58 100644 --- a/activejob/test/cases/exceptions_test.rb +++ b/activejob/test/cases/exceptions_test.rb @@ -2,115 +2,169 @@ require "helper" require "jobs/retry_job" +require "models/person" -class ExceptionsTest < ActiveJob::TestCase +class ExceptionsTest < ActiveSupport::TestCase setup do JobBuffer.clear - skip if ActiveJob::Base.queue_adapter.is_a?(ActiveJob::QueueAdapters::InlineAdapter) + skip if adapter_skips_scheduling?(ActiveJob::Base.queue_adapter) end test "successfully retry job throwing exception against defaults" do - perform_enqueued_jobs do - RetryJob.perform_later "DefaultsError", 5 + RetryJob.perform_later "DefaultsError", 5 + + assert_equal [ + "Raised DefaultsError for the 1st time", + "Raised DefaultsError for the 2nd time", + "Raised DefaultsError for the 3rd time", + "Raised DefaultsError for the 4th time", + "Successfully completed job" ], JobBuffer.values + end + + test "successfully retry job throwing exception against higher limit" do + RetryJob.perform_later "ShortWaitTenAttemptsError", 9 + assert_equal 9, JobBuffer.values.count + end + + test "keeps the same attempts counter for several exceptions listed in the same retry_on declaration" do + exceptions_to_raise = %w(FirstRetryableErrorOfTwo FirstRetryableErrorOfTwo FirstRetryableErrorOfTwo + SecondRetryableErrorOfTwo SecondRetryableErrorOfTwo) + + assert_raises SecondRetryableErrorOfTwo do + RetryJob.perform_later(exceptions_to_raise, 5) + + assert_equal [ + "Raised FirstRetryableErrorOfTwo for the 1st time", + "Raised FirstRetryableErrorOfTwo for the 2nd time", + "Raised FirstRetryableErrorOfTwo for the 3rd time", + "Raised SecondRetryableErrorOfTwo for the 4th time", + "Raised SecondRetryableErrorOfTwo for the 5th time", + ], JobBuffer.values + end + end + + test "keeps a separate attempts counter for each individual retry_on declaration" do + exceptions_to_raise = %w(DefaultsError DefaultsError DefaultsError DefaultsError + FirstRetryableErrorOfTwo FirstRetryableErrorOfTwo FirstRetryableErrorOfTwo) + + assert_nothing_raised do + RetryJob.perform_later(exceptions_to_raise, 10) assert_equal [ "Raised DefaultsError for the 1st time", "Raised DefaultsError for the 2nd time", "Raised DefaultsError for the 3rd time", "Raised DefaultsError for the 4th time", - "Successfully completed job" ], JobBuffer.values - end - end - - test "successfully retry job throwing exception against higher limit" do - perform_enqueued_jobs do - RetryJob.perform_later "ShortWaitTenAttemptsError", 9 - assert_equal 9, JobBuffer.values.count + "Raised FirstRetryableErrorOfTwo for the 5th time", + "Raised FirstRetryableErrorOfTwo for the 6th time", + "Raised FirstRetryableErrorOfTwo for the 7th time", + "Successfully completed job" + ], JobBuffer.values end end test "failed retry job when exception kept occurring against defaults" do - perform_enqueued_jobs do - begin - RetryJob.perform_later "DefaultsError", 6 - assert_equal "Raised DefaultsError for the 5th time", JobBuffer.last_value - rescue DefaultsError - pass - end - end + RetryJob.perform_later "DefaultsError", 6 + assert_equal "Raised DefaultsError for the 5th time", JobBuffer.last_value + rescue DefaultsError + pass end test "failed retry job when exception kept occurring against higher limit" do - perform_enqueued_jobs do - begin - RetryJob.perform_later "ShortWaitTenAttemptsError", 11 - assert_equal "Raised ShortWaitTenAttemptsError for the 10th time", JobBuffer.last_value - rescue ShortWaitTenAttemptsError - pass - end - end + RetryJob.perform_later "ShortWaitTenAttemptsError", 11 + assert_equal "Raised ShortWaitTenAttemptsError for the 10th time", JobBuffer.last_value + rescue ShortWaitTenAttemptsError + pass end test "discard job" do - perform_enqueued_jobs do - RetryJob.perform_later "DiscardableError", 2 - assert_equal "Raised DiscardableError for the 1st time", JobBuffer.last_value - end + RetryJob.perform_later "DiscardableError", 2 + assert_equal "Raised DiscardableError for the 1st time", JobBuffer.last_value end test "custom handling of discarded job" do - perform_enqueued_jobs do - RetryJob.perform_later "CustomDiscardableError", 2 - assert_equal "Dealt with a job that was discarded in a custom way", JobBuffer.last_value - end + RetryJob.perform_later "CustomDiscardableError", 2 + assert_equal "Dealt with a job that was discarded in a custom way. Message: CustomDiscardableError", JobBuffer.last_value end test "custom handling of job that exceeds retry attempts" do - perform_enqueued_jobs do - RetryJob.perform_later "CustomCatchError", 6 - assert_equal "Dealt with a job that failed to retry in a custom way after 6 attempts. Message: CustomCatchError", JobBuffer.last_value - end + RetryJob.perform_later "CustomCatchError", 6 + assert_equal "Dealt with a job that failed to retry in a custom way after 6 attempts. Message: CustomCatchError", JobBuffer.last_value end test "long wait job" do travel_to Time.now - perform_enqueued_jobs do - assert_performed_with at: (Time.now + 3600.seconds).to_i do - RetryJob.perform_later "LongWaitError", 5 - end - end + RetryJob.perform_later "LongWaitError", 2, :log_scheduled_at + + assert_equal [ + "Raised LongWaitError for the 1st time", + "Next execution scheduled at #{(Time.now + 3600.seconds).to_f}", + "Successfully completed job" + ], JobBuffer.values end test "exponentially retrying job" do travel_to Time.now - perform_enqueued_jobs do - assert_performed_with at: (Time.now + 3.seconds).to_i do - assert_performed_with at: (Time.now + 18.seconds).to_i do - assert_performed_with at: (Time.now + 83.seconds).to_i do - assert_performed_with at: (Time.now + 258.seconds).to_i do - RetryJob.perform_later "ExponentialWaitTenAttemptsError", 5 - end - end - end - end - end + RetryJob.perform_later "ExponentialWaitTenAttemptsError", 5, :log_scheduled_at + + assert_equal [ + "Raised ExponentialWaitTenAttemptsError for the 1st time", + "Next execution scheduled at #{(Time.now + 3.seconds).to_f}", + "Raised ExponentialWaitTenAttemptsError for the 2nd time", + "Next execution scheduled at #{(Time.now + 18.seconds).to_f}", + "Raised ExponentialWaitTenAttemptsError for the 3rd time", + "Next execution scheduled at #{(Time.now + 83.seconds).to_f}", + "Raised ExponentialWaitTenAttemptsError for the 4th time", + "Next execution scheduled at #{(Time.now + 258.seconds).to_f}", + "Successfully completed job" + ], JobBuffer.values end test "custom wait retrying job" do travel_to Time.now - perform_enqueued_jobs do - assert_performed_with at: (Time.now + 2.seconds).to_i do - assert_performed_with at: (Time.now + 4.seconds).to_i do - assert_performed_with at: (Time.now + 6.seconds).to_i do - assert_performed_with at: (Time.now + 8.seconds).to_i do - RetryJob.perform_later "CustomWaitTenAttemptsError", 5 - end - end - end - end - end + RetryJob.perform_later "CustomWaitTenAttemptsError", 5, :log_scheduled_at + + assert_equal [ + "Raised CustomWaitTenAttemptsError for the 1st time", + "Next execution scheduled at #{(Time.now + 2.seconds).to_f}", + "Raised CustomWaitTenAttemptsError for the 2nd time", + "Next execution scheduled at #{(Time.now + 4.seconds).to_f}", + "Raised CustomWaitTenAttemptsError for the 3rd time", + "Next execution scheduled at #{(Time.now + 6.seconds).to_f}", + "Raised CustomWaitTenAttemptsError for the 4th time", + "Next execution scheduled at #{(Time.now + 8.seconds).to_f}", + "Successfully completed job" + ], JobBuffer.values end + + test "successfully retry job throwing one of two retryable exceptions" do + RetryJob.perform_later "SecondRetryableErrorOfTwo", 3 + + assert_equal [ + "Raised SecondRetryableErrorOfTwo for the 1st time", + "Raised SecondRetryableErrorOfTwo for the 2nd time", + "Successfully completed job" ], JobBuffer.values + end + + test "discard job throwing one of two discardable exceptions" do + RetryJob.perform_later "SecondDiscardableErrorOfTwo", 2 + assert_equal [ "Raised SecondDiscardableErrorOfTwo for the 1st time" ], JobBuffer.values + end + + test "successfully retry job throwing DeserializationError" do + RetryJob.perform_later Person.new(404), 5 + assert_equal ["Raised ActiveJob::DeserializationError for the 5 time"], JobBuffer.values + end + + private + def adapter_skips_scheduling?(queue_adapter) + [ + ActiveJob::QueueAdapters::InlineAdapter, + ActiveJob::QueueAdapters::AsyncAdapter, + ActiveJob::QueueAdapters::SneakersAdapter + ].include?(queue_adapter.class) + end end diff --git a/activejob/test/cases/job_serialization_test.rb b/activejob/test/cases/job_serialization_test.rb index 5c9994508e..c1cec1f1d6 100644 --- a/activejob/test/cases/job_serialization_test.rb +++ b/activejob/test/cases/job_serialization_test.rb @@ -23,16 +23,16 @@ class JobSerializationTest < ActiveSupport::TestCase test "serialize and deserialize are symmetric" do # Round trip a job in memory only - h1 = HelloJob.new - h1.deserialize(h1.serialize) + h1 = HelloJob.new("Rafael") + h2 = HelloJob.deserialize(h1.serialize) + assert_equal h1.serialize, h2.serialize # Now verify it's identical to a JSON round trip. # We don't want any non-native JSON elements in the job hash, # like symbols. - payload = JSON.dump(h1.serialize) - h2 = HelloJob.new - h2.deserialize(JSON.load(payload)) - assert_equal h1.serialize, h2.serialize + payload = JSON.dump(h2.serialize) + h3 = HelloJob.deserialize(JSON.load(payload)) + assert_equal h2.serialize, h3.serialize end test "deserialize sets locale" do @@ -61,4 +61,15 @@ class JobSerializationTest < ActiveSupport::TestCase assert_equal "Hawaii", job.serialize["timezone"] end end + + test "serialize stores the enqueued_at time" do + h1 = HelloJob.new + type = h1.serialize["enqueued_at"].class + assert_equal String, type + + h2 = HelloJob.deserialize(h1.serialize) + # We should be able to parse a timestamp + type = Time.parse(h2.enqueued_at).class + assert_equal Time, type + end end diff --git a/activejob/test/cases/logging_test.rb b/activejob/test/cases/logging_test.rb index 1f8c4a5573..acd37456c9 100644 --- a/activejob/test/cases/logging_test.rb +++ b/activejob/test/cases/logging_test.rb @@ -8,9 +8,11 @@ require "jobs/logging_job" require "jobs/overridden_logging_job" require "jobs/nested_job" require "jobs/rescue_job" +require "jobs/retry_job" require "models/person" class LoggingTest < ActiveSupport::TestCase + include ActiveJob::TestHelper include ActiveSupport::LogSubscriber::TestHelper include ActiveSupport::Logger::Severity @@ -45,19 +47,31 @@ class LoggingTest < ActiveSupport::TestCase ActiveJob::Base.logger = logger end + def subscribed + [].tap do |events| + ActiveSupport::Notifications.subscribed(-> (*args) { events << args }, /enqueue.*\.active_job/) do + yield + end + end + end + def test_uses_active_job_as_tag HelloJob.perform_later "Cristian" assert_match(/\[ActiveJob\]/, @logger.messages) end def test_uses_job_name_as_tag - LoggingJob.perform_later "Dummy" - assert_match(/\[LoggingJob\]/, @logger.messages) + perform_enqueued_jobs do + LoggingJob.perform_later "Dummy" + assert_match(/\[LoggingJob\]/, @logger.messages) + end end def test_uses_job_id_as_tag - LoggingJob.perform_later "Dummy" - assert_match(/\[LOGGING-JOB-ID\]/, @logger.messages) + perform_enqueued_jobs do + LoggingJob.perform_later "Dummy" + assert_match(/\[LOGGING-JOB-ID\]/, @logger.messages) + end end def test_logs_correct_queue_name @@ -70,55 +84,74 @@ class LoggingTest < ActiveSupport::TestCase end def test_globalid_parameter_logging - person = Person.new(123) - LoggingJob.perform_later person - assert_match(%r{Enqueued.*gid://aj/Person/123}, @logger.messages) - assert_match(%r{Dummy, here is it: #<Person:.*>}, @logger.messages) - assert_match(%r{Performing.*gid://aj/Person/123}, @logger.messages) + perform_enqueued_jobs do + person = Person.new(123) + LoggingJob.perform_later person + assert_match(%r{Enqueued.*gid://aj/Person/123}, @logger.messages) + assert_match(%r{Dummy, here is it: #<Person:.*>}, @logger.messages) + assert_match(%r{Performing.*gid://aj/Person/123}, @logger.messages) + end end def test_globalid_nested_parameter_logging - person = Person.new(123) - LoggingJob.perform_later(person: person) - assert_match(%r{Enqueued.*gid://aj/Person/123}, @logger.messages) - assert_match(%r{Dummy, here is it: .*#<Person:.*>}, @logger.messages) - assert_match(%r{Performing.*gid://aj/Person/123}, @logger.messages) + perform_enqueued_jobs do + person = Person.new(123) + LoggingJob.perform_later(person: person) + assert_match(%r{Enqueued.*gid://aj/Person/123}, @logger.messages) + assert_match(%r{Dummy, here is it: .*#<Person:.*>}, @logger.messages) + assert_match(%r{Performing.*gid://aj/Person/123}, @logger.messages) + end end def test_enqueue_job_logging - HelloJob.perform_later "Cristian" + events = subscribed { HelloJob.perform_later "Cristian" } assert_match(/Enqueued HelloJob \(Job ID: .*?\) to .*?:.*Cristian/, @logger.messages) + assert_equal(events.count, 1) + key, * = events.first + assert_equal(key, "enqueue.active_job") end def test_perform_job_logging - LoggingJob.perform_later "Dummy" - assert_match(/Performing LoggingJob \(Job ID: .*?\) from .*? with arguments:.*Dummy/, @logger.messages) - assert_match(/Dummy, here is it: Dummy/, @logger.messages) - assert_match(/Performed LoggingJob \(Job ID: .*?\) from .*? in .*ms/, @logger.messages) + perform_enqueued_jobs do + LoggingJob.perform_later "Dummy" + assert_match(/Performing LoggingJob \(Job ID: .*?\) from .*? with arguments:.*Dummy/, @logger.messages) + + assert_match(/enqueued at /, @logger.messages) + assert_match(/Dummy, here is it: Dummy/, @logger.messages) + assert_match(/Performed LoggingJob \(Job ID: .*?\) from .*? in .*ms/, @logger.messages) + end end def test_perform_nested_jobs_logging - NestedJob.perform_later - assert_match(/\[LoggingJob\] \[.*?\]/, @logger.messages) - assert_match(/\[ActiveJob\] Enqueued NestedJob \(Job ID: .*\) to/, @logger.messages) - assert_match(/\[ActiveJob\] \[NestedJob\] \[NESTED-JOB-ID\] Performing NestedJob \(Job ID: .*?\) from/, @logger.messages) - assert_match(/\[ActiveJob\] \[NestedJob\] \[NESTED-JOB-ID\] Enqueued LoggingJob \(Job ID: .*?\) to .* with arguments: "NestedJob"/, @logger.messages) - assert_match(/\[ActiveJob\].*\[LoggingJob\] \[LOGGING-JOB-ID\] Performing LoggingJob \(Job ID: .*?\) from .* with arguments: "NestedJob"/, @logger.messages) - assert_match(/\[ActiveJob\].*\[LoggingJob\] \[LOGGING-JOB-ID\] Dummy, here is it: NestedJob/, @logger.messages) - assert_match(/\[ActiveJob\].*\[LoggingJob\] \[LOGGING-JOB-ID\] Performed LoggingJob \(Job ID: .*?\) from .* in/, @logger.messages) - assert_match(/\[ActiveJob\] \[NestedJob\] \[NESTED-JOB-ID\] Performed NestedJob \(Job ID: .*?\) from .* in/, @logger.messages) + perform_enqueued_jobs do + NestedJob.perform_later + assert_match(/\[LoggingJob\] \[.*?\]/, @logger.messages) + assert_match(/\[ActiveJob\] Enqueued NestedJob \(Job ID: .*\) to/, @logger.messages) + assert_match(/\[ActiveJob\] \[NestedJob\] \[NESTED-JOB-ID\] Performing NestedJob \(Job ID: .*?\) from/, @logger.messages) + assert_match(/\[ActiveJob\] \[NestedJob\] \[NESTED-JOB-ID\] Enqueued LoggingJob \(Job ID: .*?\) to .* with arguments: "NestedJob"/, @logger.messages) + assert_match(/\[ActiveJob\].*\[LoggingJob\] \[LOGGING-JOB-ID\] Performing LoggingJob \(Job ID: .*?\) from .* with arguments: "NestedJob"/, @logger.messages) + assert_match(/\[ActiveJob\].*\[LoggingJob\] \[LOGGING-JOB-ID\] Dummy, here is it: NestedJob/, @logger.messages) + assert_match(/\[ActiveJob\].*\[LoggingJob\] \[LOGGING-JOB-ID\] Performed LoggingJob \(Job ID: .*?\) from .* in/, @logger.messages) + assert_match(/\[ActiveJob\] \[NestedJob\] \[NESTED-JOB-ID\] Performed NestedJob \(Job ID: .*?\) from .* in/, @logger.messages) + end end def test_enqueue_at_job_logging - HelloJob.set(wait_until: 24.hours.from_now).perform_later "Cristian" + events = subscribed { HelloJob.set(wait_until: 24.hours.from_now).perform_later "Cristian" } assert_match(/Enqueued HelloJob \(Job ID: .*\) to .*? at.*Cristian/, @logger.messages) + assert_equal(events.count, 1) + key, * = events.first + assert_equal(key, "enqueue_at.active_job") rescue NotImplementedError skip end def test_enqueue_in_job_logging - HelloJob.set(wait: 2.seconds).perform_later "Cristian" + events = subscribed { HelloJob.set(wait: 2.seconds).perform_later "Cristian" } assert_match(/Enqueued HelloJob \(Job ID: .*\) to .*? at.*Cristian/, @logger.messages) + assert_equal(events.count, 1) + key, * = events.first + assert_equal(key, "enqueue_at.active_job") rescue NotImplementedError skip end @@ -129,9 +162,43 @@ class LoggingTest < ActiveSupport::TestCase end def test_job_error_logging - RescueJob.perform_later "other" + perform_enqueued_jobs { RescueJob.perform_later "other" } rescue RescueJob::OtherError assert_match(/Performing RescueJob \(Job ID: .*?\) from .*? with arguments:.*other/, @logger.messages) assert_match(/Error performing RescueJob \(Job ID: .*?\) from .*? in .*ms: RescueJob::OtherError \(Bad hair\):\n.*\brescue_job\.rb:\d+:in `perform'/, @logger.messages) end + + def test_enqueue_retry_logging + perform_enqueued_jobs do + RetryJob.perform_later "DefaultsError", 2 + assert_match(/Retrying RetryJob in 3 seconds, due to a DefaultsError\./, @logger.messages) + end + end + + def test_enqueue_retry_logging_on_retry_job + perform_enqueued_jobs { RescueJob.perform_later "david" } + assert_match(/Retrying RescueJob in 0 seconds\./, @logger.messages) + end + + def test_retry_stopped_logging + perform_enqueued_jobs do + RetryJob.perform_later "CustomCatchError", 6 + assert_match(/Stopped retrying RetryJob due to a CustomCatchError, which reoccurred on \d+ attempts\./, @logger.messages) + end + end + + def test_retry_stopped_logging_without_block + perform_enqueued_jobs do + RetryJob.perform_later "DefaultsError", 6 + rescue DefaultsError + assert_match(/Stopped retrying RetryJob due to a DefaultsError, which reoccurred on \d+ attempts\./, @logger.messages) + end + end + + def test_discard_logging + perform_enqueued_jobs do + RetryJob.perform_later "DiscardableError", 2 + assert_match(/Discarded RetryJob due to a DiscardableError\./, @logger.messages) + end + end end diff --git a/activejob/test/cases/queue_naming_test.rb b/activejob/test/cases/queue_naming_test.rb index b64a38f91e..4b43c7c3c5 100644 --- a/activejob/test/cases/queue_naming_test.rb +++ b/activejob/test/cases/queue_naming_test.rb @@ -7,7 +7,7 @@ require "jobs/nested_job" class QueueNamingTest < ActiveSupport::TestCase test "name derived from base" do - assert_equal "default", HelloJob.queue_name + assert_equal "default", HelloJob.new.queue_name end test "uses given queue name job" do @@ -97,6 +97,33 @@ class QueueNamingTest < ActiveSupport::TestCase end end + test "using a custom default_queue_name" do + original_default_queue_name = ActiveJob::Base.default_queue_name + + begin + ActiveJob::Base.default_queue_name = "default_queue_name" + + assert_equal "default_queue_name", HelloJob.new.queue_name + ensure + ActiveJob::Base.default_queue_name = original_default_queue_name + end + end + + test "queue_name_prefix prepended to the default_queue_name" do + original_queue_name_prefix = ActiveJob::Base.queue_name_prefix + original_default_queue_name = ActiveJob::Base.default_queue_name + + begin + ActiveJob::Base.queue_name_prefix = "prefix" + ActiveJob::Base.default_queue_name = "default_queue_name" + + assert_equal "prefix_default_queue_name", HelloJob.new.queue_name + ensure + ActiveJob::Base.queue_name_prefix = original_queue_name_prefix + ActiveJob::Base.default_queue_name = original_default_queue_name + end + end + test "uses queue passed to #set" do job = HelloJob.set(queue: :some_queue).perform_later assert_equal "some_queue", job.queue_name diff --git a/activejob/test/cases/queuing_test.rb b/activejob/test/cases/queuing_test.rb index 0e843b7215..e7bad83400 100644 --- a/activejob/test/cases/queuing_test.rb +++ b/activejob/test/cases/queuing_test.rb @@ -20,12 +20,10 @@ class QueuingTest < ActiveSupport::TestCase end test "run queued job later" do - begin - result = HelloJob.set(wait_until: 1.second.ago).perform_later "Jamie" - assert result - rescue NotImplementedError - skip - end + result = HelloJob.set(wait_until: 1.second.ago).perform_later "Jamie" + assert result + rescue NotImplementedError + skip end test "job returned by enqueue has the arguments available" do @@ -34,11 +32,9 @@ class QueuingTest < ActiveSupport::TestCase end test "job returned by perform_at has the timestamp available" do - begin - job = HelloJob.set(wait_until: Time.utc(2014, 1, 1)).perform_later - assert_equal Time.utc(2014, 1, 1).to_f, job.scheduled_at - rescue NotImplementedError - skip - end + job = HelloJob.set(wait_until: Time.utc(2014, 1, 1)).perform_later + assert_equal Time.utc(2014, 1, 1).to_f, job.scheduled_at + rescue NotImplementedError + skip end end diff --git a/activejob/test/cases/test_helper_test.rb b/activejob/test/cases/test_helper_test.rb index 66bcd8f3a0..d6607cb6b6 100644 --- a/activejob/test/cases/test_helper_test.rb +++ b/activejob/test/cases/test_helper_test.rb @@ -8,6 +8,7 @@ require "jobs/logging_job" require "jobs/nested_job" require "jobs/rescue_job" require "jobs/inherited_job" +require "jobs/multiple_kwargs_job" require "models/person" class EnqueuedJobsTest < ActiveJob::TestCase @@ -113,6 +114,16 @@ class EnqueuedJobsTest < ActiveJob::TestCase end end + def test_assert_enqueued_jobs_with_only_option_as_proc + assert_nothing_raised do + assert_enqueued_jobs(1, only: ->(job) { job.fetch(:job).name == "HelloJob" }) do + HelloJob.perform_later("jeremy") + LoggingJob.perform_later + LoggingJob.perform_later + end + end + end + def test_assert_enqueued_jobs_with_except_option assert_nothing_raised do assert_enqueued_jobs 1, except: LoggingJob do @@ -123,6 +134,16 @@ class EnqueuedJobsTest < ActiveJob::TestCase end end + def test_assert_enqueued_jobs_with_except_option_as_proc + assert_nothing_raised do + assert_enqueued_jobs(1, except: ->(job) { job.fetch(:job).name == "LoggingJob" }) do + HelloJob.perform_later("jeremy") + LoggingJob.perform_later + LoggingJob.perform_later + end + end + end + def test_assert_enqueued_jobs_with_only_and_except_option error = assert_raise ArgumentError do assert_enqueued_jobs 1, only: HelloJob, except: HelloJob do @@ -389,30 +410,123 @@ class EnqueuedJobsTest < ActiveJob::TestCase assert_match(/`:only` and `:except`/, error.message) end - def test_assert_enqueued_job + def test_assert_no_enqueued_jobs_with_queue_option + assert_nothing_raised do + assert_no_enqueued_jobs queue: :default do + HelloJob.set(queue: :other_queue).perform_later + LoggingJob.set(queue: :other_queue).perform_later + end + end + end + + def test_assert_no_enqueued_jobs_with_queue_option_failure + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_no_enqueued_jobs queue: :other_queue do + HelloJob.set(queue: :other_queue).perform_later + end + end + + assert_match(/0 .* but 1/, error.message) + end + + def test_assert_no_enqueued_jobs_with_only_and_queue_option + assert_nothing_raised do + assert_no_enqueued_jobs only: HelloJob, queue: :some_queue do + HelloJob.set(queue: :other_queue).perform_later + HelloJob.set(queue: :other_queue).perform_later + LoggingJob.set(queue: :some_queue).perform_later + end + end + end + + def test_assert_no_enqueued_jobs_with_only_and_queue_option_failure + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_no_enqueued_jobs only: HelloJob, queue: :some_queue do + HelloJob.set(queue: :other_queue).perform_later + HelloJob.set(queue: :some_queue).perform_later + LoggingJob.set(queue: :some_queue).perform_later + end + end + + assert_match(/0 .* but 1/, error.message) + end + + def test_assert_no_enqueued_jobs_with_except_and_queue_option + assert_nothing_raised do + assert_no_enqueued_jobs except: LoggingJob, queue: :some_queue do + HelloJob.set(queue: :other_queue).perform_later + HelloJob.set(queue: :other_queue).perform_later + LoggingJob.set(queue: :some_queue).perform_later + end + end + end + + def test_assert_no_enqueued_jobs_with_except_and_queue_option_failure + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_no_enqueued_jobs except: LoggingJob, queue: :some_queue do + HelloJob.set(queue: :other_queue).perform_later + HelloJob.set(queue: :some_queue).perform_later + LoggingJob.set(queue: :some_queue).perform_later + end + end + + assert_match(/0 .* but 1/, error.message) + end + + def test_assert_no_enqueued_jobs_with_only_and_except_and_queue_option + error = assert_raise ArgumentError do + assert_no_enqueued_jobs only: HelloJob, except: HelloJob, queue: :some_queue do + HelloJob.set(queue: :other_queue).perform_later + end + end + + assert_match(/`:only` and `:except`/, error.message) + end + + def test_assert_enqueued_with assert_enqueued_with(job: LoggingJob, queue: "default") do LoggingJob.set(wait_until: Date.tomorrow.noon).perform_later end end - def test_assert_enqueued_job_returns + def test_assert_enqueued_with_with_no_block + LoggingJob.set(wait_until: Date.tomorrow.noon).perform_later + assert_enqueued_with(job: LoggingJob, queue: "default") + end + + def test_assert_enqueued_with_returns job = assert_enqueued_with(job: LoggingJob) do - LoggingJob.set(wait_until: 5.minutes.from_now).perform_later(1, 2, 3) + LoggingJob.set(wait_until: 5.minutes.from_now).perform_later(1, 2, 3, keyword: true) end assert_instance_of LoggingJob, job assert_in_delta 5.minutes.from_now, job.scheduled_at, 1 assert_equal "default", job.queue_name - assert_equal [1, 2, 3], job.arguments + assert_equal [1, 2, 3, { keyword: true }], job.arguments + end + + def test_assert_enqueued_with_with_no_block_returns + LoggingJob.set(wait_until: 5.minutes.from_now).perform_later(1, 2, 3, keyword: true) + job = assert_enqueued_with(job: LoggingJob) + + assert_instance_of LoggingJob, job + assert_in_delta 5.minutes.from_now, job.scheduled_at, 1 + assert_equal "default", job.queue_name + assert_equal [1, 2, 3, { keyword: true }], job.arguments end - def test_assert_enqueued_job_failure + def test_assert_enqueued_with_failure assert_raise ActiveSupport::TestCase::Assertion do assert_enqueued_with(job: LoggingJob, queue: "default") do NestedJob.perform_later end end + assert_raise ActiveSupport::TestCase::Assertion do + LoggingJob.perform_later + assert_enqueued_with(job: LoggingJob) { } + end + error = assert_raise ActiveSupport::TestCase::Assertion do assert_enqueued_with(job: NestedJob, queue: "low") do NestedJob.perform_later @@ -422,7 +536,21 @@ class EnqueuedJobsTest < ActiveJob::TestCase assert_equal 'No enqueued job found with {:job=>NestedJob, :queue=>"low"}', error.message end - def test_assert_enqueued_job_args + def test_assert_enqueued_with_with_no_block_failure + assert_raise ActiveSupport::TestCase::Assertion do + NestedJob.perform_later + assert_enqueued_with(job: LoggingJob, queue: "default") + end + + error = assert_raise ActiveSupport::TestCase::Assertion do + NestedJob.perform_later + assert_enqueued_with(job: NestedJob, queue: "low") + end + + assert_equal 'No enqueued job found with {:job=>NestedJob, :queue=>"low"}', error.message + end + + def test_assert_enqueued_with_args assert_raise ArgumentError do assert_enqueued_with(class: LoggingJob) do NestedJob.set(wait_until: Date.tomorrow.noon).perform_later @@ -430,20 +558,94 @@ class EnqueuedJobsTest < ActiveJob::TestCase end end - def test_assert_enqueued_job_with_at_option + def test_assert_enqueued_with_selective_args + args = ->(job_args) do + assert_equal 1, job_args.first[:argument1] + assert job_args.first[:argument2].key?(:b) + end + + assert_enqueued_with(job: MultipleKwargsJob, args: args) do + MultipleKwargsJob.perform_later(argument2: { b: 2, a: 1 }, argument1: 1) + end + end + + def test_assert_enqueued_with_selective_args_fails + args = ->(job_args) do + false + end + + assert_raise ActiveSupport::TestCase::Assertion do + assert_enqueued_with(job: MultipleKwargsJob, args: args) do + MultipleKwargsJob.perform_later(argument2: { b: 2, a: 1 }, argument1: 1) + end + end + end + + def test_assert_enqueued_with_time + now = Time.now + args = [{ argument1: [now] }] + + assert_enqueued_with(job: MultipleKwargsJob, args: args) do + MultipleKwargsJob.perform_later(argument1: [now]) + end + end + + def test_assert_enqueued_with_date_time + now = DateTime.now + args = [{ argument1: [now] }] + + assert_enqueued_with(job: MultipleKwargsJob, args: args) do + MultipleKwargsJob.perform_later(argument1: [now]) + end + end + + def test_assert_enqueued_with_time_with_zone + now = Time.now.in_time_zone("Tokyo") + args = [{ argument1: [now] }] + + assert_enqueued_with(job: MultipleKwargsJob, args: args) do + MultipleKwargsJob.perform_later(argument1: [now]) + end + end + + def test_assert_enqueued_with_with_no_block_args + assert_raise ArgumentError do + NestedJob.set(wait_until: Date.tomorrow.noon).perform_later + assert_enqueued_with(class: LoggingJob) + end + end + + def test_assert_enqueued_with_with_at_option assert_enqueued_with(job: HelloJob, at: Date.tomorrow.noon) do HelloJob.set(wait_until: Date.tomorrow.noon).perform_later end end - def test_assert_enqueued_job_with_global_id_args + def test_assert_enqueued_with_with_no_block_with_at_option + HelloJob.set(wait_until: Date.tomorrow.noon).perform_later + assert_enqueued_with(job: HelloJob, at: Date.tomorrow.noon) + end + + def test_assert_enqueued_with_with_hash_arg + assert_enqueued_with(job: MultipleKwargsJob, args: [{ argument1: 1, argument2: { a: 1, b: 2 } }]) do + MultipleKwargsJob.perform_later(argument2: { b: 2, a: 1 }, argument1: 1) + end + end + + def test_assert_enqueued_with_with_global_id_args ricardo = Person.new(9) assert_enqueued_with(job: HelloJob, args: [ricardo]) do HelloJob.perform_later(ricardo) end end - def test_assert_enqueued_job_failure_with_global_id_args + def test_assert_enqueued_with_with_no_block_with_global_id_args + ricardo = Person.new(9) + HelloJob.perform_later(ricardo) + assert_enqueued_with(job: HelloJob, args: [ricardo]) + end + + def test_assert_enqueued_with_failure_with_global_id_args ricardo = Person.new(9) wilma = Person.new(11) error = assert_raise ActiveSupport::TestCase::Assertion do @@ -455,7 +657,18 @@ class EnqueuedJobsTest < ActiveJob::TestCase assert_equal "No enqueued job found with {:job=>HelloJob, :args=>[#{wilma.inspect}]}", error.message end - def test_assert_enqueued_job_does_not_change_jobs_count + def test_assert_enqueued_with_failure_with_no_block_with_global_id_args + ricardo = Person.new(9) + wilma = Person.new(11) + error = assert_raise ActiveSupport::TestCase::Assertion do + HelloJob.perform_later(ricardo) + assert_enqueued_with(job: HelloJob, args: [wilma]) + end + + assert_equal "No enqueued job found with {:job=>HelloJob, :args=>[#{wilma.inspect}]}", error.message + end + + def test_assert_enqueued_with_does_not_change_jobs_count HelloJob.perform_later assert_enqueued_with(job: HelloJob) do HelloJob.perform_later @@ -463,10 +676,18 @@ class EnqueuedJobsTest < ActiveJob::TestCase assert_equal 2, queue_adapter.enqueued_jobs.count end + + def test_assert_enqueued_with_with_no_block_does_not_change_jobs_count + HelloJob.perform_later + HelloJob.perform_later + assert_enqueued_with(job: HelloJob) + + assert_equal 2, queue_adapter.enqueued_jobs.count + end end class PerformedJobsTest < ActiveJob::TestCase - def test_performed_enqueue_jobs_with_only_option_doesnt_leak_outside_the_block + def test_perform_enqueued_jobs_with_only_option_doesnt_leak_outside_the_block assert_nil queue_adapter.filter perform_enqueued_jobs only: HelloJob do assert_equal HelloJob, queue_adapter.filter @@ -474,7 +695,13 @@ class PerformedJobsTest < ActiveJob::TestCase assert_nil queue_adapter.filter end - def test_performed_enqueue_jobs_with_except_option_doesnt_leak_outside_the_block + def test_perform_enqueued_jobs_without_block_with_only_option_doesnt_leak + perform_enqueued_jobs only: HelloJob + + assert_nil queue_adapter.filter + end + + def test_perform_enqueued_jobs_with_except_option_doesnt_leak_outside_the_block assert_nil queue_adapter.reject perform_enqueued_jobs except: HelloJob do assert_equal HelloJob, queue_adapter.reject @@ -482,6 +709,150 @@ class PerformedJobsTest < ActiveJob::TestCase assert_nil queue_adapter.reject end + def test_perform_enqueued_jobs_without_block_with_except_option_doesnt_leak + perform_enqueued_jobs except: HelloJob + + assert_nil queue_adapter.reject + end + + def test_perform_enqueued_jobs_with_queue_option_doesnt_leak_outside_the_block + assert_nil queue_adapter.queue + perform_enqueued_jobs queue: :some_queue do + assert_equal :some_queue, queue_adapter.queue + end + assert_nil queue_adapter.queue + end + + def test_perform_enqueued_jobs_without_block_with_queue_option_doesnt_leak + perform_enqueued_jobs queue: :some_queue + + assert_nil queue_adapter.reject + end + + def test_perform_enqueued_jobs_with_block + perform_enqueued_jobs do + HelloJob.perform_later("kevin") + LoggingJob.perform_later("bogdan") + end + + assert_performed_jobs 2 + end + + def test_perform_enqueued_jobs_without_block + HelloJob.perform_later("kevin") + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs + + assert_performed_jobs 2 + end + + def test_perform_enqueued_jobs_with_block_with_only_option + perform_enqueued_jobs only: LoggingJob do + HelloJob.perform_later("kevin") + LoggingJob.perform_later("bogdan") + end + + assert_performed_jobs 1 + assert_performed_jobs 1, only: LoggingJob + end + + def test_perform_enqueued_jobs_without_block_with_only_option + HelloJob.perform_later("kevin") + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs only: LoggingJob + + assert_performed_jobs 1 + assert_performed_jobs 1, only: LoggingJob + end + + def test_perform_enqueued_jobs_with_block_with_except_option + perform_enqueued_jobs except: HelloJob do + HelloJob.perform_later("kevin") + LoggingJob.perform_later("bogdan") + end + + assert_performed_jobs 1 + assert_performed_jobs 1, only: LoggingJob + end + + def test_perform_enqueued_jobs_without_block_with_except_option + HelloJob.perform_later("kevin") + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs except: HelloJob + + assert_performed_jobs 1 + assert_performed_jobs 1, only: LoggingJob + end + + def test_perform_enqueued_jobs_with_block_with_queue_option + perform_enqueued_jobs queue: :some_queue do + HelloJob.set(queue: :some_queue).perform_later("kevin") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + LoggingJob.perform_later("bogdan") + end + + assert_performed_jobs 1 + assert_performed_jobs 1, only: HelloJob, queue: :some_queue + end + + def test_perform_enqueued_jobs_without_block_with_queue_option + HelloJob.set(queue: :some_queue).perform_later("kevin") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs queue: :some_queue + + assert_performed_jobs 1 + assert_performed_jobs 1, only: HelloJob, queue: :some_queue + end + + def test_perform_enqueued_jobs_with_block_with_only_and_queue_options + perform_enqueued_jobs only: HelloJob, queue: :other_queue do + HelloJob.set(queue: :some_queue).perform_later("kevin") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + LoggingJob.set(queue: :other_queue).perform_later("bogdan") + end + + assert_performed_jobs 1 + assert_performed_jobs 1, only: HelloJob, queue: :other_queue + end + + def test_perform_enqueued_jobs_without_block_with_only_and_queue_options + HelloJob.set(queue: :some_queue).perform_later("kevin") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + LoggingJob.set(queue: :other_queue).perform_later("bogdan") + + perform_enqueued_jobs only: HelloJob, queue: :other_queue + + assert_performed_jobs 1 + assert_performed_jobs 1, only: HelloJob, queue: :other_queue + end + + def test_perform_enqueued_jobs_with_block_with_except_and_queue_options + perform_enqueued_jobs except: HelloJob, queue: :other_queue do + HelloJob.set(queue: :other_queue).perform_later("kevin") + LoggingJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :other_queue).perform_later("bogdan") + end + + assert_performed_jobs 1 + assert_performed_jobs 1, only: LoggingJob, queue: :other_queue + end + + def test_perform_enqueued_jobs_without_block_with_except_and_queue_options + HelloJob.set(queue: :other_queue).perform_later("kevin") + LoggingJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :other_queue).perform_later("bogdan") + + perform_enqueued_jobs except: HelloJob, queue: :other_queue + + assert_performed_jobs 1 + assert_performed_jobs 1, only: LoggingJob, queue: :other_queue + end + def test_assert_performed_jobs assert_nothing_raised do assert_performed_jobs 1 do @@ -587,6 +958,46 @@ class PerformedJobsTest < ActiveJob::TestCase end end + def test_assert_performed_jobs_with_only_option_as_proc + assert_nothing_raised do + assert_performed_jobs(1, only: ->(job) { job.is_a?(HelloJob) }) do + HelloJob.perform_later("jeremy") + LoggingJob.perform_later("bogdan") + end + end + end + + def test_assert_performed_jobs_without_block_with_only_option + HelloJob.perform_later("jeremy") + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs + + assert_performed_jobs 1, only: HelloJob + end + + def test_assert_performed_jobs_without_block_with_only_option_as_proc + HelloJob.perform_later("jeremy") + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs + + assert_performed_jobs(1, only: ->(job) { job.fetch(:job).name == "HelloJob" }) + end + + def test_assert_performed_jobs_without_block_with_only_option_failure + LoggingJob.perform_later("jeremy") + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs + + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_jobs 1, only: HelloJob + end + + assert_match(/1 .* but 0/, error.message) + end + def test_assert_performed_jobs_with_except_option assert_nothing_raised do assert_performed_jobs 1, except: LoggingJob do @@ -596,6 +1007,46 @@ class PerformedJobsTest < ActiveJob::TestCase end end + def test_assert_performed_jobs_with_except_option_as_proc + assert_nothing_raised do + assert_performed_jobs(1, except: ->(job) { job.is_a?(HelloJob) }) do + HelloJob.perform_later("jeremy") + LoggingJob.perform_later("bogdan") + end + end + end + + def test_assert_performed_jobs_without_block_with_except_option + HelloJob.perform_later("jeremy") + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs + + assert_performed_jobs 1, except: HelloJob + end + + def test_assert_performed_jobs_without_block_with_except_option_as_proc + HelloJob.perform_later("jeremy") + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs + + assert_performed_jobs(1, except: ->(job) { job.fetch(:job).name == "HelloJob" }) + end + + def test_assert_performed_jobs_without_block_with_except_option_failure + HelloJob.perform_later("jeremy") + HelloJob.perform_later("bogdan") + + perform_enqueued_jobs + + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_jobs 1, except: HelloJob + end + + assert_match(/1 .* but 0/, error.message) + end + def test_assert_performed_jobs_with_only_and_except_option error = assert_raise ArgumentError do assert_performed_jobs 1, only: HelloJob, except: HelloJob do @@ -607,6 +1058,19 @@ class PerformedJobsTest < ActiveJob::TestCase assert_match(/`:only` and `:except`/, error.message) end + def test_assert_performed_jobs_without_block_with_only_and_except_options + error = assert_raise ArgumentError do + HelloJob.perform_later("jeremy") + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs + + assert_performed_jobs 1, only: HelloJob, except: HelloJob + end + + assert_match(/`:only` and `:except`/, error.message) + end + def test_assert_performed_jobs_with_only_option_as_array assert_nothing_raised do assert_performed_jobs 2, only: [HelloJob, LoggingJob] do @@ -732,6 +1196,134 @@ class PerformedJobsTest < ActiveJob::TestCase assert_match(/`:only` and `:except`/, error.message) end + def test_assert_performed_jobs_with_queue_option + assert_performed_jobs 1, queue: :some_queue do + HelloJob.set(queue: :some_queue).perform_later("jeremy") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + end + end + + def test_assert_performed_jobs_with_queue_option_failure + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_jobs 1, queue: :some_queue do + HelloJob.set(queue: :other_queue).perform_later("jeremy") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + end + end + + assert_match(/1 .* but 0/, error.message) + end + + def test_assert_performed_jobs_without_block_with_queue_option + HelloJob.set(queue: :some_queue).perform_later("jeremy") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + + perform_enqueued_jobs + + assert_performed_jobs 1, queue: :some_queue + end + + def test_assert_performed_jobs_without_block_with_queue_option_failure + HelloJob.set(queue: :other_queue).perform_later("jeremy") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + + perform_enqueued_jobs + + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_jobs 1, queue: :some_queue + end + + assert_match(/1 .* but 0/, error.message) + end + + def test_assert_performed_jobs_with_only_and_queue_options + assert_performed_jobs 1, only: HelloJob, queue: :some_queue do + HelloJob.set(queue: :some_queue).perform_later("jeremy") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + end + end + + def test_assert_performed_jobs_with_only_and_queue_options_failure + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_jobs 1, only: HelloJob, queue: :some_queue do + HelloJob.set(queue: :other_queue).perform_later("jeremy") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + end + end + + assert_match(/1 .* but 0/, error.message) + end + + def test_assert_performed_jobs_without_block_with_only_and_queue_options + HelloJob.set(queue: :some_queue).perform_later("jeremy") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + + perform_enqueued_jobs + + assert_performed_jobs 1, only: HelloJob, queue: :some_queue + end + + def test_assert_performed_jobs_without_block_with_only_and_queue_options_failure + HelloJob.set(queue: :other_queue).perform_later("jeremy") + HelloJob.set(queue: :other_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + + perform_enqueued_jobs + + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_jobs 1, only: HelloJob, queue: :some_queue + end + + assert_match(/1 .* but 0/, error.message) + end + + def test_assert_performed_jobs_with_except_and_queue_options + assert_performed_jobs 1, except: HelloJob, queue: :other_queue do + HelloJob.set(queue: :other_queue).perform_later("jeremy") + LoggingJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :other_queue).perform_later("jeremy") + end + end + + def test_assert_performed_jobs_with_except_and_queue_options_failure + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_jobs 1, except: HelloJob, queue: :other_queue do + HelloJob.set(queue: :other_queue).perform_later("jeremy") + LoggingJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + end + end + + assert_match(/1 .* but 0/, error.message) + end + + def test_assert_performed_jobs_without_block_with_except_and_queue_options + HelloJob.set(queue: :other_queue).perform_later("jeremy") + LoggingJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :other_queue).perform_later("jeremy") + + perform_enqueued_jobs + + assert_performed_jobs 1, except: HelloJob, queue: :other_queue + end + + def test_assert_performed_jobs_without_block_with_except_and_queue_options_failure + HelloJob.set(queue: :other_queue).perform_later("jeremy") + LoggingJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + + perform_enqueued_jobs + + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_jobs 1, except: HelloJob, queue: :other_queue + end + + assert_match(/1 .* but 0/, error.message) + end + def test_assert_no_performed_jobs_with_only_option assert_nothing_raised do assert_no_performed_jobs only: HelloJob do @@ -740,6 +1332,26 @@ class PerformedJobsTest < ActiveJob::TestCase end end + def test_assert_no_performed_jobs_without_block_with_only_option + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs + + assert_no_performed_jobs only: HelloJob + end + + def test_assert_no_performed_jobs_without_block_with_only_option_failure + HelloJob.perform_later("bogdan") + + perform_enqueued_jobs + + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_no_performed_jobs only: HelloJob + end + + assert_match(/0 .* but 1/, error.message) + end + def test_assert_no_performed_jobs_with_except_option assert_nothing_raised do assert_no_performed_jobs except: LoggingJob do @@ -748,6 +1360,26 @@ class PerformedJobsTest < ActiveJob::TestCase end end + def test_assert_no_performed_jobs_without_block_with_except_option + HelloJob.perform_later("jeremy") + + perform_enqueued_jobs + + assert_no_performed_jobs except: HelloJob + end + + def test_assert_no_performed_jobs_without_block_with_except_option_failure + LoggingJob.perform_later("jeremy") + + perform_enqueued_jobs + + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_no_performed_jobs except: HelloJob + end + + assert_match(/0 .* but 1/, error.message) + end + def test_assert_no_performed_jobs_with_only_and_except_option error = assert_raise ArgumentError do assert_no_performed_jobs only: HelloJob, except: HelloJob do @@ -758,6 +1390,19 @@ class PerformedJobsTest < ActiveJob::TestCase assert_match(/`:only` and `:except`/, error.message) end + def test_assert_no_performed_jobs_without_block_with_only_and_except_options + error = assert_raise ArgumentError do + HelloJob.perform_later("jeremy") + LoggingJob.perform_later("bogdan") + + perform_enqueued_jobs + + assert_no_performed_jobs only: HelloJob, except: HelloJob + end + + assert_match(/`:only` and `:except`/, error.message) + end + def test_assert_no_performed_jobs_with_only_option_as_array assert_nothing_raised do assert_no_performed_jobs only: [HelloJob, RescueJob] do @@ -818,24 +1463,165 @@ class PerformedJobsTest < ActiveJob::TestCase assert_match(/`:only` and `:except`/, error.message) end - def test_assert_performed_job + def test_assert_no_performed_jobs_with_queue_option + assert_no_performed_jobs queue: :some_queue do + HelloJob.set(queue: :other_queue).perform_later("jeremy") + end + end + + def test_assert_no_performed_jobs_with_queue_option_failure + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_no_performed_jobs queue: :some_queue do + HelloJob.set(queue: :some_queue).perform_later("jeremy") + end + end + + assert_match(/0 .* but 1/, error.message) + end + + def test_assert_no_performed_jobs_without_block_with_queue_option + HelloJob.set(queue: :other_queue).perform_later("jeremy") + + perform_enqueued_jobs + + assert_no_performed_jobs queue: :some_queue + end + + def test_assert_no_performed_jobs_without_block_with_queue_option_failure + HelloJob.set(queue: :some_queue).perform_later("jeremy") + + perform_enqueued_jobs + + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_no_performed_jobs queue: :some_queue + end + + assert_match(/0 .* but 1/, error.message) + end + + def test_assert_no_performed_jobs_with_only_and_queue_options + assert_no_performed_jobs only: HelloJob, queue: :some_queue do + HelloJob.set(queue: :other_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + end + end + + def test_assert_no_performed_jobs_with_only_and_queue_options_failure + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_no_performed_jobs only: HelloJob, queue: :some_queue do + HelloJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + end + end + + assert_match(/0 .* but 1/, error.message) + end + + def test_assert_no_performed_jobs_without_block_with_only_and_queue_options + HelloJob.set(queue: :other_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + + perform_enqueued_jobs + + assert_no_performed_jobs only: HelloJob, queue: :some_queue + end + + def test_assert_no_performed_jobs_without_block_with_only_and_queue_options_failure + HelloJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + + perform_enqueued_jobs + + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_no_performed_jobs only: HelloJob, queue: :some_queue + end + + assert_match(/0 .* but 1/, error.message) + end + + def test_assert_no_performed_jobs_with_except_and_queue_options + assert_no_performed_jobs except: HelloJob, queue: :some_queue do + HelloJob.set(queue: :other_queue).perform_later("bogdan") + HelloJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :other_queue).perform_later("jeremy") + end + end + + def test_assert_no_performed_jobs_with_except_and_queue_options_failure + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_no_performed_jobs except: HelloJob, queue: :some_queue do + HelloJob.set(queue: :other_queue).perform_later("bogdan") + HelloJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + end + end + + assert_match(/0 .* but 1/, error.message) + end + + def test_assert_no_performed_jobs_without_block_with_except_and_queue_options + HelloJob.set(queue: :other_queue).perform_later("bogdan") + HelloJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :other_queue).perform_later("jeremy") + + perform_enqueued_jobs + + assert_no_performed_jobs except: HelloJob, queue: :some_queue + end + + def test_assert_no_performed_jobs_without_block_with_except_and_queue_options_failure + HelloJob.set(queue: :other_queue).perform_later("bogdan") + HelloJob.set(queue: :some_queue).perform_later("bogdan") + LoggingJob.set(queue: :some_queue).perform_later("jeremy") + + perform_enqueued_jobs + + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_no_performed_jobs except: HelloJob, queue: :some_queue + end + + assert_match(/0 .* but 1/, error.message) + end + + def test_assert_performed_with assert_performed_with(job: NestedJob, queue: "default") do NestedJob.perform_later end end - def test_assert_performed_job_returns - job = assert_performed_with(job: NestedJob, queue: "default") do - NestedJob.perform_later + def test_assert_performed_with_without_block + NestedJob.perform_later + + perform_enqueued_jobs + + assert_performed_with(job: NestedJob, queue: "default") + end + + def test_assert_performed_with_returns + job = assert_performed_with(job: LoggingJob, queue: "default") do + LoggingJob.perform_later(keyword: :sym) end - assert_instance_of NestedJob, job + assert_instance_of LoggingJob, job assert_nil job.scheduled_at - assert_equal [], job.arguments + assert_equal [{ keyword: :sym }], job.arguments assert_equal "default", job.queue_name end - def test_assert_performed_job_failure + def test_assert_performed_with_without_block_returns + LoggingJob.perform_later(keyword: :sym) + + perform_enqueued_jobs + + job = assert_performed_with(job: LoggingJob, queue: "default") + + assert_instance_of LoggingJob, job + assert_nil job.scheduled_at + assert_equal [{ keyword: :sym }], job.arguments + assert_equal "default", job.queue_name + end + + def test_assert_performed_with_failure assert_raise ActiveSupport::TestCase::Assertion do assert_performed_with(job: LoggingJob) do HelloJob.perform_later @@ -849,7 +1635,23 @@ class PerformedJobsTest < ActiveJob::TestCase end end - def test_assert_performed_job_with_at_option + def test_assert_performed_with_without_block_failure + HelloJob.perform_later + + perform_enqueued_jobs + + assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_with(job: LoggingJob) + end + + HelloJob.set(queue: "important").perform_later + + assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_with(job: HelloJob, queue: "low") + end + end + + def test_assert_performed_with_with_at_option assert_performed_with(job: HelloJob, at: Date.tomorrow.noon) do HelloJob.set(wait_until: Date.tomorrow.noon).perform_later end @@ -861,14 +1663,93 @@ class PerformedJobsTest < ActiveJob::TestCase end end - def test_assert_performed_job_with_global_id_args + def test_assert_performed_with_without_block_with_at_option + HelloJob.set(wait_until: Date.tomorrow.noon).perform_later + + perform_enqueued_jobs + + assert_performed_with(job: HelloJob, at: Date.tomorrow.noon) + + HelloJob.set(wait_until: Date.tomorrow.noon).perform_later + + perform_enqueued_jobs + + assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_with(job: HelloJob, at: Date.today.noon) + end + end + + def test_assert_performed_with_with_hash_arg + assert_performed_with(job: MultipleKwargsJob, args: [{ argument1: 1, argument2: { a: 1, b: 2 } }]) do + MultipleKwargsJob.perform_later(argument2: { b: 2, a: 1 }, argument1: 1) + end + end + + def test_assert_performed_with_selective_args + args = ->(job_args) do + assert_equal 1, job_args.first[:argument1] + assert job_args.first[:argument2].key?(:b) + end + + assert_performed_with(job: MultipleKwargsJob, args: args) do + MultipleKwargsJob.perform_later(argument2: { b: 2, a: 1 }, argument1: 1) + end + end + + def test_assert_performed_with_selective_args_fails + args = ->(job_args) do + false + end + + assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_with(job: MultipleKwargsJob, args: args) do + MultipleKwargsJob.perform_later(argument2: { b: 2, a: 1 }, argument1: 1) + end + end + end + + def test_assert_performed_with_time + now = Time.now + args = [{ argument1: { now: now } }] + + assert_enqueued_with(job: MultipleKwargsJob, args: args) do + MultipleKwargsJob.perform_later(argument1: { now: now }) + end + end + + def test_assert_performed_with_date_time + now = DateTime.now + args = [{ argument1: { now: now } }] + + assert_enqueued_with(job: MultipleKwargsJob, args: args) do + MultipleKwargsJob.perform_later(argument1: { now: now }) + end + end + + def test_assert_performed_with_time_with_zone + now = Time.now.in_time_zone("Tokyo") + args = [{ argument1: { now: now } }] + + assert_enqueued_with(job: MultipleKwargsJob, args: args) do + MultipleKwargsJob.perform_later(argument1: { now: now }) + end + end + + def test_assert_performed_with_with_global_id_args ricardo = Person.new(9) assert_performed_with(job: HelloJob, args: [ricardo]) do HelloJob.perform_later(ricardo) end end - def test_assert_performed_job_failure_with_global_id_args + def test_assert_performed_with_without_block_with_global_id_args + ricardo = Person.new(9) + HelloJob.perform_later(ricardo) + perform_enqueued_jobs + assert_performed_with(job: HelloJob, args: [ricardo]) + end + + def test_assert_performed_with_failure_with_global_id_args ricardo = Person.new(9) wilma = Person.new(11) error = assert_raise ActiveSupport::TestCase::Assertion do @@ -880,7 +1761,19 @@ class PerformedJobsTest < ActiveJob::TestCase assert_equal "No performed job found with {:job=>HelloJob, :args=>[#{wilma.inspect}]}", error.message end - def test_assert_performed_job_does_not_change_jobs_count + def test_assert_performed_with_without_block_failure_with_global_id_args + ricardo = Person.new(9) + wilma = Person.new(11) + HelloJob.perform_later(ricardo) + perform_enqueued_jobs + error = assert_raise ActiveSupport::TestCase::Assertion do + assert_performed_with(job: HelloJob, args: [wilma]) + end + + assert_equal "No performed job found with {:job=>HelloJob, :args=>[#{wilma.inspect}]}", error.message + end + + def test_assert_performed_with_does_not_change_jobs_count assert_performed_with(job: HelloJob) do HelloJob.perform_later end @@ -891,6 +1784,18 @@ class PerformedJobsTest < ActiveJob::TestCase assert_equal 2, queue_adapter.performed_jobs.count end + + def test_assert_performed_with_without_block_does_not_change_jobs_count + HelloJob.perform_later + perform_enqueued_jobs + assert_performed_with(job: HelloJob) + + perform_enqueued_jobs + HelloJob.perform_later + assert_performed_with(job: HelloJob) + + assert_equal 2, queue_adapter.performed_jobs.count + end end class OverrideQueueAdapterTest < ActiveJob::TestCase diff --git a/activejob/test/helper.rb b/activejob/test/helper.rb index 694232d7ef..d400210fef 100644 --- a/activejob/test/helper.rb +++ b/activejob/test/helper.rb @@ -16,3 +16,5 @@ else end require "active_support/testing/autorun" + +require_relative "../../tools/test_common" diff --git a/activejob/test/integration/queuing_test.rb b/activejob/test/integration/queuing_test.rb index 32afb5ca62..1fa68a8ad5 100644 --- a/activejob/test/integration/queuing_test.rb +++ b/activejob/test/integration/queuing_test.rb @@ -60,25 +60,21 @@ class QueuingTest < ActiveSupport::TestCase end test "should not run job enqueued in the future" do - begin - TestJob.set(wait: 10.minutes).perform_later @id - wait_for_jobs_to_finish_for(5.seconds) - assert_not job_executed - rescue NotImplementedError - skip - end + TestJob.set(wait: 10.minutes).perform_later @id + wait_for_jobs_to_finish_for(5.seconds) + assert_not job_executed + rescue NotImplementedError + skip end test "should run job enqueued in the future at the specified time" do - begin - TestJob.set(wait: 5.seconds).perform_later @id - wait_for_jobs_to_finish_for(2.seconds) - assert_not job_executed - wait_for_jobs_to_finish_for(10.seconds) - assert job_executed - rescue NotImplementedError - skip - end + TestJob.set(wait: 5.seconds).perform_later @id + wait_for_jobs_to_finish_for(2.seconds) + assert_not job_executed + wait_for_jobs_to_finish_for(10.seconds) + assert job_executed + rescue NotImplementedError + skip end test "should supply a provider_job_id when available for immediate jobs" do @@ -137,4 +133,16 @@ class QueuingTest < ActiveSupport::TestCase assert job_executed "#{@id}.2" assert job_executed_at("#{@id}.2") < job_executed_at("#{@id}.1") end + + test "should run job with higher priority first in Backburner" do + skip unless adapter_is?(:backburner) + + jobs_manager.tube.pause(3) + TestJob.set(priority: 20).perform_later "#{@id}.1" + TestJob.set(priority: 10).perform_later "#{@id}.2" + wait_for_jobs_to_finish_for(10.seconds) + assert job_executed "#{@id}.1" + assert job_executed "#{@id}.2" + assert job_executed_at("#{@id}.2") < job_executed_at("#{@id}.1") + end end diff --git a/activejob/test/jobs/abort_before_enqueue_job.rb b/activejob/test/jobs/abort_before_enqueue_job.rb new file mode 100644 index 0000000000..fd278eccf4 --- /dev/null +++ b/activejob/test/jobs/abort_before_enqueue_job.rb @@ -0,0 +1,9 @@ +# frozen_string_literal: true + +class AbortBeforeEnqueueJob < ActiveJob::Base + before_enqueue { throw(:abort) } + + def perform + raise "This should never be called" + end +end diff --git a/activejob/test/jobs/multiple_kwargs_job.rb b/activejob/test/jobs/multiple_kwargs_job.rb new file mode 100644 index 0000000000..b355c4ce1a --- /dev/null +++ b/activejob/test/jobs/multiple_kwargs_job.rb @@ -0,0 +1,9 @@ +# frozen_string_literal: true + +require_relative "../support/job_buffer" + +class MultipleKwargsJob < ActiveJob::Base + def perform(argument1:, argument2:) + JobBuffer.add("Job with argument1: #{argument1}, argument2: #{argument2}") + end +end diff --git a/activejob/test/jobs/retry_job.rb b/activejob/test/jobs/retry_job.rb index 82b80fe12b..112d672006 100644 --- a/activejob/test/jobs/retry_job.rb +++ b/activejob/test/jobs/retry_job.rb @@ -4,26 +4,41 @@ require_relative "../support/job_buffer" require "active_support/core_ext/integer/inflections" class DefaultsError < StandardError; end +class FirstRetryableErrorOfTwo < StandardError; end +class SecondRetryableErrorOfTwo < StandardError; end class LongWaitError < StandardError; end class ShortWaitTenAttemptsError < StandardError; end class ExponentialWaitTenAttemptsError < StandardError; end class CustomWaitTenAttemptsError < StandardError; end class CustomCatchError < StandardError; end class DiscardableError < StandardError; end +class FirstDiscardableErrorOfTwo < StandardError; end +class SecondDiscardableErrorOfTwo < StandardError; end class CustomDiscardableError < StandardError; end class RetryJob < ActiveJob::Base retry_on DefaultsError + retry_on FirstRetryableErrorOfTwo, SecondRetryableErrorOfTwo, attempts: 4 retry_on LongWaitError, wait: 1.hour, attempts: 10 retry_on ShortWaitTenAttemptsError, wait: 1.second, attempts: 10 retry_on ExponentialWaitTenAttemptsError, wait: :exponentially_longer, attempts: 10 retry_on CustomWaitTenAttemptsError, wait: ->(executions) { executions * 2 }, attempts: 10 - retry_on(CustomCatchError) { |job, exception| JobBuffer.add("Dealt with a job that failed to retry in a custom way after #{job.arguments.second} attempts. Message: #{exception.message}") } + retry_on(CustomCatchError) { |job, error| JobBuffer.add("Dealt with a job that failed to retry in a custom way after #{job.arguments.second} attempts. Message: #{error.message}") } + retry_on(ActiveJob::DeserializationError) { |job, error| JobBuffer.add("Raised #{error.class} for the #{job.executions} time") } + discard_on DiscardableError - discard_on(CustomDiscardableError) { |job, exception| JobBuffer.add("Dealt with a job that was discarded in a custom way") } + discard_on FirstDiscardableErrorOfTwo, SecondDiscardableErrorOfTwo + discard_on(CustomDiscardableError) { |job, error| JobBuffer.add("Dealt with a job that was discarded in a custom way. Message: #{error.message}") } + + before_enqueue do |job| + if job.arguments.include?(:log_scheduled_at) && job.scheduled_at + JobBuffer.add("Next execution scheduled at #{job.scheduled_at}") + end + end - def perform(raising, attempts) - if executions < attempts + def perform(raising, attempts, *) + raising = raising.shift if raising.is_a?(Array) + if raising && executions < attempts JobBuffer.add("Raised #{raising} for the #{executions.ordinalize} time") raise raising.constantize else diff --git a/activejob/test/support/integration/adapters/backburner.rb b/activejob/test/support/integration/adapters/backburner.rb index eb179011d9..0c248dda01 100644 --- a/activejob/test/support/integration/adapters/backburner.rb +++ b/activejob/test/support/integration/adapters/backburner.rb @@ -4,11 +4,13 @@ module BackburnerJobsManager def setup ActiveJob::Base.queue_adapter = :backburner Backburner.configure do |config| + config.beanstalk_url = ENV["BEANSTALK_URL"] if ENV["BEANSTALK_URL"] config.logger = Rails.logger end unless can_run? puts "Cannot run integration tests for backburner. To be able to run integration tests for backburner you need to install and start beanstalkd.\n" - exit + status = ENV["CI"] ? false : true + exit status end end diff --git a/activejob/test/support/integration/adapters/que.rb b/activejob/test/support/integration/adapters/que.rb index 2a771b08c7..f231e5e12d 100644 --- a/activejob/test/support/integration/adapters/que.rb +++ b/activejob/test/support/integration/adapters/que.rb @@ -18,8 +18,8 @@ module QueJobsManager user = uri.user || ENV["USER"] pass = uri.password db = uri.path[1..-1] - %x{#{"PGPASSWORD=\"#{pass}\"" if pass} psql -c 'drop database if exists "#{db}"' -U #{user} -t template1} - %x{#{"PGPASSWORD=\"#{pass}\"" if pass} psql -c 'create database "#{db}"' -U #{user} -t template1} + %x{#{"PGPASSWORD=\"#{pass}\"" if pass} psql -X -c 'drop database if exists "#{db}"' -U #{user} -t template1} + %x{#{"PGPASSWORD=\"#{pass}\"" if pass} psql -X -c 'create database "#{db}"' -U #{user} -t template1} Que.connection = Sequel.connect(que_url) Que.migrate! @@ -32,7 +32,8 @@ module QueJobsManager rescue Sequel::DatabaseConnectionError puts "Cannot run integration tests for que. To be able to run integration tests for que you need to install and start postgresql.\n" - exit + status = ENV["CI"] ? false : true + exit status end def stop_workers diff --git a/activejob/test/support/integration/adapters/queue_classic.rb b/activejob/test/support/integration/adapters/queue_classic.rb index 1b0685a971..2b5375461a 100644 --- a/activejob/test/support/integration/adapters/queue_classic.rb +++ b/activejob/test/support/integration/adapters/queue_classic.rb @@ -17,8 +17,8 @@ module QueueClassicJobsManager user = uri.user || ENV["USER"] pass = uri.password db = uri.path[1..-1] - %x{#{"PGPASSWORD=\"#{pass}\"" if pass} psql -c 'drop database if exists "#{db}"' -U #{user} -t template1} - %x{#{"PGPASSWORD=\"#{pass}\"" if pass} psql -c 'create database "#{db}"' -U #{user} -t template1} + %x{#{"PGPASSWORD=\"#{pass}\"" if pass} psql -X -c 'drop database if exists "#{db}"' -U #{user} -t template1} + %x{#{"PGPASSWORD=\"#{pass}\"" if pass} psql -X -c 'create database "#{db}"' -U #{user} -t template1} QC::Setup.create QC.default_conn_adapter.disconnect @@ -30,7 +30,8 @@ module QueueClassicJobsManager rescue PG::ConnectionBad puts "Cannot run integration tests for queue_classic. To be able to run integration tests for queue_classic you need to install and start postgresql.\n" - exit + status = ENV["CI"] ? false : true + exit status end def stop_workers diff --git a/activejob/test/support/integration/adapters/resque.rb b/activejob/test/support/integration/adapters/resque.rb index 2ed8302277..cd129e72b2 100644 --- a/activejob/test/support/integration/adapters/resque.rb +++ b/activejob/test/support/integration/adapters/resque.rb @@ -3,7 +3,7 @@ module ResqueJobsManager def setup ActiveJob::Base.queue_adapter = :resque - Resque.redis = Redis::Namespace.new "active_jobs_int_test", redis: Redis.new(url: "redis://127.0.0.1:6379/12", thread_safe: true) + Resque.redis = Redis::Namespace.new "active_jobs_int_test", redis: Redis.new(url: ENV["REDIS_URL"] || "redis://127.0.0.1:6379/12", thread_safe: true) Resque.logger = Rails.logger unless can_run? puts "Cannot run integration tests for resque. To be able to run integration tests for resque you need to install and start redis.\n" diff --git a/activejob/test/support/integration/adapters/sneakers.rb b/activejob/test/support/integration/adapters/sneakers.rb index 965e6e2e6c..89dc61ca28 100644 --- a/activejob/test/support/integration/adapters/sneakers.rb +++ b/activejob/test/support/integration/adapters/sneakers.rb @@ -1,24 +1,13 @@ # frozen_string_literal: true require "sneakers/runner" -require "sneakers/publisher" require "timeout" -module Sneakers - class Publisher - def safe_ensure_connected - @mutex.synchronize do - ensure_connection! unless connected? - end - end - end -end - module SneakersJobsManager def setup ActiveJob::Base.queue_adapter = :sneakers Sneakers.configure heartbeat: 2, - amqp: "amqp://guest:guest@localhost:5672", + amqp: ENV["RABBITMQ_URL"] || "amqp://guest:guest@localhost:5672", vhost: "/", exchange: "active_jobs_sneakers_int_test", exchange_type: :direct, @@ -29,7 +18,8 @@ module SneakersJobsManager log: Rails.root.join("log/sneakers.log").to_s unless can_run? puts "Cannot run integration tests for sneakers. To be able to run integration tests for sneakers you need to install and start rabbitmq.\n" - exit + status = ENV["CI"] ? false : true + exit status end end @@ -79,7 +69,7 @@ module SneakersJobsManager def bunny_publisher @bunny_publisher ||= begin p = ActiveJob::QueueAdapters::SneakersAdapter::JobWrapper.send(:publisher) - p.safe_ensure_connected + p.ensure_connection! p end end diff --git a/activejob/test/support/integration/helper.rb b/activejob/test/support/integration/helper.rb index a02d874e2e..c5fa2b136f 100644 --- a/activejob/test/support/integration/helper.rb +++ b/activejob/test/support/integration/helper.rb @@ -1,6 +1,6 @@ # frozen_string_literal: true -puts "\n\n*** rake aj:integration:#{ENV['AJ_ADAPTER']} ***\n" +puts "\n\n*** rake test:integration:#{ENV['AJ_ADAPTER']} ***\n" ENV["RAILS_ENV"] = "test" ActiveJob::Base.queue_name_prefix = nil diff --git a/activejob/test/support/integration/test_case_helpers.rb b/activejob/test/support/integration/test_case_helpers.rb index 3d9b265b66..973ee07764 100644 --- a/activejob/test/support/integration/test_case_helpers.rb +++ b/activejob/test/support/integration/test_case_helpers.rb @@ -33,14 +33,12 @@ module TestCaseHelpers end def wait_for_jobs_to_finish_for(seconds = 60) - begin - Timeout.timeout(seconds) do - while !job_executed do - sleep 0.25 - end + Timeout.timeout(seconds) do + while !job_executed do + sleep 0.25 end - rescue Timeout::Error end + rescue Timeout::Error end def job_file(id) diff --git a/activejob/test/support/queue_classic/inline.rb b/activejob/test/support/queue_classic/inline.rb index ca3cd4581b..0695a34c27 100644 --- a/activejob/test/support/queue_classic/inline.rb +++ b/activejob/test/support/queue_classic/inline.rb @@ -1,22 +1,23 @@ # frozen_string_literal: true require "queue_classic" +require "active_support/core_ext/module/redefine_method" module QC class Queue - def enqueue(method, *args) + redefine_method(:enqueue) do |method, *args| receiver_str, _, message = method.rpartition(".") receiver = eval(receiver_str) receiver.send(message, *args) end - def enqueue_in(seconds, method, *args) + redefine_method(:enqueue_in) do |seconds, method, *args| receiver_str, _, message = method.rpartition(".") receiver = eval(receiver_str) receiver.send(message, *args) end - def enqueue_at(not_before, method, *args) + redefine_method(:enqueue_at) do |not_before, method, *args| receiver_str, _, message = method.rpartition(".") receiver = eval(receiver_str) receiver.send(message, *args) diff --git a/activejob/test/support/sneakers/inline.rb b/activejob/test/support/sneakers/inline.rb index 92b69ee3bc..e772c68c6e 100644 --- a/activejob/test/support/sneakers/inline.rb +++ b/activejob/test/support/sneakers/inline.rb @@ -1,11 +1,12 @@ # frozen_string_literal: true require "sneakers" +require "active_support/core_ext/module/redefine_method" module Sneakers module Worker module ClassMethods - def enqueue(msg) + redefine_method(:enqueue) do |msg| worker = new(nil, nil, {}) worker.work(*msg) end diff --git a/activejob/test/support/stubs/strong_parameters.rb b/activejob/test/support/stubs/strong_parameters.rb new file mode 100644 index 0000000000..acba3a4504 --- /dev/null +++ b/activejob/test/support/stubs/strong_parameters.rb @@ -0,0 +1,15 @@ +# frozen_string_literal: true + +class Parameters + def initialize(parameters = {}) + @parameters = parameters.with_indifferent_access + end + + def permitted? + true + end + + def to_h + @parameters.to_h + end +end |