diff options
author | Godfrey Chan <godfreykfc@gmail.com> | 2016-10-15 12:27:18 +0200 |
---|---|---|
committer | Godfrey Chan <godfreykfc@gmail.com> | 2016-10-15 12:34:46 +0200 |
commit | f2f9b8868590bb4be8965583096862e29b9f83fb (patch) | |
tree | 2fc45870c6745532cfc29a96cd34c537ba962201 | |
parent | 77ed6b782a16f16c3cea3107f6715324ce9fa10a (diff) | |
download | rails-f2f9b8868590bb4be8965583096862e29b9f83fb.tar.gz rails-f2f9b8868590bb4be8965583096862e29b9f83fb.tar.bz2 rails-f2f9b8868590bb4be8965583096862e29b9f83fb.zip |
Introduce a benchmark template [ci skip]
This replaces boilerplate in the “benchmark your code” section of the
contributors’ guide with an executable template. I also amended the text
to encourage best practices and codified it in the template.
For now this is only good for relatively self-contained changes that can
be inlined into a simple script. In the future, this can be expanded to
cover how to measure the difference between two commits.
The output looks like this:
```
==================================== Empty =====================================
Warming up --------------------------------------
blank? 225.963k i/100ms
fast_blank? 238.147k i/100ms
Calculating -------------------------------------
blank? 8.825M (± 6.4%) i/s - 44.063M in 5.014824s
fast_blank? 9.311M (± 6.3%) i/s - 46.439M in 5.009153s
Comparison:
fast_blank?: 9310694.8 i/s
blank?: 8824801.7 i/s - same-ish: difference falls within error
================================= Single Space =================================
Warming up --------------------------------------
blank? 56.581k i/100ms
fast_blank? 232.774k i/100ms
Calculating -------------------------------------
blank? 813.985k (±16.7%) i/s - 4.017M in 5.076576s
fast_blank? 9.547M (± 5.2%) i/s - 47.719M in 5.013204s
Comparison:
fast_blank?: 9547414.0 i/s
blank?: 813985.0 i/s - 11.73x slower
================================== Two Spaces ==================================
Warming up --------------------------------------
blank? 58.265k i/100ms
fast_blank? 244.056k i/100ms
Calculating -------------------------------------
blank? 823.343k (±16.2%) i/s - 4.020M in 5.014213s
fast_blank? 9.484M (± 4.9%) i/s - 47.347M in 5.005339s
Comparison:
fast_blank?: 9484021.6 i/s
blank?: 823343.1 i/s - 11.52x slower
=============================== Mixed Whitspaces ===============================
Warming up --------------------------------------
blank? 53.919k i/100ms
fast_blank? 237.103k i/100ms
Calculating -------------------------------------
blank? 763.435k (±16.8%) i/s - 3.720M in 5.018029s
fast_blank? 9.672M (± 5.8%) i/s - 48.369M in 5.019356s
Comparison:
fast_blank?: 9672467.2 i/s
blank?: 763435.4 i/s - 12.67x slower
=============================== Very Long String ===============================
Warming up --------------------------------------
blank? 34.037k i/100ms
fast_blank? 240.366k i/100ms
Calculating -------------------------------------
blank? 409.731k (± 8.9%) i/s - 2.042M in 5.028235s
fast_blank? 9.794M (± 4.3%) i/s - 49.035M in 5.016328s
Comparison:
fast_blank?: 9794225.2 i/s
blank?: 409731.4 i/s - 23.90x slower
```
-rw-r--r-- | guides/bug_report_templates/benchmark.rb | 49 | ||||
-rw-r--r-- | guides/source/contributing_to_ruby_on_rails.md | 45 |
2 files changed, 67 insertions, 27 deletions
diff --git a/guides/bug_report_templates/benchmark.rb b/guides/bug_report_templates/benchmark.rb new file mode 100644 index 0000000000..ae51d7027f --- /dev/null +++ b/guides/bug_report_templates/benchmark.rb @@ -0,0 +1,49 @@ +begin + require "bundler/inline" +rescue LoadError => e + $stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler" + raise e +end + +gemfile(true) do + source "https://rubygems.org" + gem "rails", github: "rails/rails" + gem "benchmark-ips" +end + +require "active_support" +require "active_support/core_ext/object/blank" + +# Your patch goes here. +class String + def fast_blank? + true + end +end + +# Enumerate some representative scenarios here. +# +# It is very easy to make an optimization that improves performance for a +# specific scenario you care about but regresses on other common cases. +# Therefore, you should test your change against a list of representative +# scenarios. Ideally, they should be based on real-world scenarios extracted +# from production applications. +SCENARIOS = { + "Empty" => "", + "Single Space" => " ", + "Two Spaces" => " ", + "Mixed Whitspaces" => " \t\r\n", + "Very Long String" => " " * 100 +} + +SCENARIOS.each_pair do |name, value| + puts + puts " #{name} ".center(80, "=") + puts + + Benchmark.ips do |x| + x.report('blank?') { value.blank? } + x.report('fast_blank?') { value.fast_blank? } + x.compare! + end +end diff --git a/guides/source/contributing_to_ruby_on_rails.md b/guides/source/contributing_to_ruby_on_rails.md index 4f938f5deb..5791e7a12a 100644 --- a/guides/source/contributing_to_ruby_on_rails.md +++ b/guides/source/contributing_to_ruby_on_rails.md @@ -270,33 +270,24 @@ The above are guidelines - please use your best judgment in using them. ### Benchmark Your Code -If your change has an impact on the performance of Rails, please use the -[benchmark-ips](https://github.com/evanphx/benchmark-ips) gem to provide -benchmark results for comparison. - -Here's an example of using benchmark-ips: - -```ruby -require 'benchmark/ips' - -Benchmark.ips do |x| - x.report('addition') { 1 + 2 } - x.report('addition with send') { 1.send(:+, 2) } -end -``` - -This will generate a report with the following information: - -``` -Calculating ------------------------------------- - addition 132.013k i/100ms - addition with send 125.413k i/100ms -------------------------------------------------- - addition 9.677M (± 1.7%) i/s - 48.449M - addition with send 6.794M (± 1.1%) i/s - 33.987M -``` - -Please see the benchmark/ips [README](https://github.com/evanphx/benchmark-ips/blob/master/README.md) for more information. +For changes that might have an impact on performance, please benchmark your +code and measure the impact. Please share the benchmark script you used as well +as the results. You should consider including these information in your commit +message, which allows future contributors to easily verify your findings and +determine if they are still relevant. (For example, future optimizations in the +Ruby VM might render certain optimizations unnecessary.) + +It is very easy to make an optimization that improves performance for a +specific scenario you care about but regresses on other common cases. +Therefore, you should test your change against a list of representative +scenarios. Ideally, they should be based on real-world scenarios extracted +from production applications. + +You can use the [benchmark template](https://github.com/rails/rails/blob/master/guides/bug_report_templates/benchmark.rb) +as a starting point. It includes the boilerplate code to setup a benchmark +using the [benchmark-ips](https://github.com/evanphx/benchmark-ips) gem. The +template is designed for testing relatively self-contained changes that can be +inlined into the script. ### Running Tests |