aboutsummaryrefslogtreecommitdiffstats
path: root/guides/source/performance_testing.textile
diff options
context:
space:
mode:
authorPrem Sichanugrist <s@sikachu.com>2012-09-01 16:25:57 -0400
committerPrem Sichanugrist <s@sikac.hu>2012-09-17 15:54:22 -0400
commit5137d03cc5b2a5f0820bdcf11b0fffe5bf461470 (patch)
tree0d390e9e8aebcb60b509a8153548f5a526a18a9b /guides/source/performance_testing.textile
parent232b9ade1a5c885537052a27c5d3cfeb65e5c265 (diff)
downloadrails-5137d03cc5b2a5f0820bdcf11b0fffe5bf461470.tar.gz
rails-5137d03cc5b2a5f0820bdcf11b0fffe5bf461470.tar.bz2
rails-5137d03cc5b2a5f0820bdcf11b0fffe5bf461470.zip
Rename the rest of the guides to Markdown
Diffstat (limited to 'guides/source/performance_testing.textile')
-rw-r--r--guides/source/performance_testing.textile672
1 files changed, 0 insertions, 672 deletions
diff --git a/guides/source/performance_testing.textile b/guides/source/performance_testing.textile
deleted file mode 100644
index 6c5676c346..0000000000
--- a/guides/source/performance_testing.textile
+++ /dev/null
@@ -1,672 +0,0 @@
-h2. Performance Testing Rails Applications
-
-This guide covers the various ways of performance testing a Ruby on Rails
-application. By referring to this guide, you will be able to:
-
-* Understand the various types of benchmarking and profiling metrics.
-* Generate performance and benchmarking tests.
-* Install and use a GC-patched Ruby binary to measure memory usage and object
- allocation.
-* Understand the benchmarking information provided by Rails inside the log files.
-* Learn about various tools facilitating benchmarking and profiling.
-
-Performance testing is an integral part of the development cycle. It is very
-important that you don't make your end users wait for too long before the page
-is completely loaded. Ensuring a pleasant browsing experience for end users and
-cutting the cost of unnecessary hardware is important for any non-trivial web
-application.
-
-endprologue.
-
-h3. Performance Test Cases
-
-Rails performance tests are a special type of integration tests, designed for
-benchmarking and profiling the test code. With performance tests, you can
-determine where your application's memory or speed problems are coming from,
-and get a more in-depth picture of those problems.
-
-In a freshly generated Rails application, +test/performance/browsing_test.rb+
-contains an example of a performance test:
-
-<ruby>
-require 'test_helper'
-require 'rails/performance_test_help'
-
-class BrowsingTest < ActionDispatch::PerformanceTest
- # Refer to the documentation for all available options
- # self.profile_options = { runs: 5, metrics: [:wall_time, :memory],
- # output: 'tmp/performance', formats: [:flat] }
-
- test "homepage" do
- get '/'
- end
-end
-</ruby>
-
-This example is a simple performance test case for profiling a GET request to
-the application's homepage.
-
-h4. Generating Performance Tests
-
-Rails provides a generator called +performance_test+ for creating new
-performance tests:
-
-<shell>
-$ rails generate performance_test homepage
-</shell>
-
-This generates +homepage_test.rb+ in the +test/performance+ directory:
-
-<ruby>
-require 'test_helper'
-require 'rails/performance_test_help'
-
-class HomepageTest < ActionDispatch::PerformanceTest
- # Refer to the documentation for all available options
- # self.profile_options = { :runs => 5, :metrics => [:wall_time, :memory],
- # :output => 'tmp/performance', :formats => [:flat] }
-
- test "homepage" do
- get '/'
- end
-end
-</ruby>
-
-h4. Examples
-
-Let's assume your application has the following controller and model:
-
-<ruby>
-# routes.rb
-root to: 'home#dashboard'
-resources :posts
-
-# home_controller.rb
-class HomeController < ApplicationController
- def dashboard
- @users = User.last_ten.includes(:avatars)
- @posts = Post.all_today
- end
-end
-
-# posts_controller.rb
-class PostsController < ApplicationController
- def create
- @post = Post.create(params[:post])
- redirect_to(@post)
- end
-end
-
-# post.rb
-class Post < ActiveRecord::Base
- before_save :recalculate_costly_stats
-
- def slow_method
- # I fire gallzilion queries sleeping all around
- end
-
- private
-
- def recalculate_costly_stats
- # CPU heavy calculations
- end
-end
-</ruby>
-
-h5. Controller Example
-
-Because performance tests are a special kind of integration test, you can use
-the +get+ and +post+ methods in them.
-
-Here's the performance test for +HomeController#dashboard+ and
-+PostsController#create+:
-
-<ruby>
-require 'test_helper'
-require 'rails/performance_test_help'
-
-class PostPerformanceTest < ActionDispatch::PerformanceTest
- def setup
- # Application requires logged-in user
- login_as(:lifo)
- end
-
- test "homepage" do
- get '/dashboard'
- end
-
- test "creating new post" do
- post '/posts', post: { body: 'lifo is fooling you' }
- end
-end
-</ruby>
-
-You can find more details about the +get+ and +post+ methods in the
-"Testing Rails Applications":testing.html guide.
-
-h5. Model Example
-
-Even though the performance tests are integration tests and hence closer to
-the request/response cycle by nature, you can still performance test pure model
-code.
-
-Performance test for +Post+ model:
-
-<ruby>
-require 'test_helper'
-require 'rails/performance_test_help'
-
-class PostModelTest < ActionDispatch::PerformanceTest
- test "creation" do
- Post.create body: 'still fooling you', cost: '100'
- end
-
- test "slow method" do
- # Using posts(:awesome) fixture
- posts(:awesome).slow_method
- end
-end
-</ruby>
-
-h4. Modes
-
-Performance tests can be run in two modes: Benchmarking and Profiling.
-
-h5. Benchmarking
-
-Benchmarking makes it easy to quickly gather a few metrics about each test run.
-By default, each test case is run *4 times* in benchmarking mode.
-
-To run performance tests in benchmarking mode:
-
-<shell>
-$ rake test:benchmark
-</shell>
-
-h5. Profiling
-
-Profiling allows you to make an in-depth analysis of each of your tests by using
-an external profiler. Depending on your Ruby interpreter, this profiler can be
-native (Rubinius, JRuby) or not (MRI, which uses RubyProf). By default, each
-test case is run *once* in profiling mode.
-
-To run performance tests in profiling mode:
-
-<shell>
-$ rake test:profile
-</shell>
-
-h4. Metrics
-
-Benchmarking and profiling run performance tests and give you multiple metrics.
-The availability of each metric is determined by the interpreter being used—none
-of them support all metrics—and by the mode in use. A brief description of each
-metric and their availability across interpreters/modes is given below.
-
-h5. Wall Time
-
-Wall time measures the real world time elapsed during the test run. It is
-affected by any other processes concurrently running on the system.
-
-h5. Process Time
-
-Process time measures the time taken by the process. It is unaffected by any
-other processes running concurrently on the same system. Hence, process time
-is likely to be constant for any given performance test, irrespective of the
-machine load.
-
-h5. CPU Time
-
-Similar to process time, but leverages the more accurate CPU clock counter
-available on the Pentium and PowerPC platforms.
-
-h5. User Time
-
-User time measures the amount of time the CPU spent in user-mode, i.e. within
-the process. This is not affected by other processes and by the time it possibly
-spends blocked.
-
-h5. Memory
-
-Memory measures the amount of memory used for the performance test case.
-
-h5. Objects
-
-Objects measures the number of objects allocated for the performance test case.
-
-h5. GC Runs
-
-GC Runs measures the number of times GC was invoked for the performance test case.
-
-h5. GC Time
-
-GC Time measures the amount of time spent in GC for the performance test case.
-
-h5. Metric Availability
-
-h6(#benchmarking_1). Benchmarking
-
-|_.Interpreter|_.Wall Time|_.Process Time|_.CPU Time|_.User Time|_.Memory|_.Objects|_.GC Runs|_.GC Time|
-|_.MRI | yes | yes | yes | no | yes | yes | yes | yes |
-|_.REE | yes | yes | yes | no | yes | yes | yes | yes |
-|_.Rubinius | yes | no | no | no | yes | yes | yes | yes |
-|_.JRuby | yes | no | no | yes | yes | yes | yes | yes |
-
-h6(#profiling_1). Profiling
-
-|_.Interpreter|_.Wall Time|_.Process Time|_.CPU Time|_.User Time|_.Memory|_.Objects|_.GC Runs|_.GC Time|
-|_.MRI | yes | yes | no | no | yes | yes | yes | yes |
-|_.REE | yes | yes | no | no | yes | yes | yes | yes |
-|_.Rubinius | yes | no | no | no | no | no | no | no |
-|_.JRuby | yes | no | no | no | no | no | no | no |
-
-NOTE: To profile under JRuby you'll need to run +export JRUBY_OPTS="-Xlaunch.inproc=false --profile.api"+
-*before* the performance tests.
-
-h4. Understanding the Output
-
-Performance tests generate different outputs inside +tmp/performance+ directory
-depending on their mode and metric.
-
-h5(#output-benchmarking). Benchmarking
-
-In benchmarking mode, performance tests generate two types of outputs.
-
-h6(#output-command-line). Command Line
-
-This is the primary form of output in benchmarking mode. Example:
-
-<shell>
-BrowsingTest#test_homepage (31 ms warmup)
- wall_time: 6 ms
- memory: 437.27 KB
- objects: 5,514
- gc_runs: 0
- gc_time: 19 ms
-</shell>
-
-h6. CSV Files
-
-Performance test results are also appended to +.csv+ files inside +tmp/performance+.
-For example, running the default +BrowsingTest#test_homepage+ will generate
-following five files:
-
-* BrowsingTest#test_homepage_gc_runs.csv
-* BrowsingTest#test_homepage_gc_time.csv
-* BrowsingTest#test_homepage_memory.csv
-* BrowsingTest#test_homepage_objects.csv
-* BrowsingTest#test_homepage_wall_time.csv
-
-As the results are appended to these files each time the performance tests are
-run in benchmarking mode, you can collect data over a period of time. This can
-be very helpful in analyzing the effects of code changes.
-
-Sample output of +BrowsingTest#test_homepage_wall_time.csv+:
-
-<shell>
-measurement,created_at,app,rails,ruby,platform
-0.00738224999999992,2009-01-08T03:40:29Z,,3.0.0,ruby-1.8.7.249,x86_64-linux
-0.00755874999999984,2009-01-08T03:46:18Z,,3.0.0,ruby-1.8.7.249,x86_64-linux
-0.00762099999999993,2009-01-08T03:49:25Z,,3.0.0,ruby-1.8.7.249,x86_64-linux
-0.00603075000000008,2009-01-08T04:03:29Z,,3.0.0,ruby-1.8.7.249,x86_64-linux
-0.00619899999999995,2009-01-08T04:03:53Z,,3.0.0,ruby-1.8.7.249,x86_64-linux
-0.00755449999999991,2009-01-08T04:04:55Z,,3.0.0,ruby-1.8.7.249,x86_64-linux
-0.00595999999999997,2009-01-08T04:05:06Z,,3.0.0,ruby-1.8.7.249,x86_64-linux
-0.00740450000000004,2009-01-09T03:54:47Z,,3.0.0,ruby-1.8.7.249,x86_64-linux
-0.00603150000000008,2009-01-09T03:54:57Z,,3.0.0,ruby-1.8.7.249,x86_64-linux
-0.00771250000000012,2009-01-09T15:46:03Z,,3.0.0,ruby-1.8.7.249,x86_64-linux
-</shell>
-
-h5(#output-profiling). Profiling
-
-In profiling mode, performance tests can generate multiple types of outputs.
-The command line output is always presented but support for the others is
-dependent on the interpreter in use. A brief description of each type and
-their availability across interpreters is given below.
-
-h6. Command Line
-
-This is a very basic form of output in profiling mode:
-
-<shell>
-BrowsingTest#test_homepage (58 ms warmup)
- process_time: 63 ms
- memory: 832.13 KB
- objects: 7,882
-</shell>
-
-h6. Flat
-
-Flat output shows the metric—time, memory, etc—measure in each method.
-"Check Ruby-Prof documentation for a better explanation":http://ruby-prof.rubyforge.org/files/examples/flat_txt.html.
-
-h6. Graph
-
-Graph output shows the metric measure in each method, which methods call it and
-which methods it calls. "Check Ruby-Prof documentation for a better explanation":http://ruby-prof.rubyforge.org/files/examples/graph_txt.html.
-
-h6. Tree
-
-Tree output is profiling information in calltree format for use by "kcachegrind":http://kcachegrind.sourceforge.net/html/Home.html
-and similar tools.
-
-h6. Output Availability
-
-|_. |_.Flat|_.Graph|_.Tree|
-|_.MRI | yes | yes | yes |
-|_.REE | yes | yes | yes |
-|_.Rubinius | yes | yes | no |
-|_.JRuby | yes | yes | no |
-
-h4. Tuning Test Runs
-
-Test runs can be tuned by setting the +profile_options+ class variable on your
-test class.
-
-<ruby>
-require 'test_helper'
-require 'rails/performance_test_help'
-
-class BrowsingTest < ActionDispatch::PerformanceTest
- self.profile_options = { runs: 5, metrics: [:wall_time, :memory] }
-
- test "homepage"
- get '/'
- end
-end
-</ruby>
-
-In this example, the test would run 5 times and measure wall time and memory.
-There are a few configurable options:
-
-|_.Option |_.Description|_.Default|_.Mode|
-|+:runs+ |Number of runs.|Benchmarking: 4, Profiling: 1|Both|
-|+:output+ |Directory to use when writing the results.|+tmp/performance+|Both|
-|+:metrics+ |Metrics to use.|See below.|Both|
-|+:formats+ |Formats to output to.|See below.|Profiling|
-
-Metrics and formats have different defaults depending on the interpreter in use.
-
-|_.Interpreter|_.Mode|_.Default metrics|_.Default formats|
-|/2.MRI/REE |Benchmarking|+[:wall_time, :memory, :objects, :gc_runs, :gc_time]+|N/A|
-|Profiling |+[:process_time, :memory, :objects]+|+[:flat, :graph_html, :call_tree, :call_stack]+|
-|/2.Rubinius|Benchmarking|+[:wall_time, :memory, :objects, :gc_runs, :gc_time]+|N/A|
-|Profiling |+[:wall_time]+|+[:flat, :graph]+|
-|/2.JRuby |Benchmarking|+[:wall_time, :user_time, :memory, :gc_runs, :gc_time]+|N/A|
-|Profiling |+[:wall_time]+|+[:flat, :graph]+|
-
-As you've probably noticed by now, metrics and formats are specified using a
-symbol array with each name "underscored.":http://api.rubyonrails.org/classes/String.html#method-i-underscore
-
-h4. Performance Test Environment
-
-Performance tests are run in the +test+ environment. But running performance
-tests will set the following configuration parameters:
-
-<shell>
-ActionController::Base.perform_caching = true
-ActiveSupport::Dependencies.mechanism = :require
-Rails.logger.level = ActiveSupport::BufferedLogger::INFO
-</shell>
-
-As +ActionController::Base.perform_caching+ is set to +true+, performance tests
-will behave much as they do in the +production+ environment.
-
-h4. Installing GC-Patched MRI
-
-To get the best from Rails' performance tests under MRI, you'll need to build
-a special Ruby binary with some super powers.
-
-The recommended patches for each MRI version are:
-
-|_.Version|_.Patch|
-|1.8.6|ruby186gc|
-|1.8.7|ruby187gc|
-|1.9.2 and above|gcdata|
-
-All of these can be found on "RVM's _patches_ directory":https://github.com/wayneeseguin/rvm/tree/master/patches/ruby
-under each specific interpreter version.
-
-Concerning the installation itself, you can either do this easily by using
-"RVM":http://rvm.beginrescueend.com or you can build everything from source,
-which is a little bit harder.
-
-h5. Install Using RVM
-
-The process of installing a patched Ruby interpreter is very easy if you let RVM
-do the hard work. All of the following RVM commands will provide you with a
-patched Ruby interpreter:
-
-<shell>
-$ rvm install 1.9.2-p180 --patch gcdata
-$ rvm install 1.8.7 --patch ruby187gc
-$ rvm install 1.9.2-p180 --patch ~/Downloads/downloaded_gcdata_patch.patch
-</shell>
-
-You can even keep your regular interpreter by assigning a name to the patched
-one:
-
-<shell>
-$ rvm install 1.9.2-p180 --patch gcdata --name gcdata
-$ rvm use 1.9.2-p180 # your regular ruby
-$ rvm use 1.9.2-p180-gcdata # your patched ruby
-</shell>
-
-And it's done! You have installed a patched Ruby interpreter.
-
-h5. Install From Source
-
-This process is a bit more complicated, but straightforward nonetheless. If
-you've never compiled a Ruby binary before, follow these steps to build a
-Ruby binary inside your home directory.
-
-h6. Download and Extract
-
-<shell>
-$ mkdir rubygc
-$ wget <the version you want from ftp://ftp.ruby-lang.org/pub/ruby>
-$ tar -xzvf <ruby-version.tar.gz>
-$ cd <ruby-version>
-</shell>
-
-h6. Apply the Patch
-
-<shell>
-$ curl http://github.com/wayneeseguin/rvm/raw/master/patches/ruby/1.9.2/p180/gcdata.patch | patch -p0 # if you're on 1.9.2!
-$ curl http://github.com/wayneeseguin/rvm/raw/master/patches/ruby/1.8.7/ruby187gc.patch | patch -p0 # if you're on 1.8.7!
-</shell>
-
-h6. Configure and Install
-
-The following will install Ruby in your home directory's +/rubygc+ directory.
-Make sure to replace +&lt;homedir&gt;+ with a full patch to your actual home
-directory.
-
-<shell>
-$ ./configure --prefix=/<homedir>/rubygc
-$ make && make install
-</shell>
-
-h6. Prepare Aliases
-
-For convenience, add the following lines in your +~/.profile+:
-
-<shell>
-alias gcruby='~/rubygc/bin/ruby'
-alias gcrake='~/rubygc/bin/rake'
-alias gcgem='~/rubygc/bin/gem'
-alias gcirb='~/rubygc/bin/irb'
-alias gcrails='~/rubygc/bin/rails'
-</shell>
-
-Don't forget to use your aliases from now on.
-
-h4. Using Ruby-Prof on MRI and REE
-
-Add Ruby-Prof to your applications' Gemfile if you want to benchmark/profile
-under MRI or REE:
-
-<ruby>
-gem 'ruby-prof', git: 'git://github.com/wycats/ruby-prof.git'
-</ruby>
-
-Now run +bundle install+ and you're ready to go.
-
-h3. Command Line Tools
-
-Writing performance test cases could be an overkill when you are looking for one
-time tests. Rails ships with two command line tools that enable quick and dirty
-performance testing:
-
-h4. +benchmarker+
-
-Usage:
-
-<shell>
-Usage: rails benchmarker 'Ruby.code' 'Ruby.more_code' ... [OPTS]
- -r, --runs N Number of runs.
- Default: 4
- -o, --output PATH Directory to use when writing the results.
- Default: tmp/performance
- -m, --metrics a,b,c Metrics to use.
- Default: wall_time,memory,objects,gc_runs,gc_time
-</shell>
-
-Example:
-
-<shell>
-$ rails benchmarker 'Item.all' 'CouchItem.all' --runs 3 --metrics wall_time,memory
-</shell>
-
-h4. +profiler+
-
-Usage:
-
-<shell>
-Usage: rails profiler 'Ruby.code' 'Ruby.more_code' ... [OPTS]
- -r, --runs N Number of runs.
- Default: 1
- -o, --output PATH Directory to use when writing the results.
- Default: tmp/performance
- --metrics a,b,c Metrics to use.
- Default: process_time,memory,objects
- -m, --formats x,y,z Formats to output to.
- Default: flat,graph_html,call_tree
-</shell>
-
-Example:
-
-<shell>
-$ rails profiler 'Item.all' 'CouchItem.all' --runs 2 --metrics process_time --formats flat
-</shell>
-
-NOTE: Metrics and formats vary from interpreter to interpreter. Pass +--help+ to
-each tool to see the defaults for your interpreter.
-
-h3. Helper Methods
-
-Rails provides various helper methods inside Active Record, Action Controller
-and Action View to measure the time taken by a given piece of code. The method
-is called +benchmark()+ in all the three components.
-
-h4. Model
-
-<ruby>
-Project.benchmark("Creating project") do
- project = Project.create("name" => "stuff")
- project.create_manager("name" => "David")
- project.milestones << Milestone.all
-end
-</ruby>
-
-This benchmarks the code enclosed in the +Project.benchmark("Creating project") do...end+
-block and prints the result to the log file:
-
-<ruby>
-Creating project (185.3ms)
-</ruby>
-
-Please refer to the "API docs":http://api.rubyonrails.org/classes/ActiveSupport/Benchmarkable.html#method-i-benchmark
-for additional options to +benchmark()+.
-
-h4. Controller
-
-Similarly, you could use this helper method inside "controllers.":http://api.rubyonrails.org/classes/ActiveSupport/Benchmarkable.html
-
-<ruby>
-def process_projects
- benchmark("Processing projects") do
- Project.process(params[:project_ids])
- Project.update_cached_projects
- end
-end
-</ruby>
-
-NOTE: +benchmark+ is a class method inside controllers.
-
-h4. View
-
-And in "views":http://api.rubyonrails.org/classes/ActiveSupport/Benchmarkable.html:
-
-<erb>
-<% benchmark("Showing projects partial") do %>
- <%= render @projects %>
-<% end %>
-</erb>
-
-h3. Request Logging
-
-Rails log files contain very useful information about the time taken to serve
-each request. Here's a typical log file entry:
-
-<shell>
-Processing ItemsController#index (for 127.0.0.1 at 2009-01-08 03:06:39) [GET]
-Rendering template within layouts/items
-Rendering items/index
-Completed in 5ms (View: 2, DB: 0) | 200 OK [http://0.0.0.0/items]
-</shell>
-
-For this section, we're only interested in the last line:
-
-<shell>
-Completed in 5ms (View: 2, DB: 0) | 200 OK [http://0.0.0.0/items]
-</shell>
-
-This data is fairly straightforward to understand. Rails uses millisecond(ms) as
-the metric to measure the time taken. The complete request spent 5 ms inside
-Rails, out of which 2 ms were spent rendering views and none was spent
-communication with the database. It's safe to assume that the remaining 3 ms
-were spent inside the controller.
-
-Michael Koziarski has an "interesting blog post":http://www.therailsway.com/2009/1/6/requests-per-second
-explaining the importance of using milliseconds as the metric.
-
-h3. Useful Links
-
-h4. Rails Plugins and Gems
-
-* "Rails Analyzer":http://rails-analyzer.rubyforge.org
-* "Palmist":http://www.flyingmachinestudios.com/programming/announcing-palmist
-* "Rails Footnotes":https://github.com/josevalim/rails-footnotes/tree/master
-* "Query Reviewer":https://github.com/dsboulder/query_reviewer/tree/master
-* "MiniProfiler":http://www.miniprofiler.com
-
-h4. Generic Tools
-
-* "httperf":http://www.hpl.hp.com/research/linux/httperf/
-* "ab":http://httpd.apache.org/docs/2.2/programs/ab.html
-* "JMeter":http://jakarta.apache.org/jmeter/
-* "kcachegrind":http://kcachegrind.sourceforge.net/html/Home.html
-
-h4. Tutorials and Documentation
-
-* "ruby-prof API Documentation":http://ruby-prof.rubyforge.org
-* "Request Profiling Railscast":http://railscasts.com/episodes/98-request-profiling - Outdated, but useful for understanding call graphs.
-
-h3. Commercial Products
-
-Rails has been lucky to have a few companies dedicated to Rails-specific
-performance tools. A couple of those are:
-
-* "New Relic":http://www.newrelic.com
-* "Scout":http://scoutapp.com