Date: Tue, 21 Oct 2008 16:38:36 +0100 Subject: Fix that dir.html.html guide name --- .../doc/guides/html/actioncontroller_basics.html | 1125 ++++++++++++++++ .../guides/html/actioncontroller_basics.html.html | 1125 ---------------- .../guides/html/benchmarking_and_profiling.html | 1015 ++++++++++++++ .../html/benchmarking_and_profiling.html.html | 1015 -------------- railties/doc/guides/html/creating_plugins.html | 1402 ++++++++++++++++++++ .../doc/guides/html/creating_plugins.html.html | 1402 -------------------- railties/doc/guides/html/migrations.html | 921 +++++++++++++ railties/doc/guides/html/migrations.html.html | 921 ------------- 8 files changed, 4463 insertions(+), 4463 deletions(-) create mode 100644 railties/doc/guides/html/actioncontroller_basics.html delete mode 100644 railties/doc/guides/html/actioncontroller_basics.html.html create mode 100644 railties/doc/guides/html/benchmarking_and_profiling.html delete mode 100644 railties/doc/guides/html/benchmarking_and_profiling.html.html create mode 100644 railties/doc/guides/html/creating_plugins.html delete mode 100644 railties/doc/guides/html/creating_plugins.html.html create mode 100644 railties/doc/guides/html/migrations.html delete mode 100644 railties/doc/guides/html/migrations.html.html (limited to 'railties/doc') diff --git a/railties/doc/guides/html/actioncontroller_basics.html b/railties/doc/guides/html/actioncontroller_basics.html new file mode 100644 index 0000000000..4df3a7e9b2 --- /dev/null +++ b/railties/doc/guides/html/actioncontroller_basics.html @@ -0,0 +1,1125 @@ + + + + + Action Controller basics + + + + + + + + + +

+ + + +

Action Controller basics

In this guide you will learn how controllers work and how they fit into the request cycle in your application. You will learn how to make use of the many tools provided by Action Controller to work with the session, cookies and filters and how to use the built-in HTTP authentication and data streaming facilities. In the end, we will take a look at some tools that will be useful once your controllers are ready and working, like how to filter sensitive parameters from the log and how to rescue and deal with exceptions that may be raised during the request.

1. What does a controller do?

Action Controller is the C in MVC. After routing has determined which controller to use for a request, your controller is responsible for making sense of the request and producing the appropriate output. Luckily, Action Controller does most of the groundwork for you and uses smart conventions to make this as straight-forward as possible.

For most conventional RESTful applications, the controller will receive the request (this is invisible to you as the developer), fetch or save data from a model and use a view to create HTML output. If your controller needs to do things a little differently, that's not a problem, this is just the most common way for a controller to work.

A controller can thus be thought of as a middle man between models and views. It makes the model data available to the view so it can display it to the user, and it saves or updates data from the user to the model.

2. Methods and actions

A controller is a Ruby class which inherits from ActionController::Base and has methods just like any other class. Usually these methods correspond to actions in MVC, but they can just as well be helpful methods which can be called by actions. When your application receives a request, the routing will determine which controller and action to run. Then an instance of that controller will be created and the method corresponding to the action (the method with the same name as the action) gets run.

class ClientsController < ActionController::Base
+
+  # Actions are public methods
+  def new
+  end
+
+  # These methods are responsible for producing output
+  def edit
+  end
+
+# Helper methods are private and can not be used as actions
+private
+
+  def foo
+  end
+
+end
+

Private methods in a controller are also used as filters, which will be covered later in this guide.

As an example, if the user goes to /clients/new in your application to add a new client, a ClientsController instance will be created and the new method will be run. Note that the empty method from the example above could work just fine because Rails will by default render the new.html.erb view unless the action says otherwise. The new method could make available to the view a @client instance variable by creating a new Client:

def new
+  @client = Client.new
+end
+

The Layouts & rendering guide explains this in more detail.

3. Parameters

You will probably want to access data sent in by the user or other parameters in your controller actions. There are two kinds of parameters possible in a web application. The first are parameters that are sent as part of the URL, query string parameters. The query string is everything after "?" in the URL. The second type of parameter is usually referred to as POST data. This information usually comes from a HTML form which has been filled in by the user. It's called POST data because it can only be sent as part of an HTTP POST request. Rails does not make any distinction between query string parameters and POST parameters, and both are available in the params hash in your controller:

class ClientsController < ActionController::Base
+
+  # This action uses query string parameters because it gets run by a HTTP GET request,
+  # but this does not make any difference to the way in which the parameters are accessed.
+  # The URL for this action would look like this in order to list activated clients: /clients?status=activated
+  def index
+    if params[:status] = "activated"
+      @clients = Client.activated
+    else
+      @clients = Client.unativated
+    end
+  end
+
+  # This action uses POST parameters. They are most likely coming from an HTML
+  # form which the user has submitted. The URL for this RESTful request will
+  # be "/clients", and the data will be sent as part of the request body.
+  def create
+    @client = Client.new(params[:client])
+    if @client.save
+      redirect_to @client
+    else
+      # This line overrides the default rendering behavior, which would have been
+      # to render the "create" view.
+      render :action => "new"
+    end
+  end
+
+end
+

3.1. Hash and array parameters

The params hash is not limited to one-dimensional keys and values. It can contain arrays and (nested) hashes. To send an array of values, append "[]" to the key name:

GET /clients?ids[]=1&ids[2]&ids[]=3

The value of params[:ids] will now be ["1", "2", "3"]. Note that parameter values are always strings; Rails makes no attempt to guess or cast the type.

To send a hash you include the key name inside the brackets:

<form action="/clients" method="post">
+  <input type="text" name="client[name]" value="Acme" />
+  <input type="text" name="client[phone]" value="12345" />
+  <input type="text" name="client[address][postcode]" value="12345" />
+  <input type="text" name="client[address][city]" value="Carrot City" />
+</form>

The value of params[:client] when this form is submitted will be {:name ⇒ "Acme", :phone ⇒ "12345", :address ⇒ {:postcode ⇒ "12345", :city ⇒ "Carrot City"}}. Note the nested hash in params[:client][:address].

3.2. Routing parameters

The params hash will always contain the :controller and :action keys, but you should use the methods controller_name and action_name instead to access these values. Any other parameters defined by the routing, such as :id will also be available.

4. Session

Your application has a session for each user in which you can store small amounts of data that will be persisted between requests. The session is only available in the controller and can use one of a number of different storage mechanisms:

+
+CookieStore - Stores everything on the client. +
+
+
+DRBStore - Stores the data on a DRb client. +
+
+
+MemCacheStore - Stores the data in MemCache. +
+
+
+ActiveRecordStore - Stores the data in a database using Active Record. +
+

All session stores store the session id in a cookie - there is no other way of passing it to the server. Most stores also use this key to locate the session data on the server.

The default and recommended store, the Cookie Store, does not store session data on the server, but in the cookie itself. The data is cryptographically signed to make it tamper-proof, but it is not encrypted, so anyone with access to it can read its contents. It can only store about 4kB of data - much less than the others - but this is usually enough. Storing large amounts of data is discouraged no matter which session store your application uses. Expecially discouraged is storing complex objects (anything other than basic Ruby objects, the primary example being model instances) in the session, as the server might not be able to reassemble them between requests, which will result in an error. The Cookie Store has the added advantage that it does not require any setting up beforehand - Rails will generate a "secret key" which will be used to sign the cookie when you create the application.

If you need a different session storage mechanism, you can change it in the config/environment.rb file:

# Set to one of [:active_record_store, :drb_store, :mem_cache_store, :cookie_store]
+config.action_controller.session_store = :active_record_store
+

4.1. Disabling the session

Sometimes you don't need a session, and you can turn it off to avoid the unnecessary overhead. To do this, use the session class method in your controller:

class ApplicationController < ActionController::Base
+  session :off
+end
+

You can also turn the session on or off for a single controller:

# The session is turned off by default in ApplicationController, but we
+# want to turn it on for log in/out.
+class LoginsController < ActionController::Base
+  session :on
+end
+

Or even a single action:

class ProductsController < ActionController::Base
+  session :on, :only => [:create, :update]
+end
+

4.2. Accessing the session

In your controller you can access the session through the session instance method.

+ + + +

+ + There are two session methods, the class and the instance method. The class method which is described above is used to turn the session on and off while the instance method described below is used to access session values. The class method is used outside of method definitions while the instance methods is used inside methods, in actions or filters.

Session values are stored using key/value pairs like a hash:

class ApplicationController < ActionController::Base
+
+private
+
+  # Finds the User with the ID stored in the session with the key :current_user_id
+  # This is a common way to do user login in a Rails application; logging in sets the
+  # session value and logging out removes it.
+  def current_user
+    @_current_user ||= session[:current_user_id] && User.find(session[:current_user_id])
+  end
+
+end
+

To store something in the session, just assign it to the key like a hash:

class LoginsController < ApplicationController
+
+  # "Create" a login, aka "log the user in"
+  def create
+    if user = User.authenticate(params[:username, params[:password])
+      # Save the user ID in the session so it can be used in subsequent requests
+      session[:current_user_id] = user.id
+      redirect_to root_url
+    end
+  end
+
+end
+

To remove something from the session, assign that key to be nil:

class LoginsController < ApplicationController
+
+  # "Delete" a login, aka "log the user out"
+  def destroy
+    # Remove the user id from the session
+    session[:current_user_id] = nil
+    redirect_to root_url
+  end
+
+end
+

To reset the entire session, use reset_session.

4.3. The flash

The flash is a special part of the session which is cleared with each request. This means that values stored there will only be available in the next request, which is useful for storing error messages etc. It is accessed in much the same way as the session, like a hash. Let's use the act of logging out as an example. The controller can set a message which will be displayed to the user on the next request:

class LoginsController < ApplicationController
+
+  def destroy
+    session[:current_user_id] = nil
+    flash[:notice] = "You have successfully logged out"
+    redirect_to root_url
+  end
+
+end
+

The destroy action redirects to the application's root_url, where the message will be displayed. Note that it's entirely up to the next action to decide what, if anything, it will do with what the previous action put in the flash. It's conventional to a display eventual errors or notices from the flash in the application's layout:

<html>
+  <!-- <head/> -->
+  <body>
+    <% if flash[:notice] -%>
+      <p class="notice"><%= flash[:notice] %></p>
+    <% end -%>
+    <% if flash[:error] -%>
+      <p class="error"><%= flash[:error] %></p>
+    <% end -%>
+    <!-- more content -->
+  </body>
+</html>

This way, if an action sets an error or a notice message, the layout will display it automatically.

If you want a flash value to be carried over to another request, use the keep method:

class MainController < ApplicationController
+
+  # Let's say this action corresponds to root_url, but you want all requests here to be redirected to
+  # UsersController#index. If an action sets the flash and redirects here, the values would normally be
+  # lost when another redirect happens, but you can use keep to make it persist for another request.
+  def index
+    flash.keep # Will persist all flash values. You can also use a key to keep only that value: flash.keep(:notice)
+    redirect_to users_url
+  end
+
+end
+

4.3.1. flash.now

By default, adding values to the flash will make them available to the next request, but sometimes you may want to access those values in the same request. For example, if the create action fails to save a resource and you render the new template directly, that's not going to result in a new request, but you may still want to display a message using the flash. To do this, you can use flash.now in the same way you use the normal flash:

class ClientsController < ApplicationController
+
+  def create
+    @client = Client.new(params[:client])
+    if @client.save
+      # ...
+    else
+      flash.now[:error] = "Could not save client"
+      render :action => "new"
+    end
+  end
+
+end
+

5. Cookies

Your application can store small amounts of data on the client - called cookies - that will be persisted across requests and even sessions. Rails provides easy access to cookies via the cookies method, which - much like the session - works like a hash:

class CommentsController < ApplicationController
+
+  def new
+    #Auto-fill the commenter's name if it has been stored in a cookie
+    @comment = Comment.new(:name => cookies[:commenter_name])
+  end
+
+  def create
+    @comment = Comment.new(params[:comment])
+    if @comment.save
+      flash[:notice] = "Thanks for your comment!"
+      if params[:remember_name]
+        # Remember the commenter's name
+        cookies[:commenter_name] = @comment.name
+      else
+        # Don't remember, and delete the name if it has been remembered before
+        cookies.delete(:commenter_name)
+      end
+      redirect_to @comment.article
+    else
+      render :action => "new"
+    end
+  end
+
+end
+

Note that while for session values, you set the key to nil, to delete a cookie value, you use cookies.delete(:key).

6. Filters

Filters are methods that are run before, after or "around" a controller action. For example, one filter might check to see if the logged in user has the right credentials to access that particular controller or action. Filters are inherited, so if you set a filter on ApplicationController, it will be run on every controller in your application. A common, simple filter is one which requires that a user is logged in for an action to be run. Let's define the filter method first:

class ApplicationController < ActionController::Base
+
+private
+
+  def require_login
+    unless logged_in?
+      flash[:error] = "You must be logged in to access this section"
+      redirect_to new_login_url # Prevents the current action from running
+    end
+  end
+
+  # The logged_in? method simply returns true if the user is logged in and
+  # false otherwise. It does this by "booleanizing" the current_user method
+  # we created previously using a double ! operator. Note that this is not
+  # common in Ruby and is discouraged unless you really mean to convert something
+  # into true or false.
+  def logged_in?
+    !!current_user
+  end
+
+end
+

The method simply stores an error message in the flash and redirects to the login form if the user is not logged in. If a before filter (a filter which is run before the action) renders or redirects, the action will not run. If there are additional filters scheduled to run after the rendering/redirecting filter, they are also cancelled. To use this filter in a controller, use the before_filter method:

class ApplicationController < ActionController::Base
+
+  before_filter :require_login
+
+end
+

In this example, the filter is added to ApplicationController and thus all controllers in the application. This will make everything in the application require the user to be logged in in order to use it. For obvious reasons (the user wouldn't be able to log in in the first place!), not all controllers or actions should require this, so to prevent this filter from running you can use skip_before_filter :

class LoginsController < Application
+
+  skip_before_filter :require_login, :only => [:new, :create]
+
+end
+

Now, the LoginsController's "new" and "create" actions will work as before without requiring the user to be logged in. The :only option is used to only skip this filter for these actions, and there is also an :except option which works the other way. These options can be used when adding filters too, so you can add a filter which only runs for selected actions in the first place.

6.1. After filters and around filters

In addition to the before filters, you can run filters after an action has run or both before and after. The after filter is similar to the before filter, but because the action has already been run it has access to the response data that's about to be sent to the client. Obviously, after filters can not stop the action from running. Around filters are responsible for running the action, but they can choose not to, which is the around filter's way of stopping it.

TODO: Find a real example for an around filter

# Example taken from the Rails API filter documentation:
+# http://api.rubyonrails.org/classes/ActionController/Filters/ClassMethods.html
+class ApplicationController < Application
+
+  around_filter :catch_exceptions
+
+private
+
+  def catch_exceptions
+    yield
+  rescue => exception
+    logger.debug "Caught exception! #{exception}"
+    raise
+  end
+
+end
+

6.2. Other ways to use filters

While the most common way to use filters is by creating private methods and using *_filter to add them, there are two other ways to do the same thing.

The first is to use a block directly with the *_filter methods. The block receives the controller as an argument, and the require_login filter from above could be rewritte to use a block:

class ApplicationController < ActionController::Base
+
+  before_filter { |controller| redirect_to new_login_url unless controller.send(:logged_in?) }
+
+end
+

Note that the filter in this case uses send because the logged_in? method is private and the filter is not run in the scope of the controller. This is not the recommended way to implement this particular filter, but in more simple cases it might be useful.

The second way is to use a class (actually, any object that responds to the right methods will do) to handle the filtering. This is useful in cases that are more complex than can not be implemented in a readable and reusable way using the two other methods. As an example, we will rewrite the login filter again to use a class:

class ApplicationController < ActionController::Base
+
+  before_filter LoginFilter
+
+end
+
+class LoginFilter
+
+  def self.filter(controller)
+    unless logged_in?
+      controller.flash[:error] = "You must be logged in to access this section"
+      controller.redirect_to controller.new_login_url
+    end
+  end
+
+end
+

Again, this is not an ideal example for this filter, because it's not run in the scope of the controller but gets it passed as an argument. The filter class has a class method filter which gets run before or after the action, depending on if it's a before or after filter. Classes used as around filters can also use the same filter method, which will get run in the same way. The method must yield to execute the action. Alternatively, it can have both a before and an after method that are run before and after the action.

The Rails API documentation has more information on using filters.

7. Verification

Verifications make sure certain criterias are met in order for a controller or action to run. They can specify that a certain key (or several keys in the form of an array) is present in the params, session or flash hashes or that a certain HTTP method was used or that the request was made using XMLHTTPRequest (Ajax). The default action taken when these criterias are not met is to render a 400 Bad Request response, but you can customize this by specifying a redirect URL or rendering something else and you can also add flash messages and HTTP headers to the response. It is described in the API codumentation as "essentially a special kind of before_filter".

Let's see how we can use verification to make sure the user supplies a username and a password in order to log in:

class LoginsController < ApplicationController
+
+  verify :params => [:username, :password],
+         :render => {:action => "new"},
+         :add_flash => {:error => "Username and password required to log in"}
+
+  def create
+    @user = User.authenticate(params[:username], params[:password])
+    if @user
+      flash[:notice] = "You're logged in"
+      redirect_to root_url
+    else
+      render :action => "new"
+    end
+  end
+
+end
+

Now the create action won't run unless the "username" and "password" parameters are present, and if they're not, an error message will be added to the flash and the "new" action will be rendered. But there's something rather important missing from the verification above: It will be used for every action in LoginsController, which is not what we want. You can limit which actions it will be used for with the :only and :except options just like a filter:

class LoginsController < ApplicationController
+
+  verify :params => [:username, :password],
+         :render => {:action => "new"},
+         :add_flash => {:error => "Username and password required to log in"},
+         :only => :create #Only run this verification for the "create" action
+
+end
+

8. The request and response objects

In every controller there are two accessor methods pointing to the request and the response objects associated with the request cycle that is currently in execution. The request method contains an instance of AbstractRequest and the response method contains the response object representing what is going to be sent back to the client.

8.1. The request

The request object contains a lot of useful information about the request coming in from the client. To get a full list of the available methods, refer to the API documentation.

+
+host - The hostname used for this request. +
+
+
+domain - The hostname without the first part (usually "www"). +
+
+
+format - The content type requested by the client. +
+
+
+method - The HTTP method used for the request. +
+
+
+get?, post?, put?, delete?, head? - Returns true if the HTTP method is get/post/put/delete/head. +
+
+
+headers - Returns a hash containing the headers associated with the request. +
+
+
+port - The port number (integer) used for the request. +
+
+
+protocol - The protocol used for the request. +
+
+
+query_string - The query string part of the URL - everything after "?". +
+
+
+remote_ip - The IP address of the client. +
+
+
+url - The entire URL used for the request. +
+

8.1.1. path_parameters, query_parameters and request_parameters

TODO: Does this belong here?

Rails collects all of the parameters sent along with the request in the params hash, whether they are sent as part of the query string or the post body. The request object has three accessors that give you access to these parameters depending on where they came from. The query_parameters hash contains parameters that were sent as part of the query string while the request_parameters hash contains parameters sent as part of the post body. The path_parameters hash contains parameters that were recognised by the routing as being part of the path leading to this particular controller and action.

8.2. The response

The response objects is not usually used directly, but is built up during the execution of the action and rendering of the data that is being sent back to the user, but sometimes - like in an after filter - it can be useful to access the response directly. Some of these accessor methods also have setters, allowing you to change their values.

+
+body - This is the string of data being sent back to the client. This is most often HTML. +
+
+
+status - The HTTP status code for the response, like 200 for a successful request or 404 for file not found. +
+
+
+location - The URL the client is being redirected to, if any. +
+
+
+content_type - The content type of the response. +
+
+
+charset - The character set being used for the response. Default is "utf8". +
+

9. HTTP Basic Authentication

Rails comes with built-in HTTP Basic authentication. This is an authentication scheme that is supported by the majority of browsers and other HTTP clients. As an example, we will create an administration section which will only be available by entering a username and a password into the browser's HTTP Basic dialog window. Using the built-in authentication is quite easy and only requires you to use one method, authenticate_or_request_with_http_basic.

class AdminController < ApplicationController
+
+  USERNAME, PASSWORD = "humbaba", "f59a4805511bf4bb61978445a5380c6c"
+
+  before_filter :authenticate
+
+private
+
+  def authenticate
+    authenticate_or_request_with_http_basic do |username, password|
+      username == USERNAME && Digest::MD5.hexdigest(password) == PASSWORD
+    end
+  end
+
+end
+

With this in place, you can create namespaced controllers that inherit from AdminController. The before filter will thus be run for all actions in those controllers, protecting them with HTTP Basic authentication.

10. Streaming and file downloads

Sometimes you may want to send a file to the user instead of rendering an HTML page. All controllers in Rails have the send_data and the send_file methods, that will both stream data to the client. send_file is a convenience method which lets you provide the name of a file on the disk and it will stream the contents of that file for you.

To stream data to the client, use send_data:

require "prawn"
+class ClientsController < ApplicationController
+
+  # Generate a PDF document with information on the client and return it.
+  # The user will get the PDF as a file download.
+  def download_pdf
+    client = Client.find(params[:id])
+    send_data(generate_pdf, :filename => "#{client.name}.pdf", :type => "application/pdf")
+  end
+
+private
+
+  def generate_pdf(client)
+    Prawn::Document.new do
+      text client.name, :align => :center
+      text "Address: #{client.address}"
+      text "Email: #{client.email}"
+    end.render
+  end
+
+end
+

The download_pdf action in the example above will call a private method which actually generates the file (a PDF document) and returns it as a string. This string will then be streamed to the client as a file download and a filename will be suggested to the user. Sometimes when streaming files to the user, you may not want them to download the file. Take images, for example, which can be embedded into HTML pages. To tell the browser a file is not meant to be downloaded, you can set the :disposition option to "inline". The opposite and default value for this option is "attachment".

10.1. Sending files

If you want to send a file that already exists on disk, use the send_file method. This is usually not recommended, but can be useful if you want to perform some authentication before letting the user download the file.

class ClientsController < ApplicationController
+
+  # Stream a file that has already been generated and stored on disk
+  def download_pdf
+    client = Client.find(params[:id])
+    send_data("#{RAILS_ROOT}/files/clients/#{client.id}.pdf", :filename => "#{client.name}.pdf", :type => "application/pdf")
+  end
+
+end
+

This will read and stream the file 4Kb at the time, avoiding loading the entire file into memory at once. You can turn off streaming with the stream option or adjust the block size with the buffer_size option.

+ + + +

+ +	Be careful when using (or just don't use) "outside" data (params, cookies, etc) to locate the file on disk, as this is a security risk as someone could gain access to files they are not meant to have access to.

+ + + +

It is not recommended that you stream static files through Rails if you can instead keep them in a public folder on your web server. It is much more efficient to let the user download the file directly using Apache or another web server, keeping the request from unnecessarily going through the whole Rails stack.

10.2. RESTful downloads

While send_data works just fine, if you are creating a RESTful application having separate actions for file downloads is usually not necessary. In REST terminology, the PDF file from the example above can be considered just another representation of the client resource. Rails provides an easy and quite sleek way of doing "RESTful downloads". Let's try to rewrite the example so that the PDF download is a part of the show action:

class ClientsController < ApplicationController
+
+  # The user can request to receive this resource as HTML or PDF.
+  def show
+    @client = Client.find(params[:id])
+
+    respond_to do |format|
+      format.html
+      format.pdf{ render :pdf => generate_pdf(@client) }
+    end
+  end
+
+end
+

In order for this example to work, we have to add the PDF MIME type to Rails. This can be done by adding the following line to the file config/initializers/mime_types.rb:

Mime::Type.register "application/pdf", :pdf
+

+ + + +

+ +	Configuration files are not reloaded on each request, so you have to restart the server in order for their changes to take effect.

Now the user can request to get a PDF version of a client just by adding ".pdf" to the URL:

GET /clients/1.pdf

11. Parameter filtering

Rails keeps a log file for each environment (development, test and production) in the "log" folder. These are extremely useful when debugging what's actually going on in your application, but in a live application you may not want every bit of information to be stored in the log file. The filter_parameter_logging method can be used to filter out sensitive information from the log. It works by replacing certain keys in the params hash with "[FILTERED]" as they are written to the log. As an example, let's see how to filter all parameters with keys that include "password":

class ApplicationController < ActionController::Base
+
+  filter_parameter_logging :password
+
+end
+

The method works recursively through all levels of the params hash and takes an optional second parameter which is used as the replacement string if present. It can also take a block which receives each key in return and replaces those for which the block returns true.

12. Rescue

Most likely your application is going to contain bugs or otherwise throw an exception that needs to be handled. For example, if the user follows a link to a resource that no longer exists in the database, Active Record will throw the ActiveRecord::RecordNotFound exception. Rails' default exception handling displays a 500 Server Error message for all exceptions. If the request was made locally, a nice traceback and some added information gets displayed so you can figure out what went wrong and deal with it. If the request was remote Rails will just display a simple "500 Server Error" message to the user, or a "404 Not Found" if there was a routing error or a record could not be found. Sometimes you might want to customize how these errors are caught and how they're displayed to the user. There are several levels of exception handling available in a Rails application:

12.1. The default 500 and 404 templates

By default a production application will render either a 404 or a 500 error message. These messages are contained in static HTML files in the public folder, in 404.html and 500.html respectively. You can customize these files to add some extra information and layout, but remember that they are static; i.e. you can't use RHTML or layouts in them, just plain HTML.

12.2. `rescue_from`

If you want to do something a bit more elaborate when catching errors, you can use rescue_from, which handles exceptions of a certain type (or multiple types) in an entire controller and its subclasses. When an exception occurs which is caught by a rescue_from directive, the exception object is passed to the handler. The handler can be a method or a Proc object passed to the :with option. You can also use a block directly instead of an explicit Proc object.

Let's see how we can use rescue_from to intercept all ActiveRecord::RecordNotFound errors and do something with them.

class ApplicationController < ActionController::Base
+
+  rescue_from ActiveRecord::RecordNotFound, :with => :record_not_found
+
+private
+
+  def record_not_found
+    render :text => "404 Not Found", :status => 404
+  end
+
+end
+

Of course, this example is anything but elaborate and doesn't improve the default exception handling at all, but once you can catch all those exceptions you're free to do whatever you want with them. For example, you could create custom exception classes that will be thrown when a user doesn't have access to a certain section of your application:

class ApplicationController < ActionController::Base
+
+  rescue_from User::NotAuthorized, :with => :user_not_authorized
+
+private
+
+  def user_not_authorized
+    flash[:error] = "You don't have access to this section."
+    redirect_to :back
+  end
+
+end
+
+class ClientsController < ApplicationController
+
+  # Check that the user has the right authorization to access clients.
+  before_filter :check_authorization
+
+  # Note how the actions don't have to worry about all the auth stuff.
+  def edit
+    @client = Client.find(params[:id])
+  end
+
+private
+
+  # If the user is not authorized, just throw the exception.
+  def check_authorization
+    raise User::NotAuthorized unless current_user.admin?
+  end
+
+end
+

+ + + +

+ +	Certain exceptions are only rescuable from the ApplicationController class, as they are raised before the controller gets initialized and the action gets executed. See Pratik Naik's article on the subject for more information.

+ +

+ + diff --git a/railties/doc/guides/html/actioncontroller_basics.html.html b/railties/doc/guides/html/actioncontroller_basics.html.html deleted file mode 100644 index 4df3a7e9b2..0000000000 --- a/railties/doc/guides/html/actioncontroller_basics.html.html +++ /dev/null @@ -1,1125 +0,0 @@ - - - - - Action Controller basics - - - - - - - - - -

- - - -

Action Controller basics

1. What does a controller do?

2. Methods and actions

class ClientsController < ActionController::Base
-
-  # Actions are public methods
-  def new
-  end
-
-  # These methods are responsible for producing output
-  def edit
-  end
-
-# Helper methods are private and can not be used as actions
-private
-
-  def foo
-  end
-
-end
-

Private methods in a controller are also used as filters, which will be covered later in this guide.

def new
-  @client = Client.new
-end
-

The Layouts & rendering guide explains this in more detail.

3. Parameters

class ClientsController < ActionController::Base
-
-  # This action uses query string parameters because it gets run by a HTTP GET request,
-  # but this does not make any difference to the way in which the parameters are accessed.
-  # The URL for this action would look like this in order to list activated clients: /clients?status=activated
-  def index
-    if params[:status] = "activated"
-      @clients = Client.activated
-    else
-      @clients = Client.unativated
-    end
-  end
-
-  # This action uses POST parameters. They are most likely coming from an HTML
-  # form which the user has submitted. The URL for this RESTful request will
-  # be "/clients", and the data will be sent as part of the request body.
-  def create
-    @client = Client.new(params[:client])
-    if @client.save
-      redirect_to @client
-    else
-      # This line overrides the default rendering behavior, which would have been
-      # to render the "create" view.
-      render :action => "new"
-    end
-  end
-
-end
-

3.1. Hash and array parameters

The params hash is not limited to one-dimensional keys and values. It can contain arrays and (nested) hashes. To send an array of values, append "[]" to the key name:

GET /clients?ids[]=1&ids[2]&ids[]=3

The value of params[:ids] will now be ["1", "2", "3"]. Note that parameter values are always strings; Rails makes no attempt to guess or cast the type.

To send a hash you include the key name inside the brackets:

<form action="/clients" method="post">
-  <input type="text" name="client[name]" value="Acme" />
-  <input type="text" name="client[phone]" value="12345" />
-  <input type="text" name="client[address][postcode]" value="12345" />
-  <input type="text" name="client[address][city]" value="Carrot City" />
-</form>

3.2. Routing parameters

4. Session

-
-CookieStore - Stores everything on the client. -
-
-
-DRBStore - Stores the data on a DRb client. -
-
-
-MemCacheStore - Stores the data in MemCache. -
-
-
-ActiveRecordStore - Stores the data in a database using Active Record. -
-

All session stores store the session id in a cookie - there is no other way of passing it to the server. Most stores also use this key to locate the session data on the server.

If you need a different session storage mechanism, you can change it in the config/environment.rb file:

# Set to one of [:active_record_store, :drb_store, :mem_cache_store, :cookie_store]
-config.action_controller.session_store = :active_record_store
-

4.1. Disabling the session

Sometimes you don't need a session, and you can turn it off to avoid the unnecessary overhead. To do this, use the session class method in your controller:

class ApplicationController < ActionController::Base
-  session :off
-end
-

You can also turn the session on or off for a single controller:

# The session is turned off by default in ApplicationController, but we
-# want to turn it on for log in/out.
-class LoginsController < ActionController::Base
-  session :on
-end
-

Or even a single action:

class ProductsController < ActionController::Base
-  session :on, :only => [:create, :update]
-end
-

4.2. Accessing the session

In your controller you can access the session through the session instance method.

- - - -

- - There are two session methods, the class and the instance method. The class method which is described above is used to turn the session on and off while the instance method described below is used to access session values. The class method is used outside of method definitions while the instance methods is used inside methods, in actions or filters.

Session values are stored using key/value pairs like a hash:

class ApplicationController < ActionController::Base
-
-private
-
-  # Finds the User with the ID stored in the session with the key :current_user_id
-  # This is a common way to do user login in a Rails application; logging in sets the
-  # session value and logging out removes it.
-  def current_user
-    @_current_user ||= session[:current_user_id] && User.find(session[:current_user_id])
-  end
-
-end
-

To store something in the session, just assign it to the key like a hash:

class LoginsController < ApplicationController
-
-  # "Create" a login, aka "log the user in"
-  def create
-    if user = User.authenticate(params[:username, params[:password])
-      # Save the user ID in the session so it can be used in subsequent requests
-      session[:current_user_id] = user.id
-      redirect_to root_url
-    end
-  end
-
-end
-

To remove something from the session, assign that key to be nil:

class LoginsController < ApplicationController
-
-  # "Delete" a login, aka "log the user out"
-  def destroy
-    # Remove the user id from the session
-    session[:current_user_id] = nil
-    redirect_to root_url
-  end
-
-end
-

To reset the entire session, use reset_session.

4.3. The flash

class LoginsController < ApplicationController
-
-  def destroy
-    session[:current_user_id] = nil
-    flash[:notice] = "You have successfully logged out"
-    redirect_to root_url
-  end
-
-end
-

<html>
-  <!-- <head/> -->
-  <body>
-    <% if flash[:notice] -%>
-      <p class="notice"><%= flash[:notice] %></p>
-    <% end -%>
-    <% if flash[:error] -%>
-      <p class="error"><%= flash[:error] %></p>
-    <% end -%>
-    <!-- more content -->
-  </body>
-</html>

This way, if an action sets an error or a notice message, the layout will display it automatically.

If you want a flash value to be carried over to another request, use the keep method:

class MainController < ApplicationController
-
-  # Let's say this action corresponds to root_url, but you want all requests here to be redirected to
-  # UsersController#index. If an action sets the flash and redirects here, the values would normally be
-  # lost when another redirect happens, but you can use keep to make it persist for another request.
-  def index
-    flash.keep # Will persist all flash values. You can also use a key to keep only that value: flash.keep(:notice)
-    redirect_to users_url
-  end
-
-end
-

4.3.1. flash.now

class ClientsController < ApplicationController
-
-  def create
-    @client = Client.new(params[:client])
-    if @client.save
-      # ...
-    else
-      flash.now[:error] = "Could not save client"
-      render :action => "new"
-    end
-  end
-
-end
-

5. Cookies

class CommentsController < ApplicationController
-
-  def new
-    #Auto-fill the commenter's name if it has been stored in a cookie
-    @comment = Comment.new(:name => cookies[:commenter_name])
-  end
-
-  def create
-    @comment = Comment.new(params[:comment])
-    if @comment.save
-      flash[:notice] = "Thanks for your comment!"
-      if params[:remember_name]
-        # Remember the commenter's name
-        cookies[:commenter_name] = @comment.name
-      else
-        # Don't remember, and delete the name if it has been remembered before
-        cookies.delete(:commenter_name)
-      end
-      redirect_to @comment.article
-    else
-      render :action => "new"
-    end
-  end
-
-end
-

Note that while for session values, you set the key to nil, to delete a cookie value, you use cookies.delete(:key).

6. Filters

class ApplicationController < ActionController::Base
-
-private
-
-  def require_login
-    unless logged_in?
-      flash[:error] = "You must be logged in to access this section"
-      redirect_to new_login_url # Prevents the current action from running
-    end
-  end
-
-  # The logged_in? method simply returns true if the user is logged in and
-  # false otherwise. It does this by "booleanizing" the current_user method
-  # we created previously using a double ! operator. Note that this is not
-  # common in Ruby and is discouraged unless you really mean to convert something
-  # into true or false.
-  def logged_in?
-    !!current_user
-  end
-
-end
-

class ApplicationController < ActionController::Base
-
-  before_filter :require_login
-
-end
-

class LoginsController < Application
-
-  skip_before_filter :require_login, :only => [:new, :create]
-
-end
-

6.1. After filters and around filters

TODO: Find a real example for an around filter

# Example taken from the Rails API filter documentation:
-# http://api.rubyonrails.org/classes/ActionController/Filters/ClassMethods.html
-class ApplicationController < Application
-
-  around_filter :catch_exceptions
-
-private
-
-  def catch_exceptions
-    yield
-  rescue => exception
-    logger.debug "Caught exception! #{exception}"
-    raise
-  end
-
-end
-

6.2. Other ways to use filters

While the most common way to use filters is by creating private methods and using *_filter to add them, there are two other ways to do the same thing.

The first is to use a block directly with the *_filter methods. The block receives the controller as an argument, and the require_login filter from above could be rewritte to use a block:

class ApplicationController < ActionController::Base
-
-  before_filter { |controller| redirect_to new_login_url unless controller.send(:logged_in?) }
-
-end
-

class ApplicationController < ActionController::Base
-
-  before_filter LoginFilter
-
-end
-
-class LoginFilter
-
-  def self.filter(controller)
-    unless logged_in?
-      controller.flash[:error] = "You must be logged in to access this section"
-      controller.redirect_to controller.new_login_url
-    end
-  end
-
-end
-

The Rails API documentation has more information on using filters.

7. Verification

Let's see how we can use verification to make sure the user supplies a username and a password in order to log in:

class LoginsController < ApplicationController
-
-  verify :params => [:username, :password],
-         :render => {:action => "new"},
-         :add_flash => {:error => "Username and password required to log in"}
-
-  def create
-    @user = User.authenticate(params[:username], params[:password])
-    if @user
-      flash[:notice] = "You're logged in"
-      redirect_to root_url
-    else
-      render :action => "new"
-    end
-  end
-
-end
-

class LoginsController < ApplicationController
-
-  verify :params => [:username, :password],
-         :render => {:action => "new"},
-         :add_flash => {:error => "Username and password required to log in"},
-         :only => :create #Only run this verification for the "create" action
-
-end
-

8. The request and response objects

8.1. The request

The request object contains a lot of useful information about the request coming in from the client. To get a full list of the available methods, refer to the API documentation.

-
-host - The hostname used for this request. -
-
-
-domain - The hostname without the first part (usually "www"). -
-
-
-format - The content type requested by the client. -
-
-
-method - The HTTP method used for the request. -
-
-
-get?, post?, put?, delete?, head? - Returns true if the HTTP method is get/post/put/delete/head. -
-
-
-headers - Returns a hash containing the headers associated with the request. -
-
-
-port - The port number (integer) used for the request. -
-
-
-protocol - The protocol used for the request. -
-
-
-query_string - The query string part of the URL - everything after "?". -
-
-
-remote_ip - The IP address of the client. -
-
-
-url - The entire URL used for the request. -
-

8.1.1. path_parameters, query_parameters and request_parameters

TODO: Does this belong here?

8.2. The response

-
-body - This is the string of data being sent back to the client. This is most often HTML. -
-
-
-status - The HTTP status code for the response, like 200 for a successful request or 404 for file not found. -
-
-
-location - The URL the client is being redirected to, if any. -
-
-
-content_type - The content type of the response. -
-
-
-charset - The character set being used for the response. Default is "utf8". -
-

9. HTTP Basic Authentication

class AdminController < ApplicationController
-
-  USERNAME, PASSWORD = "humbaba", "f59a4805511bf4bb61978445a5380c6c"
-
-  before_filter :authenticate
-
-private
-
-  def authenticate
-    authenticate_or_request_with_http_basic do |username, password|
-      username == USERNAME && Digest::MD5.hexdigest(password) == PASSWORD
-    end
-  end
-
-end
-

10. Streaming and file downloads

To stream data to the client, use send_data:

require "prawn"
-class ClientsController < ApplicationController
-
-  # Generate a PDF document with information on the client and return it.
-  # The user will get the PDF as a file download.
-  def download_pdf
-    client = Client.find(params[:id])
-    send_data(generate_pdf, :filename => "#{client.name}.pdf", :type => "application/pdf")
-  end
-
-private
-
-  def generate_pdf(client)
-    Prawn::Document.new do
-      text client.name, :align => :center
-      text "Address: #{client.address}"
-      text "Email: #{client.email}"
-    end.render
-  end
-
-end
-

10.1. Sending files

class ClientsController < ApplicationController
-
-  # Stream a file that has already been generated and stored on disk
-  def download_pdf
-    client = Client.find(params[:id])
-    send_data("#{RAILS_ROOT}/files/clients/#{client.id}.pdf", :filename => "#{client.name}.pdf", :type => "application/pdf")
-  end
-
-end
-

- - - -

- -	Be careful when using (or just don't use) "outside" data (params, cookies, etc) to locate the file on disk, as this is a security risk as someone could gain access to files they are not meant to have access to.

- - - -

10.2. RESTful downloads

class ClientsController < ApplicationController
-
-  # The user can request to receive this resource as HTML or PDF.
-  def show
-    @client = Client.find(params[:id])
-
-    respond_to do |format|
-      format.html
-      format.pdf{ render :pdf => generate_pdf(@client) }
-    end
-  end
-
-end
-

In order for this example to work, we have to add the PDF MIME type to Rails. This can be done by adding the following line to the file config/initializers/mime_types.rb:

Mime::Type.register "application/pdf", :pdf
-

- - - -

- -	Configuration files are not reloaded on each request, so you have to restart the server in order for their changes to take effect.

Now the user can request to get a PDF version of a client just by adding ".pdf" to the URL:

GET /clients/1.pdf

11. Parameter filtering

class ApplicationController < ActionController::Base
-
-  filter_parameter_logging :password
-
-end
-

12. Rescue

12.1. The default 500 and 404 templates

12.2. `rescue_from`

Let's see how we can use rescue_from to intercept all ActiveRecord::RecordNotFound errors and do something with them.

class ApplicationController < ActionController::Base
-
-  rescue_from ActiveRecord::RecordNotFound, :with => :record_not_found
-
-private
-
-  def record_not_found
-    render :text => "404 Not Found", :status => 404
-  end
-
-end
-

class ApplicationController < ActionController::Base
-
-  rescue_from User::NotAuthorized, :with => :user_not_authorized
-
-private
-
-  def user_not_authorized
-    flash[:error] = "You don't have access to this section."
-    redirect_to :back
-  end
-
-end
-
-class ClientsController < ApplicationController
-
-  # Check that the user has the right authorization to access clients.
-  before_filter :check_authorization
-
-  # Note how the actions don't have to worry about all the auth stuff.
-  def edit
-    @client = Client.find(params[:id])
-  end
-
-private
-
-  # If the user is not authorized, just throw the exception.
-  def check_authorization
-    raise User::NotAuthorized unless current_user.admin?
-  end
-
-end
-

- - - -

- -	Certain exceptions are only rescuable from the ApplicationController class, as they are raised before the controller gets initialized and the action gets executed. See Pratik Naik's article on the subject for more information.

- -

- - diff --git a/railties/doc/guides/html/benchmarking_and_profiling.html b/railties/doc/guides/html/benchmarking_and_profiling.html new file mode 100644 index 0000000000..e20197b2f0 --- /dev/null +++ b/railties/doc/guides/html/benchmarking_and_profiling.html @@ -0,0 +1,1015 @@ + + + + + Benchmarking and Profiling Rails + + + + + + + + + +

+ + + +

Benchmarking and Profiling Rails

This guide covers the benchmarking and profiling tactics/tools of Rails and Ruby in general. By referring to this guide, you will be able to:

+
+Understand the various types of benchmarking and profiling metrics +
+
+
+Generate performance/benchmarking tests +
+
+
+Use GC patched Ruby binary to measure memory usage and object allocation +
+
+
+Understand the information provided by Rails inside the log files +
+
+
+Learn about various tools facilitating benchmarking and profiling +
+

1. Why Benchmark and Profile ?

Benchmarking and Profiling is an integral part of the development cycle. It is very important that you don't make your end users wait for too long before the page is completely loaded. Ensuring a plesant browsing experience to the end users and cutting cost of unnecessary hardwares is important for any web application.

1.1. What is the difference between benchmarking and profiling ?

Benchmarking is the process of finding out if a piece of code is slow or not. Whereas profiling is the process of finding out what exactly is slowing down that piece of code.

2. Using and understanding the log files

Rails logs files containt basic but very useful information about the time taken to serve every request. A typical log entry looks something like :

Processing ItemsController#index (for 127.0.0.1 at 2008-10-17 00:08:18) [GET]
+  Session ID: BAh7BiIKZmxhc2hJQzonQWN0aHsABjoKQHVzZWR7AA==--83cff4fe0a897074a65335
+  Parameters: {"action"=>"index", "controller"=>"items"}
+Rendering template within layouts/items
+Rendering items/index
+Completed in 5ms (View: 2, DB: 0) | 200 OK [http://localhost/items]
+

For this section, we're only interested in the last line from that log entry:

Completed in 5ms (View: 2, DB: 0) | 200 OK [http://localhost/items]
+

This data is fairly straight forward to understand. Rails uses millisecond(ms) as the metric to measures the time taken. The complete request spent 5 ms inside Rails, out of which 2 ms were spent rendering views and none was spent communication with the database. It's safe to assume that the remaining 3 ms were spent inside the controller.

3. Helper methods

Rails provides various helper methods inside Active Record, Action Controller and Action View to measure the time taken by a specific code. The method is called benchmark() in all three components.

Project.benchmark("Creating project") do
+  project = Project.create("name" => "stuff")
+  project.create_manager("name" => "David")
+  project.milestones << Milestone.find(:all)
+end
+

The above code benchmarks the multiple statments enclosed inside Project.benchmark("Creating project") do..end block and prints the results inside log files. The statement inside log files will look like:

Creating projectem (185.3ms)
+

Please refer to API docs for optional options to benchmark()

Similarly, you could use this helper method inside controllers ( Note that it's a class method here ):

def process_projects
+  self.class.benchmark("Processing projects") do
+    Project.process(params[:project_ids])
+    Project.update_cached_projects
+  end
+end
+

and views:

<% benchmark("Showing projects partial") do %>
+  <%= render :partial => @projects %>
+<% end %>
+

4. Performance Test Cases

Rails provides a very easy to write performance test cases, which look just like the regular integration tests.

If you have a look at test/performance/browsing_test.rb in a newly created Rails application:

require 'test_helper'
+require 'performance_test_help'
+
+# Profiling results for each test method are written to tmp/performance.
+class BrowsingTest < ActionController::PerformanceTest
+  def test_homepage
+    get '/'
+  end
+end
+

This is an automatically generated example performance test file, for testing performance of homepage(/) of the application.

4.1. Modes

4.1.1. Benchmarking

4.1.2. Profiling

4.2. Metrics

4.2.1. Process Time

CPU Cycles.

4.2.2. Memory

Memory taken.

4.2.3. Objects

Objects allocated.

4.2.4. GC Runs

Number of times the Ruby GC was run.

4.2.5. GC Time

Time spent running the Ruby GC.

4.3. Preparing Ruby and Ruby-prof

Before we go ahead, Rails performance testing requires you to build a special Ruby binary with some super powers - GC patch for measuring GC Runs/Time. This process is very straight forward. If you've never compiled a Ruby binary before, you can follow the following steps to build a ruby binary inside your home directory:

4.3.1. Compile

[lifo@null ~]$ mkdir rubygc
+[lifo@null ~]$ wget ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.6-p111.tar.gz
+[lifo@null ~]$ tar -xzvf ruby-1.8.6-p111.tar.gz
+[lifo@null ~]$ cd ruby-1.8.6-p111
+[lifo@null ruby-1.8.6-p111]$ curl http://rubyforge.org/tracker/download.php/1814/7062/17676/3291/ruby186gc.patch | patch -p0
+[lifo@null ruby-1.8.6-p111]$ ./configure --prefix=/Users/lifo/rubygc
+[lifo@null ruby-1.8.6-p111]$ make && make install
+

4.3.2. Prepare aliases

Add the following lines in your ~/.profile for convenience:

alias gcruby='/Users/lifo/rubygc/bin/ruby'
+alias gcrake='/Users/lifo/rubygc/bin/rake'
+alias gcgem='/Users/lifo/rubygc/bin/gem'
+alias gcirb='/Users/lifo/rubygc/bin/irb'
+alias gcrails='/Users/lifo/rubygc/bin/rails'

4.3.3. Install rubygems and some basic gems

[lifo@null ~]$ wget http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz
+[lifo@null ~]$ tar -xzvf rubygems-1.2.0.tgz
+[lifo@null ~]$ cd rubygems-1.2.0
+[lifo@null rubygems-1.2.0]$ gcruby setup.rb
+[lifo@null rubygems-1.2.0]$ cd ~
+[lifo@null ~]$ gcgem install rake
+[lifo@null ~]$ gcgem install rails

4.3.4. Install MySQL gem

[lifo@null ~]$ gcgem install mysql

If this fails, you can try to install it manually:

[lifo@null ~]$ cd /Users/lifo/rubygc/lib/ruby/gems/1.8/gems/mysql-2.7/
+[lifo@null mysql-2.7]$ gcruby extconf.rb --with-mysql-config
+[lifo@null mysql-2.7]$ make && make install

4.4. Installing Jeremy Kemper's ruby-prof

We also need to install Jeremy's ruby-prof gem using our newly built ruby:

[lifo@null ~]$ git clone git://github.com/jeremy/ruby-prof.git
+[lifo@null ~]$ cd ruby-prof/
+[lifo@null ruby-prof (master)]$ gcrake gem
+[lifo@null ruby-prof (master)]$ gcgem install pkg/ruby-prof-0.6.1.gem
+

4.5. Generating performance test

Rails provides a simple generator for creating new performance tests:

[User profiling_tester (master)]$ script/generate performance_test homepage
+

This will generate test/performance/homepage_test.rb:

require 'test_helper'
+require 'performance_test_help'
+
+class HomepageTest < ActionController::PerformanceTest
+  # Replace this with your real tests.
+  def test_homepage
+    get '/'
+  end
+end

Which you can modify to suit your needs.

4.6. Running tests

5. Understanding Performance Tests Outputs

5.1. Our First Performance Test

So how do we profile a request.

One of the things that is important to us is how long it takes to render the home page - so let's make a request to the home page. Once the request is complete, the results will be outputted in the terminal.

In the terminal run

[User profiling_tester]$ gcruby tests/performance/homepage.rb
+

After the tests runs for a few seconds you should see something like this.

HomepageTest#test_homepage (19 ms warmup)
+        process_time: 26 ms
+              memory: 298.79 KB
+             objects: 1917
+
+Finished in 2.207428 seconds.

Simple but efficient.

+
+Process Time refers to amount of time necessary to complete the action. +
+
+
+memory is the amount of information loaded into memory +
+
+
+object ??? #TODO find a good definition. Is it the amount of objects put into a ruby heap for this process? +
+

In addition we also gain three types of itemized log files for each of these outputs. They can be found in your tmp directory of your application.

The Three types are

+
+Flat File - A simple text file with the data laid out in a grid +
+
+
+Graphical File - A html colored coded version of the simple text file with hyperlinks between the various methods. Most useful is the bolding of the main processes for each portion of the action. +
+
+
+Tree File - A file output that can be use in conjunction with KCachegrind to visualize the process +
+

+ + + +

+ +	KCachegrind is Linux only. For Mac this means you have to do a full KDE install to have it working in your OS. Which is over 3 gigs in size. For windows there is clone called wincachegrind but it is no longer actively being developed.

Below are examples for Flat Files and Graphical Files

5.2. Flat Files

Example: Flat File Output Processing Time

Thread ID: 2279160 +Total: 0.026097

%self     total     self     wait    child    calls  name
+ 6.41      0.06     0.04     0.00     0.02      571  Kernel#===
+ 3.17      0.00     0.00     0.00     0.00      172  Hash#[]
+ 2.42      0.00     0.00     0.00     0.00       13  MonitorMixin#mon_exit
+ 2.05      0.00     0.00     0.00     0.00       15  Array#each
+ 1.56      0.00     0.00     0.00     0.00        6  Logger#add
+ 1.55      0.00     0.00     0.00     0.00       13  MonitorMixin#mon_enter
+ 1.36      0.03     0.00     0.00     0.03        1  ActionController::Integration::Session#process
+ 1.31      0.00     0.00     0.00     0.00       13  MonitorMixin#mon_release
+ 1.15      0.00     0.00     0.00     0.00        8  MonitorMixin#synchronize-1
+ 1.09      0.00     0.00     0.00     0.00       23  Class#new
+ 1.03      0.01     0.00     0.00     0.01        5  MonitorMixin#synchronize
+ 0.89      0.00     0.00     0.00     0.00       74  Hash#default
+ 0.89      0.00     0.00     0.00     0.00        6  Hodel3000CompliantLogger#format_message
+ 0.80      0.00     0.00     0.00     0.00        9  c
+ 0.80      0.00     0.00     0.00     0.00       11  ActiveRecord::ConnectionAdapters::ConnectionHandler#retrieve_connection_pool
+ 0.79      0.01     0.00     0.00     0.01        1  ActionController::Benchmarking#perform_action_without_rescue
+ 0.18      0.00     0.00     0.00     0.00       17  <Class::Object>#allocate

So what do these columns tell us:

+
+%self - The percentage of time spent processing the method. This is derived from self_time/total_time +
+
+
+total - The time spent in this method and its children. +
+
+
+self - The time spent in this method. +
+
+
+wait - Time processed was queued +
+
+
+child - The time spent in this method's children. +
+
+
+calls - The number of times this method was called. +
+
+
+name - The name of the method. +
+

Name can be displayed three seperate ways: + #toplevel - The root method that calls all other methods + MyObject#method - Example Hash#each, The class Hash is calling the method each + * <Object:MyObject>#test - The <> characters indicate a singleton method on a singleton class. Example <Class::Object>#allocate

Methods are sorted based on %self. Hence the ones taking the most time and resources will be at the top.

So for Array#each which is calling each on the class array. We find that it processing time is 2% of the total and was called 15 times. The rest of the information is 0.00 because the process is so fast it isn't recording times less then 100 ms.

Example: Flat File Memory Output

Thread ID: 2279160 +Total: 509.724609

%self     total     self     wait    child    calls  name
+ 4.62     23.57    23.57     0.00     0.00       34  String#split
+ 3.95     57.66    20.13     0.00    37.53        3  <Module::YAML>#quick_emit
+ 2.82     23.70    14.35     0.00     9.34        2  <Module::YAML>#quick_emit-1
+ 1.37     35.87     6.96     0.00    28.91        1  ActionView::Helpers::FormTagHelper#form_tag
+ 1.35      7.69     6.88     0.00     0.81        1  ActionController::HttpAuthentication::Basic::ControllerMethods#authenticate_with_http_basic
+ 1.06      6.09     5.42     0.00     0.67       90  String#gsub
+ 1.01      5.13     5.13     0.00     0.00       27  Array#-

Very similar to the processing time format. The main difference here is that instead of calculating time we are now concerned with the amount of KB put into memory (or is it strictly into the heap) can I get clarification on this minor point?

So for <Module::YAML>#quick_emit which is singleton method on the class YAML it uses 57.66 KB in total, 23.57 through its own actions, 6.69 from actions it calls itself and that it was called twice.

Example: Flat File Objects

Thread ID: 2279160 +Total: 6537.000000

%self     total     self     wait    child    calls  name
+15.16   1096.00   991.00     0.00   105.00       66  Hash#each
+ 5.25    343.00   343.00     0.00     0.00        4  Mysql::Result#each_hash
+ 4.74   2203.00   310.00     0.00  1893.00       42  Array#each
+ 3.75   4529.00   245.00     0.00  4284.00        1  ActionView::Base::CompiledTemplates#_run_erb_47app47views47layouts47application46html46erb
+ 2.00    136.00   131.00     0.00     5.00       90  String#gsub
+ 1.73    113.00   113.00     0.00     0.00       34  String#split
+ 1.44    111.00    94.00     0.00    17.00       31  Array#each-1

#TODO Find correct terminology for how to describe what this is exactly profiling as in are there really 2203 array objects or 2203 pointers to array objects?.

5.3. Graph Files

While the information gleamed from flat files is very useful we still don't know which processes each method is calling. We only know how many. This is not true for a graph file. Below is a text representation of a graph file. The actual graph file is an html entity and an example of which can be found Here

#TODO (Handily the graph file has links both between it many processes and to the files that actually contain them for debugging. + )

Example: Graph File

Thread ID: 21277412

  %total   %self     total      self    children               calls   Name
+/____________________________________________________________________________/
+100.00%   0.00%      8.77      0.00      8.77                   1     #toplevel*
+                     8.77      0.00      8.77                 1/1     Object#run_primes
+/____________________________________________________________________________/
+                     8.77      0.00      8.77                 1/1     #toplevel
+100.00%   0.00%      8.77      0.00      8.77                   1     Object#run_primes*
+                     0.02      0.00      0.02                 1/1     Object#make_random_array
+                     2.09      0.00      2.09                 1/1     Object#find_largest
+                     6.66      0.00      6.66                 1/1     Object#find_primes
+/____________________________________________________________________________/
+                     0.02      0.02      0.00                 1/1     Object#make_random_array
+0.18%     0.18%      0.02      0.02      0.00                   1     Array#each_index
+                     0.00      0.00      0.00             500/500     Kernel.rand
+                     0.00      0.00      0.00             500/501     Array#[]=
+/____________________________________________________________________________/

As you can see the calls have been separated into slices, no longer is the order determined by process time but instead from hierarchy. Each slice profiles a primary entry, with the primary entry's parents being shown above itself and it's children found below. A primary entry can be ascertained by it having values in the %total and %self columns. Here the main entry here have been bolded for connivence.

So if we look at the last slice. The primary entry would be Array#each_index. It takes 0.18% of the total process time and it is only called once. It is called from Object#make_random_array which is only called once. It's children are Kernal.rand which is called by it all 500 its times that it was call in this action and Arry#[]= which was called 500 times by Array#each_index and once by some other entry.

5.4. Tree Files

It's pointless trying to represent a tree file textually so here's a few pretty pictures of it's usefulness

KCachegrind Graph

+ Graph created by KCachegrind +

KCachegrind List

+ List created by KCachegrind +

#TODO Add a bit more information to this.

6. Getting to the Point of all of this

Now I know all of this is a bit dry and academic. But it's a very powerful tool when you know how to leverage it properly. Which we are going to take a look at in our next section

7. Real Life Example

7.1. The setup

So I have been building this application for the last month and feel pretty good about the ruby code. I'm readying it for beta testers when I discover to my shock that with less then twenty people it starts to crash. It's a pretty simple Ecommerce site so I'm very confused by what I'm seeing. On running looking through my log files I find to my shock that the lowest time for a page run is running around 240 ms. My database finds aren't the problems so I'm lost as to what is happening to cause all this. Lets run a benchmark.

class HomepageTest < ActionController::PerformanceTest
+  # Replace this with your real tests.
+  def test_homepage
+    get '/'
+  end
+end
+

Example: Output

HomepageTest#test_homepage (115 ms warmup)
+        process_time: 591 ms
+        memory: 3052.90 KB
+        objects: 59471

Obviously something is very very wrong here. 3052.90 Kb to load my minimal homepage. For Comparison for another site running well I get this for my homepage test.

Example: Default

HomepageTest#test_homepage (19 ms warmup)
+        process_time: 26 ms
+              memory: 298.79 KB
+             objects: 1917

that over a factor of ten difference. Lets look at our flat process time file to see if anything pops out at us.

Example: Process time

20.73      0.39     0.12     0.00     0.27      420  Pathname#cleanpath_aggressive
+17.07      0.14     0.10     0.00     0.04     3186  Pathname#chop_basename
+ 6.47      0.06     0.04     0.00     0.02     6571  Kernel#===
+ 5.04      0.06     0.03     0.00     0.03      840  Pathname#initialize
+ 5.03      0.05     0.03     0.00     0.02        4  ERB::Compiler::ExplicitScanner#scan
+ 4.51      0.03     0.03     0.00     0.00     9504  String#==
+ 2.94      0.46     0.02     0.00     0.44     1393  String#gsub
+ 2.66      0.09     0.02     0.00     0.07      480  Array#each
+ 2.46      0.01     0.01     0.00     0.00     3606  Regexp#to_s

Yes indeed we seem to have found the problem. Pathname#cleanpath_aggressive is taking nearly a quarter our process time and Pathname#chop_basename another 17%. From here I do a few more benchmarks to make sure that these processes are slowing down the other pages. They are so now I know what I must do. If we can get rid of or shorten these processes we can make our pages run much quicker.

Now both of these are main ruby processes so are goal right now is to find out what other process is calling them. Glancing at our Graph file I see that #cleanpath is calling #cleanpath_aggressive. #cleanpath is being called by String#gsub and from there some html template errors. But my page seems to be rendering fine. why would it be calling template errors. I'm decide to check my object flat file to see if I can find any more information.

Example: Objects Created

20.74  34800.00 12324.00     0.00 22476.00      420  Pathname#cleanpath_aggressive
+16.79  18696.00  9978.00     0.00  8718.00     3186  Pathname#chop_basename
+11.47  13197.00  6813.00     0.00  6384.00      480  Array#each
+ 8.51  41964.00  5059.00     0.00 36905.00     1386  String#gsub
+ 6.07   3606.00  3606.00     0.00     0.00     3606  Regexp#to_s

nope nothing new here. Lets look at memory usage

Example: Memory Consuption

 40.17   1706.80  1223.70     0.00   483.10     3186  Pathname#chop_basename
+ 14.92    454.47   454.47     0.00     0.00     3606  Regexp#to_s
+  7.09   2381.36   215.99     0.00  2165.37     1386  String#gsub
+  5.08    231.19   154.73     0.00    76.46      420  Pathname#prepend_prefix
+  2.34     71.35    71.35     0.00     0.00     1265  String#initialize_copy

Ok so it seems Regexp#to_s is the second costliest process. At this point I try to figure out what could be calling a regular expression cause I very rarely use them. Going over my standard layout I discover at the top.

<%if request.env["HTTP_USER_AGENT"].match(/Opera/)%>
+<%= stylesheet_link_tag "opera" %>
+<% end %>
+

That's wrong. I mistakenly am using a search function for a simple compare function. Lets fix that.

<%if request.env["HTTP_USER_AGENT"] =~ /Opera/%>
+<%= stylesheet_link_tag "opera" %>
+<% end %>
+

I'll now try my test again.

process_time: 75 ms
+              memory: 519.95 KB
+             objects: 6537

Much better. The problem has been solved. Now I should have realized earlier due to the String#gsub that my problem had to be with reqexp serch function but such knowledge comes with time. Looking through the mass output data is a skill.

8. Get Yourself a Game Plan

You end up dealing with a large amount of data whenever you profile an application. It's crucial to use a rigorous approach to analyzing your application's performance else fail miserably in a vortex of numbers. This leads us to -

8.1. The Analysis Process

I’m going to give an example methodology for conducting your benchmarking and profiling on an application. It is based on your typical scientific method.

For something as complex as Benchmarking you need to take any methodology with a grain of salt but there are some basic strictures that you can depend on.

Formulate a question you need to answer which is simple, tests the smallest measurable thing possible, and is exact. This is typically the hardest part of the experiment. From there some steps that you should follow are.

+
+Develop a set of variables and processes to measure in order to answer this question! +
+
+
+Profile based on the question and variables. Key problems to avoid when designing this experiment are: +
+
- +
  +Confounding: Test one thing at a time, keep everything the same so you don't poison the data with uncontrolled processes. +
  +
- +
  +Cross Contamination: Make sure that runs from one test do not harm the other tests. +
  +
- +
  +Steady States: If you’re testing long running process. You must take the ramp up time and performance hit into your initial measurements. +
  +
- +
  +Sampling Error: Data should perform have a steady variance or range. If you get wild swings or sudden spikes, etc. then you must either account for the reason why or you have a sampling error. +
  +
- +
  +Measurement Error: Aka Human error, always go through your calculations at least twice to make sure there are no mathematical errors. . +
  +
+
+
+Do a small run of the experiment to verify the design. +
+
+
+Use the small run to determine a proper sample size. +
+
+
+Run the test. +
+
+
+Perform the analysis on the results and determine where to go from there. +
+

Note: Even though we are using the typical scientific method; developing a hypothesis is not always useful in terms of profiling.

9. Other Profiling Tools

There are a lot of great profiling tools out there. Some free, some not so free. This is a sort list detailing some of them.

9.1. httperf

http://www.hpl.hp.com/research/linux/httperf/

A necessary tool in your arsenal. Very useful for load testing your website.

#TODO write and link to a short article on how to use httperf. Anybody have a good tutorial availble.

9.2. Rails Analyzer

The Rails Analyzer project contains a collection of tools for Rails. It's open source and pretty speedy. It's not being actively worked on but is still contains some very useful tools.

+
+The Production Log Analyzer examines Rails log files and gives back a report. It also includes action_grep which will give you all log results for a particular action. +
+
+
+The Action Profiler similar to Ruby-Prof profiler. +
+
+
+rails_stat which gives a live counter of requests per second of a running Rails app. +
+
+
+The SQL Dependency Grapher allows you to visualize the frequency of table dependencies in a Rails application. +
+

Their project homepage can be found at http://rails-analyzer.rubyforge.org/

The one major caveat is that it needs your log to be in a different format from how rails sets it up specifically SyslogLogger.

9.2.1. SyslogLogger

SyslogLogger is a Logger work-alike that logs via syslog instead of to a file. You can add SyslogLogger to your Rails production environment to aggregate logs between multiple machines.

More information can be found out at http://rails-analyzer.rubyforge.org/hacks/classes/SyslogLogger.html

If you don't have access to your machines root system or just want something a bit easier to implement there is also a module developed by Geoffrey Grosenbach

9.2.2. A Hodel 3000 Compliant Logger for the Rest of Us

Directions taken from +link to module file

Just put the module in your lib directory and add this to your environment.rb in it's config portion.

require 'hodel_3000_compliant_logger'
+config.logger = Hodel3000CompliantLogger.new(config.log_path)

It's that simple. Your log output on restart should look like this.

Example: Hodel 3000 Example

Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
+Parameters: {"action"=>"shipping", "controller"=>"checkout"}
+Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
+[4;36;1mBook Columns (0.003155)[0m   [0;1mSHOW FIELDS FROM `books`[0m
+Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
+[4;35;1mBook Load (0.000881)[0m   [0mSELECT * FROM `books` WHERE (`books`.`id` = 1 AND (`books`.`sold` = 1)) [0m
+Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
+[4;36;1mShippingAddress Columns (0.002683)[0m   [0;1mSHOW FIELDS FROM `shipping_addresses`[0m
+Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
+[4;35;1mBook Load (0.000362)[0m   [0mSELECT ounces FROM `books` WHERE (`books`.`id` = 1) [0m
+Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
+Rendering template within layouts/application
+Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
+Rendering checkout/shipping
+Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
+[4;36;1mBook Load (0.000548)[0m   [0;1mSELECT * FROM `books`
+WHERE (sold = 0) LIMIT 3[0m
+Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
+[4;35;1mAuthor Columns (0.002571)[0m   [0mSHOW FIELDS FROM `authors`[0m
+Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
+[4;36;1mAuthor Load (0.000811)[0m   [0;1mSELECT * FROM `authors` WHERE (`authors`.`id` = 1) [0m
+Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
+Rendered store/_new_books (0.01358)
+Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
+Completed in 0.37297 (2 reqs/sec) | Rendering: 0.02971 (7%) | DB: 0.01697 (4%) | 200 OK [https://secure.jeffbooks/checkout/shipping]

9.3. Palmist

An open source mysql query analyzer. Full featured and easy to work with. Also requires Hodel 3000 +http://www.flyingmachinestudios.com/projects/

9.4. New Relic

http://www.newrelic.com/

Pretty nifty performance tools, pricey though. They do have a basic free +service both for when in development and when you put your application into production. Very simple installation and signup.

#TODO more in-depth without being like an advertisement.

9.4.1. Manage

Like new relic a production monitoring tool.

10. Changelog

Lighthouse ticket

+
+October 17, 2008: First revision by Pratik +
+
+
+September 6, 2008: Initial version by Matthew Bergman <MzbPhoto@gmail.com> +
+

+ +

+ + diff --git a/railties/doc/guides/html/benchmarking_and_profiling.html.html b/railties/doc/guides/html/benchmarking_and_profiling.html.html deleted file mode 100644 index e20197b2f0..0000000000 --- a/railties/doc/guides/html/benchmarking_and_profiling.html.html +++ /dev/null @@ -1,1015 +0,0 @@ - - - - - Benchmarking and Profiling Rails - - - - - - - - - -

- - - -

Benchmarking and Profiling Rails

This guide covers the benchmarking and profiling tactics/tools of Rails and Ruby in general. By referring to this guide, you will be able to:

-
-Understand the various types of benchmarking and profiling metrics -
-
-
-Generate performance/benchmarking tests -
-
-
-Use GC patched Ruby binary to measure memory usage and object allocation -
-
-
-Understand the information provided by Rails inside the log files -
-
-
-Learn about various tools facilitating benchmarking and profiling -
-

1. Why Benchmark and Profile ?

1.1. What is the difference between benchmarking and profiling ?

Benchmarking is the process of finding out if a piece of code is slow or not. Whereas profiling is the process of finding out what exactly is slowing down that piece of code.

2. Using and understanding the log files

Rails logs files containt basic but very useful information about the time taken to serve every request. A typical log entry looks something like :

Processing ItemsController#index (for 127.0.0.1 at 2008-10-17 00:08:18) [GET]
-  Session ID: BAh7BiIKZmxhc2hJQzonQWN0aHsABjoKQHVzZWR7AA==--83cff4fe0a897074a65335
-  Parameters: {"action"=>"index", "controller"=>"items"}
-Rendering template within layouts/items
-Rendering items/index
-Completed in 5ms (View: 2, DB: 0) | 200 OK [http://localhost/items]
-

For this section, we're only interested in the last line from that log entry:

Completed in 5ms (View: 2, DB: 0) | 200 OK [http://localhost/items]
-

3. Helper methods

Rails provides various helper methods inside Active Record, Action Controller and Action View to measure the time taken by a specific code. The method is called benchmark() in all three components.

Project.benchmark("Creating project") do
-  project = Project.create("name" => "stuff")
-  project.create_manager("name" => "David")
-  project.milestones << Milestone.find(:all)
-end
-

Creating projectem (185.3ms)
-

Please refer to API docs for optional options to benchmark()

Similarly, you could use this helper method inside controllers ( Note that it's a class method here ):

def process_projects
-  self.class.benchmark("Processing projects") do
-    Project.process(params[:project_ids])
-    Project.update_cached_projects
-  end
-end
-

and views:

<% benchmark("Showing projects partial") do %>
-  <%= render :partial => @projects %>
-<% end %>
-

4. Performance Test Cases

Rails provides a very easy to write performance test cases, which look just like the regular integration tests.

If you have a look at test/performance/browsing_test.rb in a newly created Rails application:

require 'test_helper'
-require 'performance_test_help'
-
-# Profiling results for each test method are written to tmp/performance.
-class BrowsingTest < ActionController::PerformanceTest
-  def test_homepage
-    get '/'
-  end
-end
-

This is an automatically generated example performance test file, for testing performance of homepage(/) of the application.

4.1. Modes

4.1.1. Benchmarking

4.1.2. Profiling

4.2. Metrics

4.2.1. Process Time

CPU Cycles.

4.2.2. Memory

Memory taken.

4.2.3. Objects

Objects allocated.

4.2.4. GC Runs

Number of times the Ruby GC was run.

4.2.5. GC Time

Time spent running the Ruby GC.

4.3. Preparing Ruby and Ruby-prof

4.3.1. Compile

[lifo@null ~]$ mkdir rubygc
-[lifo@null ~]$ wget ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.6-p111.tar.gz
-[lifo@null ~]$ tar -xzvf ruby-1.8.6-p111.tar.gz
-[lifo@null ~]$ cd ruby-1.8.6-p111
-[lifo@null ruby-1.8.6-p111]$ curl http://rubyforge.org/tracker/download.php/1814/7062/17676/3291/ruby186gc.patch | patch -p0
-[lifo@null ruby-1.8.6-p111]$ ./configure --prefix=/Users/lifo/rubygc
-[lifo@null ruby-1.8.6-p111]$ make && make install
-

4.3.2. Prepare aliases

Add the following lines in your ~/.profile for convenience:

alias gcruby='/Users/lifo/rubygc/bin/ruby'
-alias gcrake='/Users/lifo/rubygc/bin/rake'
-alias gcgem='/Users/lifo/rubygc/bin/gem'
-alias gcirb='/Users/lifo/rubygc/bin/irb'
-alias gcrails='/Users/lifo/rubygc/bin/rails'

4.3.3. Install rubygems and some basic gems

[lifo@null ~]$ wget http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz
-[lifo@null ~]$ tar -xzvf rubygems-1.2.0.tgz
-[lifo@null ~]$ cd rubygems-1.2.0
-[lifo@null rubygems-1.2.0]$ gcruby setup.rb
-[lifo@null rubygems-1.2.0]$ cd ~
-[lifo@null ~]$ gcgem install rake
-[lifo@null ~]$ gcgem install rails

4.3.4. Install MySQL gem

[lifo@null ~]$ gcgem install mysql

If this fails, you can try to install it manually:

[lifo@null ~]$ cd /Users/lifo/rubygc/lib/ruby/gems/1.8/gems/mysql-2.7/
-[lifo@null mysql-2.7]$ gcruby extconf.rb --with-mysql-config
-[lifo@null mysql-2.7]$ make && make install

4.4. Installing Jeremy Kemper's ruby-prof

We also need to install Jeremy's ruby-prof gem using our newly built ruby:

[lifo@null ~]$ git clone git://github.com/jeremy/ruby-prof.git
-[lifo@null ~]$ cd ruby-prof/
-[lifo@null ruby-prof (master)]$ gcrake gem
-[lifo@null ruby-prof (master)]$ gcgem install pkg/ruby-prof-0.6.1.gem
-

4.5. Generating performance test

Rails provides a simple generator for creating new performance tests:

[User profiling_tester (master)]$ script/generate performance_test homepage
-

This will generate test/performance/homepage_test.rb:

require 'test_helper'
-require 'performance_test_help'
-
-class HomepageTest < ActionController::PerformanceTest
-  # Replace this with your real tests.
-  def test_homepage
-    get '/'
-  end
-end

Which you can modify to suit your needs.

4.6. Running tests

5. Understanding Performance Tests Outputs

5.1. Our First Performance Test

So how do we profile a request.

In the terminal run

[User profiling_tester]$ gcruby tests/performance/homepage.rb
-

After the tests runs for a few seconds you should see something like this.

HomepageTest#test_homepage (19 ms warmup)
-        process_time: 26 ms
-              memory: 298.79 KB
-             objects: 1917
-
-Finished in 2.207428 seconds.

Simple but efficient.

-
-Process Time refers to amount of time necessary to complete the action. -
-
-
-memory is the amount of information loaded into memory -
-
-
-object ??? #TODO find a good definition. Is it the amount of objects put into a ruby heap for this process? -
-

In addition we also gain three types of itemized log files for each of these outputs. They can be found in your tmp directory of your application.

The Three types are

-
-Flat File - A simple text file with the data laid out in a grid -
-
-
-Graphical File - A html colored coded version of the simple text file with hyperlinks between the various methods. Most useful is the bolding of the main processes for each portion of the action. -
-
-
-Tree File - A file output that can be use in conjunction with KCachegrind to visualize the process -
-

- - - -

- -	KCachegrind is Linux only. For Mac this means you have to do a full KDE install to have it working in your OS. Which is over 3 gigs in size. For windows there is clone called wincachegrind but it is no longer actively being developed.

Below are examples for Flat Files and Graphical Files

5.2. Flat Files

Example: Flat File Output Processing Time

Thread ID: 2279160 -Total: 0.026097

%self     total     self     wait    child    calls  name
- 6.41      0.06     0.04     0.00     0.02      571  Kernel#===
- 3.17      0.00     0.00     0.00     0.00      172  Hash#[]
- 2.42      0.00     0.00     0.00     0.00       13  MonitorMixin#mon_exit
- 2.05      0.00     0.00     0.00     0.00       15  Array#each
- 1.56      0.00     0.00     0.00     0.00        6  Logger#add
- 1.55      0.00     0.00     0.00     0.00       13  MonitorMixin#mon_enter
- 1.36      0.03     0.00     0.00     0.03        1  ActionController::Integration::Session#process
- 1.31      0.00     0.00     0.00     0.00       13  MonitorMixin#mon_release
- 1.15      0.00     0.00     0.00     0.00        8  MonitorMixin#synchronize-1
- 1.09      0.00     0.00     0.00     0.00       23  Class#new
- 1.03      0.01     0.00     0.00     0.01        5  MonitorMixin#synchronize
- 0.89      0.00     0.00     0.00     0.00       74  Hash#default
- 0.89      0.00     0.00     0.00     0.00        6  Hodel3000CompliantLogger#format_message
- 0.80      0.00     0.00     0.00     0.00        9  c
- 0.80      0.00     0.00     0.00     0.00       11  ActiveRecord::ConnectionAdapters::ConnectionHandler#retrieve_connection_pool
- 0.79      0.01     0.00     0.00     0.01        1  ActionController::Benchmarking#perform_action_without_rescue
- 0.18      0.00     0.00     0.00     0.00       17  <Class::Object>#allocate

So what do these columns tell us:

-
-%self - The percentage of time spent processing the method. This is derived from self_time/total_time -
-
-
-total - The time spent in this method and its children. -
-
-
-self - The time spent in this method. -
-
-
-wait - Time processed was queued -
-
-
-child - The time spent in this method's children. -
-
-
-calls - The number of times this method was called. -
-
-
-name - The name of the method. -
-

Name can be displayed three seperate ways: - #toplevel - The root method that calls all other methods - MyObject#method - Example Hash#each, The class Hash is calling the method each - * <Object:MyObject>#test - The <> characters indicate a singleton method on a singleton class. Example <Class::Object>#allocate

Methods are sorted based on %self. Hence the ones taking the most time and resources will be at the top.

Example: Flat File Memory Output

Thread ID: 2279160 -Total: 509.724609

%self     total     self     wait    child    calls  name
- 4.62     23.57    23.57     0.00     0.00       34  String#split
- 3.95     57.66    20.13     0.00    37.53        3  <Module::YAML>#quick_emit
- 2.82     23.70    14.35     0.00     9.34        2  <Module::YAML>#quick_emit-1
- 1.37     35.87     6.96     0.00    28.91        1  ActionView::Helpers::FormTagHelper#form_tag
- 1.35      7.69     6.88     0.00     0.81        1  ActionController::HttpAuthentication::Basic::ControllerMethods#authenticate_with_http_basic
- 1.06      6.09     5.42     0.00     0.67       90  String#gsub
- 1.01      5.13     5.13     0.00     0.00       27  Array#-

So for <Module::YAML>#quick_emit which is singleton method on the class YAML it uses 57.66 KB in total, 23.57 through its own actions, 6.69 from actions it calls itself and that it was called twice.

Example: Flat File Objects

Thread ID: 2279160 -Total: 6537.000000

%self     total     self     wait    child    calls  name
-15.16   1096.00   991.00     0.00   105.00       66  Hash#each
- 5.25    343.00   343.00     0.00     0.00        4  Mysql::Result#each_hash
- 4.74   2203.00   310.00     0.00  1893.00       42  Array#each
- 3.75   4529.00   245.00     0.00  4284.00        1  ActionView::Base::CompiledTemplates#_run_erb_47app47views47layouts47application46html46erb
- 2.00    136.00   131.00     0.00     5.00       90  String#gsub
- 1.73    113.00   113.00     0.00     0.00       34  String#split
- 1.44    111.00    94.00     0.00    17.00       31  Array#each-1

#TODO Find correct terminology for how to describe what this is exactly profiling as in are there really 2203 array objects or 2203 pointers to array objects?.

5.3. Graph Files

#TODO (Handily the graph file has links both between it many processes and to the files that actually contain them for debugging. - )

Example: Graph File

Thread ID: 21277412

  %total   %self     total      self    children               calls   Name
-/____________________________________________________________________________/
-100.00%   0.00%      8.77      0.00      8.77                   1     #toplevel*
-                     8.77      0.00      8.77                 1/1     Object#run_primes
-/____________________________________________________________________________/
-                     8.77      0.00      8.77                 1/1     #toplevel
-100.00%   0.00%      8.77      0.00      8.77                   1     Object#run_primes*
-                     0.02      0.00      0.02                 1/1     Object#make_random_array
-                     2.09      0.00      2.09                 1/1     Object#find_largest
-                     6.66      0.00      6.66                 1/1     Object#find_primes
-/____________________________________________________________________________/
-                     0.02      0.02      0.00                 1/1     Object#make_random_array
-0.18%     0.18%      0.02      0.02      0.00                   1     Array#each_index
-                     0.00      0.00      0.00             500/500     Kernel.rand
-                     0.00      0.00      0.00             500/501     Array#[]=
-/____________________________________________________________________________/

5.4. Tree Files

It's pointless trying to represent a tree file textually so here's a few pretty pictures of it's usefulness

KCachegrind Graph

- Graph created by KCachegrind -

KCachegrind List

- List created by KCachegrind -

#TODO Add a bit more information to this.

6. Getting to the Point of all of this

Now I know all of this is a bit dry and academic. But it's a very powerful tool when you know how to leverage it properly. Which we are going to take a look at in our next section

7. Real Life Example

7.1. The setup

class HomepageTest < ActionController::PerformanceTest
-  # Replace this with your real tests.
-  def test_homepage
-    get '/'
-  end
-end
-

Example: Output

HomepageTest#test_homepage (115 ms warmup)
-        process_time: 591 ms
-        memory: 3052.90 KB
-        objects: 59471

Obviously something is very very wrong here. 3052.90 Kb to load my minimal homepage. For Comparison for another site running well I get this for my homepage test.

Example: Default

HomepageTest#test_homepage (19 ms warmup)
-        process_time: 26 ms
-              memory: 298.79 KB
-             objects: 1917

that over a factor of ten difference. Lets look at our flat process time file to see if anything pops out at us.

Example: Process time

20.73      0.39     0.12     0.00     0.27      420  Pathname#cleanpath_aggressive
-17.07      0.14     0.10     0.00     0.04     3186  Pathname#chop_basename
- 6.47      0.06     0.04     0.00     0.02     6571  Kernel#===
- 5.04      0.06     0.03     0.00     0.03      840  Pathname#initialize
- 5.03      0.05     0.03     0.00     0.02        4  ERB::Compiler::ExplicitScanner#scan
- 4.51      0.03     0.03     0.00     0.00     9504  String#==
- 2.94      0.46     0.02     0.00     0.44     1393  String#gsub
- 2.66      0.09     0.02     0.00     0.07      480  Array#each
- 2.46      0.01     0.01     0.00     0.00     3606  Regexp#to_s

Example: Objects Created

20.74  34800.00 12324.00     0.00 22476.00      420  Pathname#cleanpath_aggressive
-16.79  18696.00  9978.00     0.00  8718.00     3186  Pathname#chop_basename
-11.47  13197.00  6813.00     0.00  6384.00      480  Array#each
- 8.51  41964.00  5059.00     0.00 36905.00     1386  String#gsub
- 6.07   3606.00  3606.00     0.00     0.00     3606  Regexp#to_s

nope nothing new here. Lets look at memory usage

Example: Memory Consuption

 40.17   1706.80  1223.70     0.00   483.10     3186  Pathname#chop_basename
- 14.92    454.47   454.47     0.00     0.00     3606  Regexp#to_s
-  7.09   2381.36   215.99     0.00  2165.37     1386  String#gsub
-  5.08    231.19   154.73     0.00    76.46      420  Pathname#prepend_prefix
-  2.34     71.35    71.35     0.00     0.00     1265  String#initialize_copy

<%if request.env["HTTP_USER_AGENT"].match(/Opera/)%>
-<%= stylesheet_link_tag "opera" %>
-<% end %>
-

That's wrong. I mistakenly am using a search function for a simple compare function. Lets fix that.

<%if request.env["HTTP_USER_AGENT"] =~ /Opera/%>
-<%= stylesheet_link_tag "opera" %>
-<% end %>
-

I'll now try my test again.

process_time: 75 ms
-              memory: 519.95 KB
-             objects: 6537

8. Get Yourself a Game Plan

8.1. The Analysis Process

I’m going to give an example methodology for conducting your benchmarking and profiling on an application. It is based on your typical scientific method.

For something as complex as Benchmarking you need to take any methodology with a grain of salt but there are some basic strictures that you can depend on.

-
-Develop a set of variables and processes to measure in order to answer this question! -
-
-
-Profile based on the question and variables. Key problems to avoid when designing this experiment are: -
-
- -
  -Confounding: Test one thing at a time, keep everything the same so you don't poison the data with uncontrolled processes. -
  -
- -
  -Cross Contamination: Make sure that runs from one test do not harm the other tests. -
  -
- -
  -Steady States: If you’re testing long running process. You must take the ramp up time and performance hit into your initial measurements. -
  -
- -
  -Sampling Error: Data should perform have a steady variance or range. If you get wild swings or sudden spikes, etc. then you must either account for the reason why or you have a sampling error. -
  -
- -
  -Measurement Error: Aka Human error, always go through your calculations at least twice to make sure there are no mathematical errors. . -
  -
-
-
-Do a small run of the experiment to verify the design. -
-
-
-Use the small run to determine a proper sample size. -
-
-
-Run the test. -
-
-
-Perform the analysis on the results and determine where to go from there. -
-

Note: Even though we are using the typical scientific method; developing a hypothesis is not always useful in terms of profiling.

9. Other Profiling Tools

There are a lot of great profiling tools out there. Some free, some not so free. This is a sort list detailing some of them.

9.1. httperf

http://www.hpl.hp.com/research/linux/httperf/

A necessary tool in your arsenal. Very useful for load testing your website.

#TODO write and link to a short article on how to use httperf. Anybody have a good tutorial availble.

9.2. Rails Analyzer

The Rails Analyzer project contains a collection of tools for Rails. It's open source and pretty speedy. It's not being actively worked on but is still contains some very useful tools.

-
-The Production Log Analyzer examines Rails log files and gives back a report. It also includes action_grep which will give you all log results for a particular action. -
-
-
-The Action Profiler similar to Ruby-Prof profiler. -
-
-
-rails_stat which gives a live counter of requests per second of a running Rails app. -
-
-
-The SQL Dependency Grapher allows you to visualize the frequency of table dependencies in a Rails application. -
-

Their project homepage can be found at http://rails-analyzer.rubyforge.org/

The one major caveat is that it needs your log to be in a different format from how rails sets it up specifically SyslogLogger.

9.2.1. SyslogLogger

SyslogLogger is a Logger work-alike that logs via syslog instead of to a file. You can add SyslogLogger to your Rails production environment to aggregate logs between multiple machines.

More information can be found out at http://rails-analyzer.rubyforge.org/hacks/classes/SyslogLogger.html

If you don't have access to your machines root system or just want something a bit easier to implement there is also a module developed by Geoffrey Grosenbach

9.2.2. A Hodel 3000 Compliant Logger for the Rest of Us

Directions taken from -link to module file

Just put the module in your lib directory and add this to your environment.rb in it's config portion.

require 'hodel_3000_compliant_logger'
-config.logger = Hodel3000CompliantLogger.new(config.log_path)

It's that simple. Your log output on restart should look like this.

Example: Hodel 3000 Example

Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
-Parameters: {"action"=>"shipping", "controller"=>"checkout"}
-Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
-[4;36;1mBook Columns (0.003155)[0m   [0;1mSHOW FIELDS FROM `books`[0m
-Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
-[4;35;1mBook Load (0.000881)[0m   [0mSELECT * FROM `books` WHERE (`books`.`id` = 1 AND (`books`.`sold` = 1)) [0m
-Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
-[4;36;1mShippingAddress Columns (0.002683)[0m   [0;1mSHOW FIELDS FROM `shipping_addresses`[0m
-Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
-[4;35;1mBook Load (0.000362)[0m   [0mSELECT ounces FROM `books` WHERE (`books`.`id` = 1) [0m
-Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
-Rendering template within layouts/application
-Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
-Rendering checkout/shipping
-Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
-[4;36;1mBook Load (0.000548)[0m   [0;1mSELECT * FROM `books`
-WHERE (sold = 0) LIMIT 3[0m
-Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]: 
-[4;35;1mAuthor Columns (0.002571)[0m   [0mSHOW FIELDS FROM `authors`[0m
-Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
-[4;36;1mAuthor Load (0.000811)[0m   [0;1mSELECT * FROM `authors` WHERE (`authors`.`id` = 1) [0m
-Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
-Rendered store/_new_books (0.01358)
-Jul 15 11:45:43 matthew-bergmans-macbook-pro-15 rails[16207]:
-Completed in 0.37297 (2 reqs/sec) | Rendering: 0.02971 (7%) | DB: 0.01697 (4%) | 200 OK [https://secure.jeffbooks/checkout/shipping]

9.3. Palmist

An open source mysql query analyzer. Full featured and easy to work with. Also requires Hodel 3000 -http://www.flyingmachinestudios.com/projects/

9.4. New Relic

http://www.newrelic.com/

Pretty nifty performance tools, pricey though. They do have a basic free -service both for when in development and when you put your application into production. Very simple installation and signup.

#TODO more in-depth without being like an advertisement.

9.4.1. Manage

Like new relic a production monitoring tool.

10. Changelog

Lighthouse ticket

-
-October 17, 2008: First revision by Pratik -
-
-
-September 6, 2008: Initial version by Matthew Bergman <MzbPhoto@gmail.com> -
-

- -

- - diff --git a/railties/doc/guides/html/creating_plugins.html b/railties/doc/guides/html/creating_plugins.html new file mode 100644 index 0000000000..7c0419659d --- /dev/null +++ b/railties/doc/guides/html/creating_plugins.html @@ -0,0 +1,1402 @@ + + + + + The Basics of Creating Rails Plugins + + + + + + + + + +

+ + + +

The Basics of Creating Rails Plugins

Pretend for a moment that you are an avid bird watcher. Your favorite bird is the Yaffle, and you want to create a plugin that allows other developers to share in the Yaffle goodness.

In this tutorial you will learn how to create a plugin that includes:

+
+Core Extensions - extending String with a to_squawk method: +
+
+
+
```
# Anywhere
+"hello!".to_squawk # => "squawk! hello!"
+
```
+

+An acts_as_yaffle method for ActiveRecord models that adds a squawk method: +

class Hickwall < ActiveRecord::Base
+  acts_as_yaffle :yaffle_text_field => :last_sang_at
+end
+
+Hickwall.new.squawk("Hello World")
+

+
+A view helper that will print out squawking info: +
+
+
+
```
squawk_info_for(@hickwall)
+
```
+
+
+A generator that creates a migration to add squawk columns to a model: +
+
+
+
```
script/generate yaffle hickwall
```
+
+

+A custom generator command: +

class YaffleGenerator < Rails::Generator::NamedBase
+  def manifest
+    m.yaffle_definition
+  end
+end
+

+A custom route method: +

ActionController::Routing::Routes.draw do |map|
+  map.yaffles
+end
+

In addition you'll learn how to:

+
+test your plugins. +
+
+
+work with init.rb, how to store model, views, controllers, helpers and even other plugins in your plugins. +
+
+
+create documentation for your plugin. +
+
+
+write custom Rake tasks in your plugin. +
+

1. Preparation

1.1. Create the basic app

In this tutorial we will create a basic rails application with 1 resource: bird. Start out by building the basic rails app:

rails plugin_demo
+cd plugin_demo
+script/generate scaffold bird name:string
+rake db:migrate
+script/server

Then navigate to http://localhost:3000/birds. Make sure you have a functioning rails app before continuing.

+ + + +

+ +	The aforementioned instructions will work for sqlite3. For more detailed instructions on how to create a rails app for other databases see the API docs.

1.2. Create the plugin

The built-in Rails plugin generator stubs out a new plugin. Pass the plugin name, either CamelCased or under_scored, as an argument. Pass --with-generator to add an example generator also.

This creates a plugin in vendor/plugins including an init.rb and README as well as standard lib, task, and test directories.

Examples:

./script/generate plugin BrowserFilters
+./script/generate plugin BrowserFilters --with-generator

Later in the plugin we will create a generator, so go ahead and add the --with-generator option now:

script/generate plugin yaffle --with-generator

You should see the following output:

create  vendor/plugins/yaffle/lib
+create  vendor/plugins/yaffle/tasks
+create  vendor/plugins/yaffle/test
+create  vendor/plugins/yaffle/README
+create  vendor/plugins/yaffle/MIT-LICENSE
+create  vendor/plugins/yaffle/Rakefile
+create  vendor/plugins/yaffle/init.rb
+create  vendor/plugins/yaffle/install.rb
+create  vendor/plugins/yaffle/uninstall.rb
+create  vendor/plugins/yaffle/lib/yaffle.rb
+create  vendor/plugins/yaffle/tasks/yaffle_tasks.rake
+create  vendor/plugins/yaffle/test/core_ext_test.rb
+create  vendor/plugins/yaffle/generators
+create  vendor/plugins/yaffle/generators/yaffle
+create  vendor/plugins/yaffle/generators/yaffle/templates
+create  vendor/plugins/yaffle/generators/yaffle/yaffle_generator.rb
+create  vendor/plugins/yaffle/generators/yaffle/USAGE

For this plugin you won't need the file vendor/plugins/yaffle/lib/yaffle.rb so you can delete that.

rm vendor/plugins/yaffle/lib/yaffle.rb

+ + + +

Editor's note:

Many plugin authors prefer to keep this file, and add all of the require statements in it. That way, they only line in init.rb would be require "yaffle". If you are developing a plugin that has a lot of files in the lib directory, you may want to create a subdirectory like lib/yaffle and store your files in there. That way your init.rb file stays clean

1.3. Setup the plugin for testing

Testing plugins that use the entire Rails stack can be complex, and the generator doesn't offer any help. In this tutorial you will learn how to test your plugin against multiple different adapters using ActiveRecord. This tutorial will not cover how to use fixtures in plugin tests.

To setup your plugin to allow for easy testing you'll need to add 3 files:

+
+A database.yml file with all of your connection strings. +
+
+
+A schema.rb file with your table definitions. +
+
+
+A test helper that sets up the database before your tests. +
+

For this plugin you'll need 2 tables/models, Hickwalls and Wickwalls, so add the following files:

vendor/plugins/yaffle/test/database.yml:

sqlite:
+  :adapter: sqlite
+  :dbfile: yaffle_plugin.sqlite.db
+
+sqlite3:
+  :adapter: sqlite3
+  :dbfile: yaffle_plugin.sqlite3.db
+
+postgresql:
+  :adapter: postgresql
+  :username: postgres
+  :password: postgres
+  :database: yaffle_plugin_test
+  :min_messages: ERROR
+
+mysql:
+  :adapter: mysql
+  :host: localhost
+  :username: rails
+  :password:
+  :database: yaffle_plugin_test

vendor/plugins/yaffle/test/test_helper.rb:

ActiveRecord::Schema.define(:version => 0) do
+  create_table :hickwalls, :force => true do |t|
+    t.string :name
+    t.string :last_squawk
+    t.datetime :last_squawked_at
+  end
+  create_table :wickwalls, :force => true do |t|
+    t.string :name
+    t.string :last_tweet
+    t.datetime :last_tweeted_at
+  end
+end
+
+# File: vendor/plugins/yaffle/test/test_helper.rb
+
+ENV['RAILS_ENV'] = 'test'
+ENV['RAILS_ROOT'] ||= File.dirname(__FILE__) + '/../../../..'
+
+require 'test/unit'
+require File.expand_path(File.join(ENV['RAILS_ROOT'], 'config/environment.rb'))
+
+config = YAML::load(IO.read(File.dirname(__FILE__) + '/database.yml'))
+ActiveRecord::Base.logger = Logger.new(File.dirname(__FILE__) + "/debug.log")
+
+db_adapter = ENV['DB']
+
+# no db passed, try one of these fine config-free DBs before bombing.
+db_adapter ||=
+  begin
+    require 'rubygems'
+    require 'sqlite'
+    'sqlite'
+  rescue MissingSourceFile
+    begin
+      require 'sqlite3'
+      'sqlite3'
+    rescue MissingSourceFile
+    end
+  end
+
+if db_adapter.nil?
+  raise "No DB Adapter selected. Pass the DB= option to pick one, or install Sqlite or Sqlite3."
+end
+
+ActiveRecord::Base.establish_connection(config[db_adapter])
+
+load(File.dirname(__FILE__) + "/schema.rb")
+
+require File.dirname(__FILE__) + '/../init.rb'
+
+class Hickwall < ActiveRecord::Base
+  acts_as_yaffle
+end
+
+class Wickwall < ActiveRecord::Base
+  acts_as_yaffle :yaffle_text_field => :last_tweet, :yaffle_date_field => :last_tweeted_at
+end
+

2. Add a `to_squawk` method to String

To update a core class you will have to:

+
+Write tests for the desired functionality. +
+
+
+Create a file for the code you wish to use. +
+
+
+Require that file from your init.rb. +
+

Most plugins store their code classes in the plugin's lib directory. When you add a file to the lib directory, you must also require that file from init.rb. The file you are going to add for this tutorial is lib/core_ext.rb.

First, you need to write the tests. Testing plugins is very similar to testing rails apps. The generated test file should look something like this:

# File: vendor/plugins/yaffle/test/core_ext_test.rb
+
+require 'test/unit'
+
+class CoreExtTest < Test::Unit::TestCase
+  # Replace this with your real tests.
+  def test_this_plugin
+    flunk
+  end
+end
+

Start off by removing the default test, and adding a require statement for your test helper.

# File: vendor/plugins/yaffle/test/core_ext_test.rb
+
+require 'test/unit'
+require File.dirname(__FILE__) + '/test_helper.rb'
+
+class CoreExtTest < Test::Unit::TestCase
+end
+

Navigate to your plugin directory and run rake test:

cd vendor/plugins/yaffle
+rake test

Your test should fail with no such file to load — ./test/../lib/core_ext.rb (LoadError) because we haven't created any file yet. Create the file lib/core_ext.rb and re-run the tests. You should see a different error message:

1.) Failure ...
+No tests were specified

Great - now you are ready to start development. The first thing we'll do is to add a method to String called to_squawk which will prefix the string with the word “squawk!”. The test will look something like this:

# File: vendor/plugins/yaffle/init.rb
+
+class CoreExtTest < Test::Unit::TestCase
+  def test_string_should_respond_to_squawk
+    assert_equal true, "".respond_to?(:to_squawk)
+  end
+
+  def test_string_prepend_empty_strings_with_the_word_squawk
+    assert_equal "squawk!", "".to_squawk
+  end
+
+  def test_string_prepend_non_empty_strings_with_the_word_squawk
+    assert_equal "squawk! Hello World", "Hello World".to_squawk
+  end
+end
+

# File: vendor/plugins/yaffle/init.rb
+
+require "core_ext"
+

# File: vendor/plugins/yaffle/lib/core_ext.rb
+
+String.class_eval do
+  def to_squawk
+    "squawk! #{self}".strip
+  end
+end
+

When monkey-patching existing classes it's often better to use class_eval instead of opening the class directly.

To test that your method does what it says it does, run the unit tests. To test this manually, fire up a console and start squawking:

$ ./script/console
+>> "Hello World".to_squawk
+=> "squawk! Hello World"

If that worked, congratulations! You just created your first test-driven plugin that extends a core ruby class.

3. Add an `acts_as_yaffle` method to ActiveRecord

A common pattern in plugins is to add a method called acts_as_something to models. In this case, you want to write a method called acts_as_yaffle that adds a squawk method to your models.

To keep things clean, create a new test file called acts_as_yaffle_test.rb in your plugin's test directory and require your test helper.

# File: vendor/plugins/yaffle/test/acts_as_yaffle_test.rb
+
+require File.dirname(__FILE__) + '/test_helper.rb'
+
+class Hickwall < ActiveRecord::Base
+  acts_as_yaffle
+end
+
+class ActsAsYaffleTest < Test::Unit::TestCase
+end
+

# File: vendor/plugins/lib/acts_as_yaffle.rb
+
+module Yaffle
+end
+

One of the most common plugin patterns for acts_as_yaffle plugins is to structure your file like so:

module Yaffle
+  def self.included(base)
+    base.send :extend, ClassMethods
+  end
+
+  module ClassMethods
+    # any method placed here will apply to classes, like Hickwall
+    def acts_as_something
+      send :include, InstanceMethods
+    end
+  end
+
+  module InstanceMethods
+    # any method placed here will apply to instaces, like @hickwall
+  end
+end
+

With structure you can easily separate the methods that will be used for the class (like Hickwall.some_method) and the instance (like @hickwell.some_method).

Let's add class method named acts_as_yaffle - testing it out first. You already defined the ActiveRecord models in your test helper, so if you run tests now they will fail.

Back in your acts_as_yaffle file, update ClassMethods like so:

module ClassMethods
+  def acts_as_yaffle(options = {})
+    send :include, InstanceMethods
+  end
+end
+

Now that test should pass. Since your plugin is going to work with field names, you need to allow people to define the field names, in case there is a naming conflict. You can write a few simple tests for this:

# File: vendor/plugins/yaffle/test/acts_as_yaffle_test.rb
+
+require File.dirname(__FILE__) + '/test_helper.rb'
+
+class ActsAsYaffleTest < Test::Unit::TestCase
+  def test_a_hickwalls_yaffle_text_field_should_be_last_squawk
+    assert_equal "last_squawk", Hickwall.yaffle_text_field
+  end
+
+  def test_a_hickwalls_yaffle_date_field_should_be_last_squawked_at
+    assert_equal "last_squawked_at", Hickwall.yaffle_date_field
+  end
+
+  def test_a_wickwalls_yaffle_text_field_should_be_last_tweet
+    assert_equal "last_tweet", Wickwall.yaffle_text_field
+  end
+
+  def test_a_wickwalls_yaffle_date_field_should_be_last_tweeted_at
+    assert_equal "last_tweeted_at", Wickwall.yaffle_date_field
+  end
+end
+

To make these tests pass, you could modify your acts_as_yaffle file like so:

# File: vendor/plugins/yaffle/lib/acts_as_yaffle.rb
+
+module Yaffle
+  def self.included(base)
+    base.send :extend, ClassMethods
+  end
+
+  module ClassMethods
+    def acts_as_yaffle(options = {})
+      cattr_accessor :yaffle_text_field, :yaffle_date_field
+      self.yaffle_text_field = (options[:yaffle_text_field] || :last_squawk).to_s
+      self.yaffle_date_field = (options[:yaffle_date_field] || :last_squawked_at).to_s
+      send :include, InstanceMethods
+    end
+  end
+
+  module InstanceMethods
+  end
+end
+

Now you can add tests for the instance methods, and the instance method itself:

# File: vendor/plugins/yaffle/test/acts_as_yaffle_test.rb
+
+require File.dirname(__FILE__) + '/test_helper.rb'
+
+class ActsAsYaffleTest < Test::Unit::TestCase
+
+  def test_a_hickwalls_yaffle_text_field_should_be_last_squawk
+    assert_equal "last_squawk", Hickwall.yaffle_text_field
+  end
+  def test_a_hickwalls_yaffle_date_field_should_be_last_squawked_at
+    assert_equal "last_squawked_at", Hickwall.yaffle_date_field
+  end
+
+  def test_a_wickwalls_yaffle_text_field_should_be_last_squawk
+    assert_equal "last_tweet", Wickwall.yaffle_text_field
+  end
+  def test_a_wickwalls_yaffle_date_field_should_be_last_squawked_at
+    assert_equal "last_tweeted_at", Wickwall.yaffle_date_field
+  end
+
+  def test_hickwalls_squawk_should_populate_last_squawk
+    hickwall = Hickwall.new
+    hickwall.squawk("Hello World")
+    assert_equal "squawk! Hello World", hickwall.last_squawk
+  end
+  def test_hickwalls_squawk_should_populate_last_squawked_at
+    hickwall = Hickwall.new
+    hickwall.squawk("Hello World")
+    assert_equal Date.today, hickwall.last_squawked_at
+  end
+
+  def test_wickwalls_squawk_should_populate_last_tweet
+    wickwall = Wickwall.new
+    wickwall.squawk("Hello World")
+    assert_equal "squawk! Hello World", wickwall.last_tweet
+  end
+  def test_wickwalls_squawk_should_populate_last_tweeted_at
+    wickwall = Wickwall.new
+    wickwall.squawk("Hello World")
+    assert_equal Date.today, wickwall.last_tweeted_at
+  end
+end
+

# File: vendor/plugins/yaffle/lib/acts_as_yaffle.rb
+
+module Yaffle
+  def self.included(base)
+    base.send :extend, ClassMethods
+  end
+
+  module ClassMethods
+    def acts_as_yaffle(options = {})
+      cattr_accessor :yaffle_text_field, :yaffle_date_field
+      self.yaffle_text_field = (options[:yaffle_text_field] || :last_squawk).to_s
+      self.yaffle_date_field = (options[:yaffle_date_field] || :last_squawked_at).to_s
+      send :include, InstanceMethods
+    end
+  end
+
+  module InstanceMethods
+    def squawk(string)
+      write_attribute(self.class.yaffle_text_field, string.to_squawk)
+      write_attribute(self.class.yaffle_date_field, Date.today)
+    end
+  end
+end
+

Note the use of write_attribute to write to the field in model.

4. Create a `squawk_info_for` view helper

Creating a view helper is a 3-step process:

+
+Add an appropriately named file to the lib directory. +
+
+
+Require the file and hooks in init.rb. +
+
+
+Write the tests. +
+

First, create the test to define the functionality you want:

# File: vendor/plugins/yaffle/test/view_helpers_test.rb
+
+require File.dirname(__FILE__) + '/test_helper.rb'
+include YaffleViewHelper
+
+class ViewHelpersTest < Test::Unit::TestCase
+  def test_squawk_info_for_should_return_the_text_and_date
+    time = Time.now
+    hickwall = Hickwall.new
+    hickwall.last_squawk = "Hello World"
+    hickwall.last_squawked_at = time
+    assert_equal "Hello World, #{time.to_s}", squawk_info_for(hickwall)
+  end
+end
+

Then add the following statements to init.rb:

# File: vendor/plugins/yaffle/init.rb
+
+require "view_helpers"
+ActionView::Base.send :include, YaffleViewHelper
+

Then add the view helpers file and

# File: vendor/plugins/yaffle/lib/view_helpers.rb
+
+module YaffleViewHelper
+  def squawk_info_for(yaffle)
+    returning "" do |result|
+      result << yaffle.read_attribute(yaffle.class.yaffle_text_field)
+      result << ", "
+      result << yaffle.read_attribute(yaffle.class.yaffle_date_field).to_s
+    end
+  end
+end
+

You can also test this in script/console by using the helper method:

$ ./script/console
+>> helper.squawk_info_for(@some_yaffle_instance)

5. Create a migration generator

When you created the plugin above, you specified the —with-generator option, so you already have the generator stubs in your plugin.

We'll be relying on the built-in rails generate template for this tutorial. Going into the details of generators is beyond the scope of this tutorial.

Type:

script/generate

You should see the line:

Plugins (vendor/plugins): yaffle

When you run script/generate yaffle you should see the contents of your USAGE file. For this plugin, the USAGE file looks like this:

Description:
+    Creates a migration that adds yaffle squawk fields to the given model
+
+Example:
+    ./script/generate yaffle hickwall
+
+    This will create:
+        db/migrate/TIMESTAMP_add_yaffle_fields_to_hickwall

Now you can add code to your generator:

# File: vendor/plugins/yaffle/generators/yaffle/yaffle_generator.rb
+
+class YaffleGenerator < Rails::Generator::NamedBase
+  def manifest
+    record do |m|
+      m.migration_template 'migration:migration.rb', "db/migrate", {:assigns => yaffle_local_assigns,
+        :migration_file_name => "add_yaffle_fields_to_#{custom_file_name}"
+       }
+    end
+  end
+
+  private
+    def custom_file_name
+      custom_name = class_name.underscore.downcase
+      custom_name = custom_name.pluralize if ActiveRecord::Base.pluralize_table_names
+    end
+
+    def yaffle_local_assigns
+      returning(assigns = {}) do
+        assigns[:migration_action] = "add"
+        assigns[:class_name] = "add_yaffle_fields_to_#{custom_file_name}"
+        assigns[:table_name] = custom_file_name
+        assigns[:attributes] = [Rails::Generator::GeneratedAttribute.new("last_squawk", "string")]
+        assigns[:attributes] << Rails::Generator::GeneratedAttribute.new("last_squawked_at", "datetime")
+      end
+    end
+end
+

Note that you need to be aware of whether or not table names are pluralized.

This does a few things:

+
+Reuses the built in rails migration_template method. +
+
+
+Reuses the built-in rails migration template. +
+

When you run the generator like

script/generate yaffle bird

You will see a new file:

# File: db/migrate/20080529225649_add_yaffle_fields_to_birds.rb
+
+class AddYaffleFieldsToBirds < ActiveRecord::Migration
+  def self.up
+    add_column :birds, :last_squawk, :string
+    add_column :birds, :last_squawked_at, :datetime
+  end
+
+  def self.down
+    remove_column :birds, :last_squawked_at
+    remove_column :birds, :last_squawk
+  end
+end
+

6. Add a custom generator command

You may have noticed above that you can used one of the built-in rails migration commands m.migration_template. You can create your own commands for these, using the following steps:

+
+Add the require and hook statements to init.rb. +
+
+
+Create the commands - creating 3 sets, Create, Destroy, List. +
+
+
+Add the method to your generator. +
+

Working with the internals of generators is beyond the scope of this tutorial, but here is a basic example:

# File: vendor/plugins/yaffle/init.rb
+require "commands"
+Rails::Generator::Commands::Create.send   :include,  Yaffle::Generator::Commands::Create
+Rails::Generator::Commands::Destroy.send  :include,  Yaffle::Generator::Commands::Destroy
+Rails::Generator::Commands::List.send     :include,  Yaffle::Generator::Commands::List
+

# File: vendor/plugins/yaffle/lib/commands.rb
+
+require 'rails_generator'
+require 'rails_generator/commands'
+
+module Yaffle #:nodoc:
+  module Generator #:nodoc:
+    module Commands #:nodoc:
+      module Create
+        def yaffle_definition
+          file("definition.txt", "definition.txt")
+        end
+      end
+
+      module Destroy
+        def yaffle_definition
+          file("definition.txt", "definition.txt")
+        end
+      end
+
+      module List
+        def yaffle_definition
+          file("definition.txt", "definition.txt")
+        end
+      end
+    end
+  end
+end
+

# File: vendor/plugins/yaffle/generators/yaffle/templates/definition.txt
+
+Yaffle is a bird

# File: vendor/plugins/yaffle/generators/yaffle/yaffle_generator.rb
+
+class YaffleGenerator < Rails::Generator::NamedBase
+  def manifest
+    m.yaffle_definition
+  end
+end
+

This example just uses the built-in "file" method, but you could do anything that Ruby allows.

7. Add a Custom Route

Testing routes in plugins can be complex, especially if the controllers are also in the plugin itself. Jamis Buck showed a great example of this in http://weblog.jamisbuck.org/2006/10/26/monkey-patching-rails-extending-routes-2.

# File: vendor/plugins/yaffle/test/routing_test.rb
+
+require "#{File.dirname(__FILE__)}/test_helper"
+
+class RoutingTest < Test::Unit::TestCase
+
+  def setup
+    ActionController::Routing::Routes.draw do |map|
+      map.yaffles
+    end
+  end
+
+  def test_yaffles_route
+    assert_recognition :get, "/yaffles", :controller => "yaffles_controller", :action => "index"
+  end
+
+  private
+
+    # yes, I know about assert_recognizes, but it has proven problematic to
+    # use in these tests, since it uses RouteSet#recognize (which actually
+    # tries to instantiate the controller) and because it uses an awkward
+    # parameter order.
+    def assert_recognition(method, path, options)
+      result = ActionController::Routing::Routes.recognize_path(path, :method => method)
+      assert_equal options, result
+    end
+end
+

# File: vendor/plugins/yaffle/init.rb
+
+require "routing"
+ActionController::Routing::RouteSet::Mapper.send :include, Yaffle::Routing::MapperExtensions
+

# File: vendor/plugins/yaffle/lib/routing.rb
+
+module Yaffle #:nodoc:
+  module Routing #:nodoc:
+    module MapperExtensions
+      def yaffles
+        @set.add_route("/yaffles", {:controller => "yaffles_controller", :action => "index"})
+      end
+    end
+  end
+end
+

# File: config/routes.rb
+
+ActionController::Routing::Routes.draw do |map|
+  ...
+  map.yaffles
+end
+

You can also see if your routes work by running rake routes from your app directory.

8. Odds and ends

8.1. Work with init.rb

The plugin initializer script init.rb is invoked via eval (not require) so it has slightly different behavior.

If you reopen any classes in init.rb itself your changes will potentially be made to the wrong module. There are 2 ways around this:

The first way is to explicitly define the top-level module space for all modules and classes, like ::Hash:

# File: vendor/plugins/yaffle/init.rb
+
+class ::Hash
+  def is_a_special_hash?
+    true
+  end
+end
+

OR you can use module_eval or class_eval:

# File: vendor/plugins/yaffle/init.rb
+
+Hash.class_eval do
+  def is_a_special_hash?
+    true
+  end
+end

8.2. Generate RDoc Documentation

Once your plugin is stable, the tests pass on all database and you are ready to deploy do everyone else a favor and document it! Luckily, writing documentation for your plugin is easy.

The first step is to update the README file with detailed information about how to use your plugin. A few key things to include are:

+
+Your name. +
+
+
+How to install. +
+
+
+How to add the functionality to the app (several examples of common use cases). +
+
+
+Warning, gotchas or tips that might help save users time. +
+

Once your README is solid, go through and add rdoc comments to all of the methods that developers will use.

Before you generate your documentation, be sure to go through and add nodoc comments to those modules and methods that are not important to your users.

Once your comments are good to go, navigate to your plugin directory and run:

rake rdoc

8.3. Store models, views, helpers, and controllers in your plugins

You can easily store models, views, helpers and controllers in plugins. Just create a folder for each in the lib folder, add them to the load path and remove them from the load once path:

# File: vendor/plugins/yaffle/init.rb
+
+%w{ models controllers helpers }.each do |dir|
+  path = File.join(directory, 'lib', dir)
+  $LOAD_PATH << path
+  Dependencies.load_paths << path
+  Dependencies.load_once_paths.delete(path)
+end
+

Adding directories to the load path makes them appear just like files in the the main app directory - except that they are only loaded once, so you have to restart the web server to see the changes in the browser.

Adding directories to the load once paths allow those changes to picked up as soon as you save the file - without having to restart the web server.

8.4. Write custom Rake tasks in your plugin

When you created the plugin with the built-in rails generator, it generated a rake file for you in vendor/plugins/yaffle/tasks/yaffle.rake. Any rake task you add here will be available to the app.

Many plugin authors put all of their rake tasks into a common namespace that is the same as the plugin, like so:

# File: vendor/plugins/yaffle/tasks/yaffle.rake
+
+namespace :yaffle do
+  desc "Prints out the word 'Yaffle'"
+  task :squawk => :environment do
+    puts "squawk!"
+  end
+end
+

When you run rake -T from your plugin you will see:

yaffle:squawk             # Prints out the word 'Yaffle'

You can add as many files as you want in the tasks directory, and if they end in .rake Rails will pick them up.

8.5. Store plugins in alternate locations

You can store plugins wherever you want - you just have to add those plugins to the plugins path in environment.rb.

Since the plugin is only loaded after the plugin paths are defined, you can't redefine this in your plugins - but it may be good to now.

You can even store plugins inside of other plugins for complete plugin madness!

config.plugin_paths << File.join(RAILS_ROOT,"vendor","plugins","yaffle","lib","plugins")
+

8.6. Create your own Plugin Loaders and Plugin Locators

If the built-in plugin behavior is inadequate, you can change almost every aspect of the location and loading process. You can write your own plugin locators and plugin loaders, but that's beyond the scope of this tutorial.

8.7. Use Custom Plugin Generators

If you are an RSpec fan, you can install the rspec_plugin_generator gem, which will generate the spec folder and database for you. See http://github.com/pat-maddox/rspec-plugin-generator/tree/master.

9. Appendix

9.1. References

9.2. Final plugin directory structure

The final plugin should have a directory structure that looks something like this:

  |-- MIT-LICENSE
+  |-- README
+  |-- Rakefile
+  |-- generators
+  |   `-- yaffle
+  |       |-- USAGE
+  |       |-- templates
+  |       |   `-- definition.txt
+  |       `-- yaffle_generator.rb
+  |-- init.rb
+  |-- install.rb
+  |-- lib
+  |   |-- acts_as_yaffle.rb
+  |   |-- commands.rb
+  |   |-- core_ext.rb
+  |   |-- routing.rb
+  |   `-- view_helpers.rb
+  |-- tasks
+  |   `-- yaffle_tasks.rake
+  |-- test
+  |   |-- acts_as_yaffle_test.rb
+  |   |-- core_ext_test.rb
+  |   |-- database.yml
+  |   |-- debug.log
+  |   |-- routing_test.rb
+  |   |-- schema.rb
+  |   |-- test_helper.rb
+  |   `-- view_helpers_test.rb
+  |-- uninstall.rb
+  `-- yaffle_plugin.sqlite3.db

+ +

+ + diff --git a/railties/doc/guides/html/creating_plugins.html.html b/railties/doc/guides/html/creating_plugins.html.html deleted file mode 100644 index 7c0419659d..0000000000 --- a/railties/doc/guides/html/creating_plugins.html.html +++ /dev/null @@ -1,1402 +0,0 @@ - - - - - The Basics of Creating Rails Plugins - - - - - - - - - -

- - - -

The Basics of Creating Rails Plugins

Pretend for a moment that you are an avid bird watcher. Your favorite bird is the Yaffle, and you want to create a plugin that allows other developers to share in the Yaffle goodness.

In this tutorial you will learn how to create a plugin that includes:

-
-Core Extensions - extending String with a to_squawk method: -
-
-
-
```
# Anywhere
-"hello!".to_squawk # => "squawk! hello!"
-
```
-

-An acts_as_yaffle method for ActiveRecord models that adds a squawk method: -

class Hickwall < ActiveRecord::Base
-  acts_as_yaffle :yaffle_text_field => :last_sang_at
-end
-
-Hickwall.new.squawk("Hello World")
-

-
-A view helper that will print out squawking info: -
-
-
-
```
squawk_info_for(@hickwall)
-
```
-
-
-A generator that creates a migration to add squawk columns to a model: -
-
-
-
```
script/generate yaffle hickwall
```
-
-

-A custom generator command: -

class YaffleGenerator < Rails::Generator::NamedBase
-  def manifest
-    m.yaffle_definition
-  end
-end
-

-A custom route method: -

ActionController::Routing::Routes.draw do |map|
-  map.yaffles
-end
-

In addition you'll learn how to:

-
-test your plugins. -
-
-
-work with init.rb, how to store model, views, controllers, helpers and even other plugins in your plugins. -
-
-
-create documentation for your plugin. -
-
-
-write custom Rake tasks in your plugin. -
-

1. Preparation

1.1. Create the basic app

In this tutorial we will create a basic rails application with 1 resource: bird. Start out by building the basic rails app:

rails plugin_demo
-cd plugin_demo
-script/generate scaffold bird name:string
-rake db:migrate
-script/server

Then navigate to http://localhost:3000/birds. Make sure you have a functioning rails app before continuing.

- - - -

- -	The aforementioned instructions will work for sqlite3. For more detailed instructions on how to create a rails app for other databases see the API docs.

1.2. Create the plugin

The built-in Rails plugin generator stubs out a new plugin. Pass the plugin name, either CamelCased or under_scored, as an argument. Pass --with-generator to add an example generator also.

This creates a plugin in vendor/plugins including an init.rb and README as well as standard lib, task, and test directories.

Examples:

./script/generate plugin BrowserFilters
-./script/generate plugin BrowserFilters --with-generator

Later in the plugin we will create a generator, so go ahead and add the --with-generator option now:

script/generate plugin yaffle --with-generator

You should see the following output:

create  vendor/plugins/yaffle/lib
-create  vendor/plugins/yaffle/tasks
-create  vendor/plugins/yaffle/test
-create  vendor/plugins/yaffle/README
-create  vendor/plugins/yaffle/MIT-LICENSE
-create  vendor/plugins/yaffle/Rakefile
-create  vendor/plugins/yaffle/init.rb
-create  vendor/plugins/yaffle/install.rb
-create  vendor/plugins/yaffle/uninstall.rb
-create  vendor/plugins/yaffle/lib/yaffle.rb
-create  vendor/plugins/yaffle/tasks/yaffle_tasks.rake
-create  vendor/plugins/yaffle/test/core_ext_test.rb
-create  vendor/plugins/yaffle/generators
-create  vendor/plugins/yaffle/generators/yaffle
-create  vendor/plugins/yaffle/generators/yaffle/templates
-create  vendor/plugins/yaffle/generators/yaffle/yaffle_generator.rb
-create  vendor/plugins/yaffle/generators/yaffle/USAGE

For this plugin you won't need the file vendor/plugins/yaffle/lib/yaffle.rb so you can delete that.

rm vendor/plugins/yaffle/lib/yaffle.rb

- - - -

Editor's note:

1.3. Setup the plugin for testing

To setup your plugin to allow for easy testing you'll need to add 3 files:

-
-A database.yml file with all of your connection strings. -
-
-
-A schema.rb file with your table definitions. -
-
-
-A test helper that sets up the database before your tests. -
-

For this plugin you'll need 2 tables/models, Hickwalls and Wickwalls, so add the following files:

vendor/plugins/yaffle/test/database.yml:

sqlite:
-  :adapter: sqlite
-  :dbfile: yaffle_plugin.sqlite.db
-
-sqlite3:
-  :adapter: sqlite3
-  :dbfile: yaffle_plugin.sqlite3.db
-
-postgresql:
-  :adapter: postgresql
-  :username: postgres
-  :password: postgres
-  :database: yaffle_plugin_test
-  :min_messages: ERROR
-
-mysql:
-  :adapter: mysql
-  :host: localhost
-  :username: rails
-  :password:
-  :database: yaffle_plugin_test

vendor/plugins/yaffle/test/test_helper.rb:

ActiveRecord::Schema.define(:version => 0) do
-  create_table :hickwalls, :force => true do |t|
-    t.string :name
-    t.string :last_squawk
-    t.datetime :last_squawked_at
-  end
-  create_table :wickwalls, :force => true do |t|
-    t.string :name
-    t.string :last_tweet
-    t.datetime :last_tweeted_at
-  end
-end
-
-# File: vendor/plugins/yaffle/test/test_helper.rb
-
-ENV['RAILS_ENV'] = 'test'
-ENV['RAILS_ROOT'] ||= File.dirname(__FILE__) + '/../../../..'
-
-require 'test/unit'
-require File.expand_path(File.join(ENV['RAILS_ROOT'], 'config/environment.rb'))
-
-config = YAML::load(IO.read(File.dirname(__FILE__) + '/database.yml'))
-ActiveRecord::Base.logger = Logger.new(File.dirname(__FILE__) + "/debug.log")
-
-db_adapter = ENV['DB']
-
-# no db passed, try one of these fine config-free DBs before bombing.
-db_adapter ||=
-  begin
-    require 'rubygems'
-    require 'sqlite'
-    'sqlite'
-  rescue MissingSourceFile
-    begin
-      require 'sqlite3'
-      'sqlite3'
-    rescue MissingSourceFile
-    end
-  end
-
-if db_adapter.nil?
-  raise "No DB Adapter selected. Pass the DB= option to pick one, or install Sqlite or Sqlite3."
-end
-
-ActiveRecord::Base.establish_connection(config[db_adapter])
-
-load(File.dirname(__FILE__) + "/schema.rb")
-
-require File.dirname(__FILE__) + '/../init.rb'
-
-class Hickwall < ActiveRecord::Base
-  acts_as_yaffle
-end
-
-class Wickwall < ActiveRecord::Base
-  acts_as_yaffle :yaffle_text_field => :last_tweet, :yaffle_date_field => :last_tweeted_at
-end
-

2. Add a `to_squawk` method to String

To update a core class you will have to:

-
-Write tests for the desired functionality. -
-
-
-Create a file for the code you wish to use. -
-
-
-Require that file from your init.rb. -
-

First, you need to write the tests. Testing plugins is very similar to testing rails apps. The generated test file should look something like this:

# File: vendor/plugins/yaffle/test/core_ext_test.rb
-
-require 'test/unit'
-
-class CoreExtTest < Test::Unit::TestCase
-  # Replace this with your real tests.
-  def test_this_plugin
-    flunk
-  end
-end
-

Start off by removing the default test, and adding a require statement for your test helper.

# File: vendor/plugins/yaffle/test/core_ext_test.rb
-
-require 'test/unit'
-require File.dirname(__FILE__) + '/test_helper.rb'
-
-class CoreExtTest < Test::Unit::TestCase
-end
-

Navigate to your plugin directory and run rake test:

cd vendor/plugins/yaffle
-rake test

1.) Failure ...
-No tests were specified

# File: vendor/plugins/yaffle/init.rb
-
-class CoreExtTest < Test::Unit::TestCase
-  def test_string_should_respond_to_squawk
-    assert_equal true, "".respond_to?(:to_squawk)
-  end
-
-  def test_string_prepend_empty_strings_with_the_word_squawk
-    assert_equal "squawk!", "".to_squawk
-  end
-
-  def test_string_prepend_non_empty_strings_with_the_word_squawk
-    assert_equal "squawk! Hello World", "Hello World".to_squawk
-  end
-end
-

# File: vendor/plugins/yaffle/init.rb
-
-require "core_ext"
-

# File: vendor/plugins/yaffle/lib/core_ext.rb
-
-String.class_eval do
-  def to_squawk
-    "squawk! #{self}".strip
-  end
-end
-

When monkey-patching existing classes it's often better to use class_eval instead of opening the class directly.

To test that your method does what it says it does, run the unit tests. To test this manually, fire up a console and start squawking:

$ ./script/console
->> "Hello World".to_squawk
-=> "squawk! Hello World"

If that worked, congratulations! You just created your first test-driven plugin that extends a core ruby class.

3. Add an `acts_as_yaffle` method to ActiveRecord

A common pattern in plugins is to add a method called acts_as_something to models. In this case, you want to write a method called acts_as_yaffle that adds a squawk method to your models.

To keep things clean, create a new test file called acts_as_yaffle_test.rb in your plugin's test directory and require your test helper.

# File: vendor/plugins/yaffle/test/acts_as_yaffle_test.rb
-
-require File.dirname(__FILE__) + '/test_helper.rb'
-
-class Hickwall < ActiveRecord::Base
-  acts_as_yaffle
-end
-
-class ActsAsYaffleTest < Test::Unit::TestCase
-end
-

# File: vendor/plugins/lib/acts_as_yaffle.rb
-
-module Yaffle
-end
-

One of the most common plugin patterns for acts_as_yaffle plugins is to structure your file like so:

module Yaffle
-  def self.included(base)
-    base.send :extend, ClassMethods
-  end
-
-  module ClassMethods
-    # any method placed here will apply to classes, like Hickwall
-    def acts_as_something
-      send :include, InstanceMethods
-    end
-  end
-
-  module InstanceMethods
-    # any method placed here will apply to instaces, like @hickwall
-  end
-end
-

With structure you can easily separate the methods that will be used for the class (like Hickwall.some_method) and the instance (like @hickwell.some_method).

Let's add class method named acts_as_yaffle - testing it out first. You already defined the ActiveRecord models in your test helper, so if you run tests now they will fail.

Back in your acts_as_yaffle file, update ClassMethods like so:

module ClassMethods
-  def acts_as_yaffle(options = {})
-    send :include, InstanceMethods
-  end
-end
-

# File: vendor/plugins/yaffle/test/acts_as_yaffle_test.rb
-
-require File.dirname(__FILE__) + '/test_helper.rb'
-
-class ActsAsYaffleTest < Test::Unit::TestCase
-  def test_a_hickwalls_yaffle_text_field_should_be_last_squawk
-    assert_equal "last_squawk", Hickwall.yaffle_text_field
-  end
-
-  def test_a_hickwalls_yaffle_date_field_should_be_last_squawked_at
-    assert_equal "last_squawked_at", Hickwall.yaffle_date_field
-  end
-
-  def test_a_wickwalls_yaffle_text_field_should_be_last_tweet
-    assert_equal "last_tweet", Wickwall.yaffle_text_field
-  end
-
-  def test_a_wickwalls_yaffle_date_field_should_be_last_tweeted_at
-    assert_equal "last_tweeted_at", Wickwall.yaffle_date_field
-  end
-end
-

To make these tests pass, you could modify your acts_as_yaffle file like so:

# File: vendor/plugins/yaffle/lib/acts_as_yaffle.rb
-
-module Yaffle
-  def self.included(base)
-    base.send :extend, ClassMethods
-  end
-
-  module ClassMethods
-    def acts_as_yaffle(options = {})
-      cattr_accessor :yaffle_text_field, :yaffle_date_field
-      self.yaffle_text_field = (options[:yaffle_text_field] || :last_squawk).to_s
-      self.yaffle_date_field = (options[:yaffle_date_field] || :last_squawked_at).to_s
-      send :include, InstanceMethods
-    end
-  end
-
-  module InstanceMethods
-  end
-end
-

Now you can add tests for the instance methods, and the instance method itself:

# File: vendor/plugins/yaffle/test/acts_as_yaffle_test.rb
-
-require File.dirname(__FILE__) + '/test_helper.rb'
-
-class ActsAsYaffleTest < Test::Unit::TestCase
-
-  def test_a_hickwalls_yaffle_text_field_should_be_last_squawk
-    assert_equal "last_squawk", Hickwall.yaffle_text_field
-  end
-  def test_a_hickwalls_yaffle_date_field_should_be_last_squawked_at
-    assert_equal "last_squawked_at", Hickwall.yaffle_date_field
-  end
-
-  def test_a_wickwalls_yaffle_text_field_should_be_last_squawk
-    assert_equal "last_tweet", Wickwall.yaffle_text_field
-  end
-  def test_a_wickwalls_yaffle_date_field_should_be_last_squawked_at
-    assert_equal "last_tweeted_at", Wickwall.yaffle_date_field
-  end
-
-  def test_hickwalls_squawk_should_populate_last_squawk
-    hickwall = Hickwall.new
-    hickwall.squawk("Hello World")
-    assert_equal "squawk! Hello World", hickwall.last_squawk
-  end
-  def test_hickwalls_squawk_should_populate_last_squawked_at
-    hickwall = Hickwall.new
-    hickwall.squawk("Hello World")
-    assert_equal Date.today, hickwall.last_squawked_at
-  end
-
-  def test_wickwalls_squawk_should_populate_last_tweet
-    wickwall = Wickwall.new
-    wickwall.squawk("Hello World")
-    assert_equal "squawk! Hello World", wickwall.last_tweet
-  end
-  def test_wickwalls_squawk_should_populate_last_tweeted_at
-    wickwall = Wickwall.new
-    wickwall.squawk("Hello World")
-    assert_equal Date.today, wickwall.last_tweeted_at
-  end
-end
-

# File: vendor/plugins/yaffle/lib/acts_as_yaffle.rb
-
-module Yaffle
-  def self.included(base)
-    base.send :extend, ClassMethods
-  end
-
-  module ClassMethods
-    def acts_as_yaffle(options = {})
-      cattr_accessor :yaffle_text_field, :yaffle_date_field
-      self.yaffle_text_field = (options[:yaffle_text_field] || :last_squawk).to_s
-      self.yaffle_date_field = (options[:yaffle_date_field] || :last_squawked_at).to_s
-      send :include, InstanceMethods
-    end
-  end
-
-  module InstanceMethods
-    def squawk(string)
-      write_attribute(self.class.yaffle_text_field, string.to_squawk)
-      write_attribute(self.class.yaffle_date_field, Date.today)
-    end
-  end
-end
-

Note the use of write_attribute to write to the field in model.

4. Create a `squawk_info_for` view helper

Creating a view helper is a 3-step process:

-
-Add an appropriately named file to the lib directory. -
-
-
-Require the file and hooks in init.rb. -
-
-
-Write the tests. -
-

First, create the test to define the functionality you want:

# File: vendor/plugins/yaffle/test/view_helpers_test.rb
-
-require File.dirname(__FILE__) + '/test_helper.rb'
-include YaffleViewHelper
-
-class ViewHelpersTest < Test::Unit::TestCase
-  def test_squawk_info_for_should_return_the_text_and_date
-    time = Time.now
-    hickwall = Hickwall.new
-    hickwall.last_squawk = "Hello World"
-    hickwall.last_squawked_at = time
-    assert_equal "Hello World, #{time.to_s}", squawk_info_for(hickwall)
-  end
-end
-

Then add the following statements to init.rb:

# File: vendor/plugins/yaffle/init.rb
-
-require "view_helpers"
-ActionView::Base.send :include, YaffleViewHelper
-

Then add the view helpers file and

# File: vendor/plugins/yaffle/lib/view_helpers.rb
-
-module YaffleViewHelper
-  def squawk_info_for(yaffle)
-    returning "" do |result|
-      result << yaffle.read_attribute(yaffle.class.yaffle_text_field)
-      result << ", "
-      result << yaffle.read_attribute(yaffle.class.yaffle_date_field).to_s
-    end
-  end
-end
-

You can also test this in script/console by using the helper method:

$ ./script/console
->> helper.squawk_info_for(@some_yaffle_instance)

5. Create a migration generator

When you created the plugin above, you specified the —with-generator option, so you already have the generator stubs in your plugin.

We'll be relying on the built-in rails generate template for this tutorial. Going into the details of generators is beyond the scope of this tutorial.

Type:

script/generate

You should see the line:

Plugins (vendor/plugins): yaffle

When you run script/generate yaffle you should see the contents of your USAGE file. For this plugin, the USAGE file looks like this:

Description:
-    Creates a migration that adds yaffle squawk fields to the given model
-
-Example:
-    ./script/generate yaffle hickwall
-
-    This will create:
-        db/migrate/TIMESTAMP_add_yaffle_fields_to_hickwall

Now you can add code to your generator:

# File: vendor/plugins/yaffle/generators/yaffle/yaffle_generator.rb
-
-class YaffleGenerator < Rails::Generator::NamedBase
-  def manifest
-    record do |m|
-      m.migration_template 'migration:migration.rb', "db/migrate", {:assigns => yaffle_local_assigns,
-        :migration_file_name => "add_yaffle_fields_to_#{custom_file_name}"
-       }
-    end
-  end
-
-  private
-    def custom_file_name
-      custom_name = class_name.underscore.downcase
-      custom_name = custom_name.pluralize if ActiveRecord::Base.pluralize_table_names
-    end
-
-    def yaffle_local_assigns
-      returning(assigns = {}) do
-        assigns[:migration_action] = "add"
-        assigns[:class_name] = "add_yaffle_fields_to_#{custom_file_name}"
-        assigns[:table_name] = custom_file_name
-        assigns[:attributes] = [Rails::Generator::GeneratedAttribute.new("last_squawk", "string")]
-        assigns[:attributes] << Rails::Generator::GeneratedAttribute.new("last_squawked_at", "datetime")
-      end
-    end
-end
-

Note that you need to be aware of whether or not table names are pluralized.

This does a few things:

-
-Reuses the built in rails migration_template method. -
-
-
-Reuses the built-in rails migration template. -
-

When you run the generator like

script/generate yaffle bird

You will see a new file:

# File: db/migrate/20080529225649_add_yaffle_fields_to_birds.rb
-
-class AddYaffleFieldsToBirds < ActiveRecord::Migration
-  def self.up
-    add_column :birds, :last_squawk, :string
-    add_column :birds, :last_squawked_at, :datetime
-  end
-
-  def self.down
-    remove_column :birds, :last_squawked_at
-    remove_column :birds, :last_squawk
-  end
-end
-

6. Add a custom generator command

You may have noticed above that you can used one of the built-in rails migration commands m.migration_template. You can create your own commands for these, using the following steps:

-
-Add the require and hook statements to init.rb. -
-
-
-Create the commands - creating 3 sets, Create, Destroy, List. -
-
-
-Add the method to your generator. -
-

Working with the internals of generators is beyond the scope of this tutorial, but here is a basic example:

# File: vendor/plugins/yaffle/init.rb
-require "commands"
-Rails::Generator::Commands::Create.send   :include,  Yaffle::Generator::Commands::Create
-Rails::Generator::Commands::Destroy.send  :include,  Yaffle::Generator::Commands::Destroy
-Rails::Generator::Commands::List.send     :include,  Yaffle::Generator::Commands::List
-

# File: vendor/plugins/yaffle/lib/commands.rb
-
-require 'rails_generator'
-require 'rails_generator/commands'
-
-module Yaffle #:nodoc:
-  module Generator #:nodoc:
-    module Commands #:nodoc:
-      module Create
-        def yaffle_definition
-          file("definition.txt", "definition.txt")
-        end
-      end
-
-      module Destroy
-        def yaffle_definition
-          file("definition.txt", "definition.txt")
-        end
-      end
-
-      module List
-        def yaffle_definition
-          file("definition.txt", "definition.txt")
-        end
-      end
-    end
-  end
-end
-

# File: vendor/plugins/yaffle/generators/yaffle/templates/definition.txt
-
-Yaffle is a bird

# File: vendor/plugins/yaffle/generators/yaffle/yaffle_generator.rb
-
-class YaffleGenerator < Rails::Generator::NamedBase
-  def manifest
-    m.yaffle_definition
-  end
-end
-

This example just uses the built-in "file" method, but you could do anything that Ruby allows.

7. Add a Custom Route

# File: vendor/plugins/yaffle/test/routing_test.rb
-
-require "#{File.dirname(__FILE__)}/test_helper"
-
-class RoutingTest < Test::Unit::TestCase
-
-  def setup
-    ActionController::Routing::Routes.draw do |map|
-      map.yaffles
-    end
-  end
-
-  def test_yaffles_route
-    assert_recognition :get, "/yaffles", :controller => "yaffles_controller", :action => "index"
-  end
-
-  private
-
-    # yes, I know about assert_recognizes, but it has proven problematic to
-    # use in these tests, since it uses RouteSet#recognize (which actually
-    # tries to instantiate the controller) and because it uses an awkward
-    # parameter order.
-    def assert_recognition(method, path, options)
-      result = ActionController::Routing::Routes.recognize_path(path, :method => method)
-      assert_equal options, result
-    end
-end
-

# File: vendor/plugins/yaffle/init.rb
-
-require "routing"
-ActionController::Routing::RouteSet::Mapper.send :include, Yaffle::Routing::MapperExtensions
-

# File: vendor/plugins/yaffle/lib/routing.rb
-
-module Yaffle #:nodoc:
-  module Routing #:nodoc:
-    module MapperExtensions
-      def yaffles
-        @set.add_route("/yaffles", {:controller => "yaffles_controller", :action => "index"})
-      end
-    end
-  end
-end
-

# File: config/routes.rb
-
-ActionController::Routing::Routes.draw do |map|
-  ...
-  map.yaffles
-end
-

You can also see if your routes work by running rake routes from your app directory.

8. Odds and ends

8.1. Work with init.rb

The plugin initializer script init.rb is invoked via eval (not require) so it has slightly different behavior.

If you reopen any classes in init.rb itself your changes will potentially be made to the wrong module. There are 2 ways around this:

The first way is to explicitly define the top-level module space for all modules and classes, like ::Hash:

# File: vendor/plugins/yaffle/init.rb
-
-class ::Hash
-  def is_a_special_hash?
-    true
-  end
-end
-

OR you can use module_eval or class_eval:

# File: vendor/plugins/yaffle/init.rb
-
-Hash.class_eval do
-  def is_a_special_hash?
-    true
-  end
-end

8.2. Generate RDoc Documentation

Once your plugin is stable, the tests pass on all database and you are ready to deploy do everyone else a favor and document it! Luckily, writing documentation for your plugin is easy.

The first step is to update the README file with detailed information about how to use your plugin. A few key things to include are:

-
-Your name. -
-
-
-How to install. -
-
-
-How to add the functionality to the app (several examples of common use cases). -
-
-
-Warning, gotchas or tips that might help save users time. -
-

Once your README is solid, go through and add rdoc comments to all of the methods that developers will use.

Before you generate your documentation, be sure to go through and add nodoc comments to those modules and methods that are not important to your users.

Once your comments are good to go, navigate to your plugin directory and run:

rake rdoc

8.3. Store models, views, helpers, and controllers in your plugins

You can easily store models, views, helpers and controllers in plugins. Just create a folder for each in the lib folder, add them to the load path and remove them from the load once path:

# File: vendor/plugins/yaffle/init.rb
-
-%w{ models controllers helpers }.each do |dir|
-  path = File.join(directory, 'lib', dir)
-  $LOAD_PATH << path
-  Dependencies.load_paths << path
-  Dependencies.load_once_paths.delete(path)
-end
-

Adding directories to the load once paths allow those changes to picked up as soon as you save the file - without having to restart the web server.

8.4. Write custom Rake tasks in your plugin

When you created the plugin with the built-in rails generator, it generated a rake file for you in vendor/plugins/yaffle/tasks/yaffle.rake. Any rake task you add here will be available to the app.

Many plugin authors put all of their rake tasks into a common namespace that is the same as the plugin, like so:

# File: vendor/plugins/yaffle/tasks/yaffle.rake
-
-namespace :yaffle do
-  desc "Prints out the word 'Yaffle'"
-  task :squawk => :environment do
-    puts "squawk!"
-  end
-end
-

When you run rake -T from your plugin you will see:

yaffle:squawk             # Prints out the word 'Yaffle'

You can add as many files as you want in the tasks directory, and if they end in .rake Rails will pick them up.

8.5. Store plugins in alternate locations

You can store plugins wherever you want - you just have to add those plugins to the plugins path in environment.rb.

Since the plugin is only loaded after the plugin paths are defined, you can't redefine this in your plugins - but it may be good to now.

You can even store plugins inside of other plugins for complete plugin madness!

config.plugin_paths << File.join(RAILS_ROOT,"vendor","plugins","yaffle","lib","plugins")
-

8.6. Create your own Plugin Loaders and Plugin Locators

8.7. Use Custom Plugin Generators

9. Appendix

9.1. References

9.2. Final plugin directory structure

The final plugin should have a directory structure that looks something like this:

  |-- MIT-LICENSE
-  |-- README
-  |-- Rakefile
-  |-- generators
-  |   `-- yaffle
-  |       |-- USAGE
-  |       |-- templates
-  |       |   `-- definition.txt
-  |       `-- yaffle_generator.rb
-  |-- init.rb
-  |-- install.rb
-  |-- lib
-  |   |-- acts_as_yaffle.rb
-  |   |-- commands.rb
-  |   |-- core_ext.rb
-  |   |-- routing.rb
-  |   `-- view_helpers.rb
-  |-- tasks
-  |   `-- yaffle_tasks.rake
-  |-- test
-  |   |-- acts_as_yaffle_test.rb
-  |   |-- core_ext_test.rb
-  |   |-- database.yml
-  |   |-- debug.log
-  |   |-- routing_test.rb
-  |   |-- schema.rb
-  |   |-- test_helper.rb
-  |   `-- view_helpers_test.rb
-  |-- uninstall.rb
-  `-- yaffle_plugin.sqlite3.db

- -

- - diff --git a/railties/doc/guides/html/migrations.html b/railties/doc/guides/html/migrations.html new file mode 100644 index 0000000000..927cafda99 --- /dev/null +++ b/railties/doc/guides/html/migrations.html @@ -0,0 +1,921 @@ + + + + + Migrations + + + + + + + + + +

+ + + +

Migrations

Migrations are a convenient way for you to alter your database in a structured and organised manner. You could edit fragments of SQL by hand but you would then be responsible for telling other developers that they need to go and run it. You'd also have to keep track of which changes need to be run against the production machines next time you deploy. Active Record tracks which migrations have already been run so all you have to do is update your source and run rake db:migrate. Active Record will work out which migrations should be run.

Migrations also allow you to describe these transformations using Ruby. The great thing about this is that (like most of Active Record's functionality) it is database independent: you don't need to worry about the precise syntax of CREATE TABLE any more that you worry about variations on SELECT * (you can drop down to raw SQL for database specific features). For example you could use SQLite3 in development, but MySQL in production.

You'll learn all about migrations including:

+
+The generators you can use to create them +
+
+
+The methods Active Record provides to manipulate your database +
+
+
+The Rake tasks that manipulate them +
+
+
+How they relate to schema.rb +
+

1. Anatomy Of A Migration

Before I dive into the details of a migration, here are a few examples of the sorts of things you can do:

class CreateProducts < ActiveRecord::Migration
+  def self.up
+    create_table :products do |t|
+      t.string :name
+      t.text :description
+
+      t.timestamps
+    end
+  end
+
+  def self.down
+    drop_table :products
+  end
+end
+

This migration adds a table called products with a string column called name and a text column called description. A primary key column called id will also be added, however since this is the default we do not need to ask for this. The timestamp columns created_at and updated_at which Active Record populates automatically will also be added. Reversing this migration is as simple as dropping the table.

Migrations are not limited to changing the schema. You can also use them to fix bad data in the database or populate new fields:

class AddReceiveNewsletterToUsers < ActiveRecord::Migration
+  def self.up
+    change_table :users do |t|
+      t.boolean :receive_newsletter, :default => false
+    end
+    User.update_all ["receive_newsletter = ?", true]
+  end
+
+  def self.down
+    remove_column :users, :receive_newsletter
+  end
+end
+

This migration adds an receive_newsletter column to the users table. We want it to default to false for new users, but existing users are considered +to have already opted in, so we use the User model to set the flag to true for existing users.

+ + + +

+ +	Some caveats apply to using models in your migrations.

1.1. Migrations are classes

A migration is a subclass of ActiveRecord::Migration that implements two class methods: up (perform the required transformations) and down (revert them).

Active Record provides methods that perform common data definition tasks in a database independent way (you'll read about them in detail later):

+
+create_table +
+
+
+change_table +
+
+
+drop_table +
+
+
+add_column +
+
+
+remove_column +
+
+
+change_column +
+
+
+rename_column +
+
+
+add_index +
+
+
+remove_index +
+

If you need to perform tasks specific to your database (for example create a foreign key constraint) then the execute function allows you to execute arbitrary SQL. A migration is just a regular Ruby class so you're not limited to these functions. For example after adding a column you could +write code to set the value of that column for existing records (if necessary using your models).

On databases that support transactions with statements that change the schema (such as PostgreSQL), migrations are wrapped in a transaction. If the database does not support this (for example MySQL and SQLite) then when a migration fails the parts of it that succeeded will not be rolled back. You will have to unpick the changes that were made by hand.

1.2. What's in a name

Migrations are stored in files in db/migrate, one for each migration class. The name of the file is of the form YYYYMMDDHHMMSS_create_products.rb, that is to say a UTC timestamp identifying the migration followed by an underscore followed by the name of the migration. The migration class' name must match (the camelcased version of) the latter part of the file name. For example 20080906120000_create_products.rb should define CreateProducts and 20080906120001_add_details_to_products.rb should define AddDetailsToProducts. If you do feel the need to change the file name then you MUST update the name of the class inside or Rails will complain about a missing class.

Internally Rails only uses the migration's number (the timestamp) to identify them. Prior to Rails 2.1 the migration number started at 1 and was incremented each time a migration was generated. With multiple developers it was easy for these to clash requiring you to rollback migrations and renumber them. With Rails 2.1 this is largely avoided by using the creation time of the migration to identify them. You can revert to the old numbering scheme by setting config.active_record.timestamped_migrations to false in environment.rb.

The combination of timestamps and recording which migrations have been run allows Rails to handle common situations that occur with multiple developers.

For example Alice adds migrations 20080906120000 and 20080906123000 and Bob adds 20080906124500 and runs it. Alice finishes her changes and checks in her migrations and Bob pulls down the latest changes. Rails knows that it has not run Alice's two migrations so rake db:migrate would run them (even though Bob's migration with a later timestamp has been run), and similarly migrating down would not run their down methods.

Of course this is no substitution for communication within the team, for example if Alice's migration removed a table that Bob's migration assumed the existence of then trouble will still occur.

1.3. Changing migrations

Occasionally you will make a mistake while writing a migration. If you have already run the migration then you cannot just edit the migration and run the migration again: Rails thinks it has already run the migration and so will do nothing when you run rake db:migrate. You must rollback the migration (for example with rake db:rollback), edit your migration and then run rake db:migrate to run the corrected version.

In general editing existing migrations is not a good idea: you will be creating extra work for yourself and your co-workers and cause major headaches if the existing version of the migration has already been run on production machines. Instead you should write a new migration that performs the changes you require. Editing a freshly generated migration that has not yet been committed to source control (or more generally which has not been propagated beyond your development machine) is relatively harmless. Just use some common sense.

2. Creating A Migration

2.1. Creating a model

The model and scaffold generators will create migrations appropriate for adding a new model. This migration will already contain instructions for creating the relevant table. If you tell Rails what columns you want then statements for adding those will also be created. For example, running

ruby script/generate model Product name:string description:text will create a migration that looks like this

class CreateProducts < ActiveRecord::Migration
+  def self.up
+    create_table :products do |t|
+      t.string :name
+      t.text :description
+
+      t.timestamps
+    end
+  end
+
+  def self.down
+    drop_table :products
+  end
+end
+

You can append as many column name/type pairs as you want. By default t.timestamps (which creates the updated_at and created_at columns that +are automatically populated by Active Record) will be added for you.

2.2. Creating a standalone migration

If you are creating migrations for other purposes (for example to add a column to an existing table) then you can use the migration generator:

ruby script/generate migration AddPartNumberToProducts

This will create an empty but appropriately named migration:

class AddPartNumberToProducts < ActiveRecord::Migration
+  def self.up
+  end
+
+  def self.down
+  end
+end
+

If the migration name is of the form AddXXXToYYY or RemoveXXXFromY and is followed by a list of column names and types then a migration containing +the appropriate add and remove column statements will be created.

ruby script/generate migration AddPartNumberToProducts part_number:string

will generate

class AddPartNumberToProducts < ActiveRecord::Migration
+  def self.up
+    add_column :products, :part_number, :string
+  end
+
+  def self.down
+    remove_column :products, :part_number
+  end
+end
+

Similarly,

ruby script/generate migration RemovePartNumberFromProducts part_number:string

generates

class RemovePartNumberFromProducts < ActiveRecord::Migration
+  def self.up
+    remove_column :products, :part_number
+  end
+
+  def self.down
+    add_column :products, :part_number, :string
+  end
+end
+

You are not limited to one magically generated column, for example

ruby script/generate migration AddDetailsToProducts part_number:string price:decimal

generates

class AddDetailsToProducts < ActiveRecord::Migration
+  def self.up
+    add_column :products, :part_number, :string
+    add_column :products, :price, :decimal
+  end
+
+  def self.down
+    remove_column :products, :price
+    remove_column :products, :part_number
+  end
+end
+

As always, what has been generated for you is just a starting point. You can add or remove from it as you see fit.

3. Writing a Migration

Once you have created your migration using one of the generators it's time to get to work!

3.1. Creating a table

create_table will be one of your workhorses. A typical use would be

create_table :products do |t|
+  t.string :name
+end
+

which creates a products table with a column called name (and as discussed below, an implicit id column).

The object yielded to the block allows you create columns on the table. There are two ways of doing this. The first looks like

create_table :products do |t|
+  t.column :name, :string, :null => false
+end
+

the second form, the so called "sexy" migrations, drops the somewhat redundant column method. Instead, the string, integer etc. methods create a column of that type. Subsequent parameters are identical.

create_table :products do |t|
+  t.string :name, :null => false
+end
+

By default create_table will create a primary key called id. You can change the name of the primary key with the :primary_key option (don't forget to update the corresponding model) or if you don't want a primary key at all (for example for a HABTM join table) you can pass :id ⇒ false. If you need to pass database specific options you can place an sql fragment in the :options option. For example

create_table :products, :options => "ENGINE=BLACKHOLE" do |t|
+  t.string :name, :null => false
+end
+

Will append ENGINE=BLACKHOLE to the sql used to create the table (when using MySQL the default is "ENGINE=InnoDB").

The types Active Record supports are :primary_key, :string, :text, :integer, :float, :decimal, :datetime, :timestamp, :time, :date, :binary, :boolean.

These will be mapped onto an appropriate underlying database type, for example with MySQL :string is mapped to VARCHAR(255). You can create columns of +types not supported by Active Record when using the non sexy syntax, for example

create_table :products do |t|
+  t.column :name, 'polygon', :null => false
+end
+

This may however hinder portability to other databases.

3.2. Changing tables

create_table's close cousin is change_table. Used for changing existing tables, it is used in a similar fashion to create_table but the object yielded to the block knows more tricks. For example

change_table :products do |t|
+  t.remove :description, :name
+  t.string :part_number
+  t.index :part_number
+  t.rename :upccode, :upc_code
+end
+

removes the description column, creates a part_number column and adds an index on it. Finally it renames the upccode column. This is the same as doing

remove_column :products, :description
+remove_column :products, :name
+add_column :products, :part_number, :string
+add_index :products, :part_number
+rename_column :products, :upccode, :upc_code
+

You don't have to keep repeating the table name and it groups all the statements related to modifying one particular table. The individual transformation names are also shorter, for example remove_column becomes just remove and add_index becomes just index.

3.3. Special helpers

Active Record provides some shortcuts for common functionality. It is for example very common to add both the created_at and updated_at columns and so there is a method that does exactly that:

create_table :products do |t|
+  t.timestamps
+end
+

will create a new products table with those two columns whereas

change_table :products do |t|
+  t.timestamps
+end
+

adds those columns to an existing table.

The other helper is called references (also available as belongs_to). In its simplest form it just adds some readability

create_table :products do |t|
+  t.references :category
+end
+

will create a category_id column of the appropriate type. Note that you pass the model name, not the column name. Active Record adds the _id for you. If you have polymorphic belongs_to associations then references will add both of the columns required:

create_table :products do |t|
+  t.references :attachment, :polymorphic => {:default => 'Photo'}
+end
+

will add an attachment_id column and a string attachment_type column with a default value of Photo.

+ + + +

+ +	The `references` helper does not actually create foreign key constraints for you. You will need to use `execute` for that or a plugin that adds foreign key support.

If the helpers provided by Active Record aren't enough you can use the execute function to execute arbitrary SQL.

For more details and examples of individual methods check the API documentation, in particular the documentation for ActiveRecord::ConnectionAdapters::SchemaStatements (which provides the methods available in the up and down methods), ActiveRecord::ConnectionAdapters::TableDefinition (which provides the methods available on the object yielded by create_table) and ActiveRecord::ConnectionAdapters::Table (which provides the methods available on the object yielded by change_table).

3.4. Writing your down method

The down method of your migration should revert the transformations done by the up method. In other words the database should be unchanged if you do an up followed by a down. For example if you create a table in the up you should drop it in the down method. It is wise to do things in precisely the reverse order to in the up method. For example

class ExampleMigration < ActiveRecord::Migration
+
+  def self.up
+    create_table :products do |t|
+      t.references :category
+    end
+    #add a foreign key
+    execute "ALTER TABLE products ADD CONSTRAINT fk_products_categories FOREIGN KEY (category_id) REFERENCES categories(id)"
+
+    add_column :users, :home_page_url, :string
+
+    rename_column :users, :email, :email_address
+  end
+
+  def self.down
+    rename_column :users, :email_address, :email
+    remove_column :users, :home_page_url
+    execute "ALTER TABLE products DROP FOREIGN KEY fk_products_categories"
+    drop_table :products
+  end
+end
+

Sometimes your migration will do something which is just plain irreversible, for example it might destroy some data. In cases like those when you can't reverse the migration you can raise IrreversibleMigration from your down method. If someone tries to revert your migration an error message will be +displayed saying that it can't be done.

4. Running Migrations

Rails provides a set of rake tasks to work with migrations which boils down to running certain sets of migrations. The very first migration related rake task you use will probably be db:migrate. In its most basic form it just runs the up method for all the migrations that have not yet been run. If there are no such migrations it exits.

If you specify a target version, Active Record will run the required migrations (up or down) until it has reached the specified version. The +version is the numerical prefix on the migration's filename. For example to migrate to version 20080906120000 run

rake db:migrate VERSION=20080906120000

If this is greater than the current version (i.e. it is migrating upwards) this will run the up method on all migrations up to and including 20080906120000, if migrating downwards this will run the down method on all the migrations down to, but not including, 20080906120000.

4.1. Rolling back

A common task is to rollback the last migration, for example if you made a mistake in it and wish to correct it. Rather than tracking down the version number associated with the previous migration you can run

rake db:rollback

This will run the down method from the latest migration. If you need to undo several migrations you can provide a STEP parameter:

rake db:rollback STEP=3

will run the down method fron the last 3 migrations.

The db:migrate:redo task is a shortcut for doing a rollback and then migrating back up again. As with the db:rollback task you can use the STEP parameter if you need to go more than one version back, for example

rake db:migrate:redo STEP=3

Neither of these Rake tasks do anything you could not do with db:migrate, they are simply more convenient since you do not need to explicitly specify the version to migrate to.

Lastly, the db:reset task will drop the database, recreate it and load the current schema into it.

+ + + +

+ +	This is not the same as running all the migrations - see the section on schema.rb.

4.2. Being Specific

If you need to run a specific migration up or down the db:migrate:up and db:migrate:down tasks will do that. Just specify the appropriate version and the corresponding migration will have its up or down method invoked, for example

rake db:migrate:up VERSION=20080906120000

will run the up method from the 20080906120000 migration. These tasks check whether the migration has already run, so for example db:migrate:up VERSION=20080906120000 will do nothing if Active Record believes that 20080906120000 has already been run.

4.3. Being talkative

By default migrations tell you exactly what they're doing and how long it took. +A migration creating a table and adding an index might produce output like this

== 20080906170109 CreateProducts: migrating ===================================
+-- create_table(:products)
+   -> 0.0021s
+-- add_index(:products, :name)
+   -> 0.0026s
+== 20080906170109 CreateProducts: migrated (0.0059s) ==========================

Several methods are provided that allow you to control all this:

+
+suppress_messages suppresses any output generated by its block +
+
+
+say outputs text (the second argument controls whether it is indented or not) +
+
+
+say_with_time outputs text along with how long it took to run its block. If the block returns an integer it assumes it is the number of rows affected. +
+

For example, this migration

class CreateProducts < ActiveRecord::Migration
+  def self.up
+    suppress_messages do
+      create_table :products do |t|
+        t.string :name
+        t.text :description
+        t.timestamps
+      end
+    end
+    say "Created a table"
+    suppress_messages {add_index :products, :name}
+    say "and an index!", true
+    say_with_time 'Waiting for a while' do
+      sleep 10
+      250
+    end
+  end
+
+  def self.down
+    drop_table :products
+  end
+end
+

generates the following output

== 20080906170109 CreateProducts: migrating ===================================
+-- Created a table
+   -> and an index!
+-- Waiting for a while
+   -> 10.0001s
+   -> 250 rows
+== 20080906170109 CreateProducts: migrated (10.0097s) =========================

If you just want Active Record to shut up then running rake db:migrate VERBOSE=false will suppress any output.

5. Using Models In Your Migrations

When creating or updating data in a migration it is often tempting to use one of your models. After all they exist to provide easy access to the underlying data. This can be done but some caution should be observed.

Consider for example a migration that uses the Product model to update a row in the corresponding table. Alice later updates the Product model, adding a new column and a validation on it. Bob comes back from holiday, updates the source and runs outstanding migrations with rake db:migrate, including the one that used the Product model. When the migration runs the source is up to date and so the Product model has the validation added by Alice. The database however is still old and so does not have that column and an error ensues because that validation is on a column that does not yet exist.

Frequently I just want to update rows in the database without writing out the SQL by hand: I'm not using anything specific to the model. One pattern for this is to define a copy of the model inside the migration itself, for example:

class AddPartNumberToProducts < ActiveRecord::Migration
+  class Product < ActiveRecord::Base
+  end
+
+  def self.up
+    ...
+  end
+
+  def self.down
+    ...
+  end
+end
+

The migration has its own minimal copy of the Product model and no longer cares about the Product model defined in the application.

5.1. Dealing with changing models

For performance reasons information about the columns a model has is cached. For example if you add a column to a table and then try and use the corresponding model to insert a new row it may try and use the old column information. You can force Active Record to re-read the column information with the reset_column_information method, for example

class AddPartNumberToProducts < ActiveRecord::Migration
+  class Product < ActiveRecord::Base
+  end
+
+  def self.up
+    add_column :product, :part_number, :string
+    Product.reset_column_information
+    ...
+  end
+
+  def self.down
+    ...
+  end
+end
+

6. Schema dumping and you

6.1. What are schema files for?

Migrations, mighty as they may be, are not the authoritative source for your database schema. That role falls to either schema.rb or an SQL file which Active Record generates by examining the database. They are not designed to be edited, they just represent the current state of the database.

There is no need (and it is error prone) to deploy a new instance of an app by replaying the entire migration history. It is much simpler and faster to just load into the database a description of the current schema.

For example, this is how the test database is created: the current development database is dumped (either to schema.rb or development.sql) and then loaded into the test database.

Schema files are also useful if want a quick look at what attributes an Active Record object has. This information is not in the model's code and is frequently spread across several migrations but is all summed up in the schema file. The annotate_models plugin, which automatically adds (and updates) comments at the top of each model summarising the schema, may also be of interest.

6.2. Types of schema dumps

There are two ways to dump the schema. This is set in config/environment.rb by the config.active_record.schema_format setting, which may be either :sql or :ruby.

If :ruby is selected then the schema is stored in db/schema.rb. If you look at this file you'll find that it looks an awful lot like one very big migration:

ActiveRecord::Schema.define(:version => 20080906171750) do
+  create_table "authors", :force => true do |t|
+    t.string   "name"
+    t.datetime "created_at"
+    t.datetime "updated_at"
+  end
+
+  create_table "products", :force => true do |t|
+    t.string   "name"
+    t.text     "description"
+    t.datetime "created_at"
+    t.datetime "updated_at"
+    t.string   "part_number"
+  end
+end
+

In many ways this is exactly what it is. This file is created by inspecting the database and expressing its structure using create_table, add_index and so on. Because this is database independent it could be loaded into any database that Active Record supports. This could be very useful if you were to distribute an application that is able to run against multiple databases.

There is however a trade-off: schema.rb cannot express database specific items such as foreign key constraints, triggers or stored procedures. While in a migration you can execute custom SQL statements, the schema dumper cannot reconstitute those statements from the database. If you are using features like this then you should set the schema format to :sql.

Instead of using Active Record's schema dumper the database's structure will be dumped using a tool specific to that database (via the db:structure:dump Rake task) into db/#{RAILS_ENV}_structure.sql. For example for PostgreSQL the pg_dump utility is used and for MySQL this file will contain the output of SHOW CREATE TABLE for the various tables. Loading this schema is simply a question of executing the SQL statements contained inside.

By definition this will be a perfect copy of the database's structure but this will usually prevent loading the schema into a database other than the one used to create it.

6.3. Schema dumps and source control

Because they are the authoritative source for your database schema, it is strongly recommended that you check them into source control.

7. Active Record and Referential Integrity

The Active Record way is that intelligence belongs in your models, not in the database. As such features such as triggers or foreign key constraints, which push some of that intelligence back into the database are not heavily used.

Validations such as validates_uniqueness_of are one way in which models can enforce data integrity. The :dependent option on associations allows models to automatically destroy child objects when the parent is destroyed. Like anything which operates at the application level these cannot guarantee referential integrity and so some people augment them with foreign key constraints.

Although Active Record does not provide any tools for working directly with such features, the execute method can be used to execute arbitrary SQL. There are also a number of plugins such as redhillonrails which add foreign key support to Active Record (including support for dumping foreign keys in schema.rb).

8. Changelog

Lighthouse ticket

+
+September 14, 2008: initial version by Frederick Cheung +
+

+ +

+ + diff --git a/railties/doc/guides/html/migrations.html.html b/railties/doc/guides/html/migrations.html.html deleted file mode 100644 index 927cafda99..0000000000 --- a/railties/doc/guides/html/migrations.html.html +++ /dev/null @@ -1,921 +0,0 @@ - - - - - Migrations - - - - - - - - - -

- - - -

Migrations

You'll learn all about migrations including:

-
-The generators you can use to create them -
-
-
-The methods Active Record provides to manipulate your database -
-
-
-The Rake tasks that manipulate them -
-
-
-How they relate to schema.rb -
-

1. Anatomy Of A Migration

Before I dive into the details of a migration, here are a few examples of the sorts of things you can do:

class CreateProducts < ActiveRecord::Migration
-  def self.up
-    create_table :products do |t|
-      t.string :name
-      t.text :description
-
-      t.timestamps
-    end
-  end
-
-  def self.down
-    drop_table :products
-  end
-end
-

Migrations are not limited to changing the schema. You can also use them to fix bad data in the database or populate new fields:

class AddReceiveNewsletterToUsers < ActiveRecord::Migration
-  def self.up
-    change_table :users do |t|
-      t.boolean :receive_newsletter, :default => false
-    end
-    User.update_all ["receive_newsletter = ?", true]
-  end
-
-  def self.down
-    remove_column :users, :receive_newsletter
-  end
-end
-

This migration adds an receive_newsletter column to the users table. We want it to default to false for new users, but existing users are considered -to have already opted in, so we use the User model to set the flag to true for existing users.

- - - -

- -	Some caveats apply to using models in your migrations.

1.1. Migrations are classes

A migration is a subclass of ActiveRecord::Migration that implements two class methods: up (perform the required transformations) and down (revert them).

Active Record provides methods that perform common data definition tasks in a database independent way (you'll read about them in detail later):

-
-create_table -
-
-
-change_table -
-
-
-drop_table -
-
-
-add_column -
-
-
-remove_column -
-
-
-change_column -
-
-
-rename_column -
-
-
-add_index -
-
-
-remove_index -
-

If you need to perform tasks specific to your database (for example create a foreign key constraint) then the execute function allows you to execute arbitrary SQL. A migration is just a regular Ruby class so you're not limited to these functions. For example after adding a column you could -write code to set the value of that column for existing records (if necessary using your models).

1.2. What's in a name

The combination of timestamps and recording which migrations have been run allows Rails to handle common situations that occur with multiple developers.

Of course this is no substitution for communication within the team, for example if Alice's migration removed a table that Bob's migration assumed the existence of then trouble will still occur.

1.3. Changing migrations

2. Creating A Migration

2.1. Creating a model

ruby script/generate model Product name:string description:text will create a migration that looks like this

class CreateProducts < ActiveRecord::Migration
-  def self.up
-    create_table :products do |t|
-      t.string :name
-      t.text :description
-
-      t.timestamps
-    end
-  end
-
-  def self.down
-    drop_table :products
-  end
-end
-

You can append as many column name/type pairs as you want. By default t.timestamps (which creates the updated_at and created_at columns that -are automatically populated by Active Record) will be added for you.

2.2. Creating a standalone migration

If you are creating migrations for other purposes (for example to add a column to an existing table) then you can use the migration generator:

ruby script/generate migration AddPartNumberToProducts

This will create an empty but appropriately named migration:

class AddPartNumberToProducts < ActiveRecord::Migration
-  def self.up
-  end
-
-  def self.down
-  end
-end
-

If the migration name is of the form AddXXXToYYY or RemoveXXXFromY and is followed by a list of column names and types then a migration containing -the appropriate add and remove column statements will be created.

ruby script/generate migration AddPartNumberToProducts part_number:string

will generate

class AddPartNumberToProducts < ActiveRecord::Migration
-  def self.up
-    add_column :products, :part_number, :string
-  end
-
-  def self.down
-    remove_column :products, :part_number
-  end
-end
-

Similarly,

ruby script/generate migration RemovePartNumberFromProducts part_number:string

generates

class RemovePartNumberFromProducts < ActiveRecord::Migration
-  def self.up
-    remove_column :products, :part_number
-  end
-
-  def self.down
-    add_column :products, :part_number, :string
-  end
-end
-

You are not limited to one magically generated column, for example

ruby script/generate migration AddDetailsToProducts part_number:string price:decimal

generates

class AddDetailsToProducts < ActiveRecord::Migration
-  def self.up
-    add_column :products, :part_number, :string
-    add_column :products, :price, :decimal
-  end
-
-  def self.down
-    remove_column :products, :price
-    remove_column :products, :part_number
-  end
-end
-

As always, what has been generated for you is just a starting point. You can add or remove from it as you see fit.

3. Writing a Migration

Once you have created your migration using one of the generators it's time to get to work!

3.1. Creating a table

create_table will be one of your workhorses. A typical use would be

create_table :products do |t|
-  t.string :name
-end
-

which creates a products table with a column called name (and as discussed below, an implicit id column).

The object yielded to the block allows you create columns on the table. There are two ways of doing this. The first looks like

create_table :products do |t|
-  t.column :name, :string, :null => false
-end
-

create_table :products do |t|
-  t.string :name, :null => false
-end
-

create_table :products, :options => "ENGINE=BLACKHOLE" do |t|
-  t.string :name, :null => false
-end
-

Will append ENGINE=BLACKHOLE to the sql used to create the table (when using MySQL the default is "ENGINE=InnoDB").

The types Active Record supports are :primary_key, :string, :text, :integer, :float, :decimal, :datetime, :timestamp, :time, :date, :binary, :boolean.

These will be mapped onto an appropriate underlying database type, for example with MySQL :string is mapped to VARCHAR(255). You can create columns of -types not supported by Active Record when using the non sexy syntax, for example

create_table :products do |t|
-  t.column :name, 'polygon', :null => false
-end
-

This may however hinder portability to other databases.

3.2. Changing tables

change_table :products do |t|
-  t.remove :description, :name
-  t.string :part_number
-  t.index :part_number
-  t.rename :upccode, :upc_code
-end
-

removes the description column, creates a part_number column and adds an index on it. Finally it renames the upccode column. This is the same as doing

remove_column :products, :description
-remove_column :products, :name
-add_column :products, :part_number, :string
-add_index :products, :part_number
-rename_column :products, :upccode, :upc_code
-

3.3. Special helpers

Active Record provides some shortcuts for common functionality. It is for example very common to add both the created_at and updated_at columns and so there is a method that does exactly that:

create_table :products do |t|
-  t.timestamps
-end
-

will create a new products table with those two columns whereas

change_table :products do |t|
-  t.timestamps
-end
-

adds those columns to an existing table.

The other helper is called references (also available as belongs_to). In its simplest form it just adds some readability

create_table :products do |t|
-  t.references :category
-end
-

create_table :products do |t|
-  t.references :attachment, :polymorphic => {:default => 'Photo'}
-end
-

will add an attachment_id column and a string attachment_type column with a default value of Photo.

- - - -

- -	The `references` helper does not actually create foreign key constraints for you. You will need to use `execute` for that or a plugin that adds foreign key support.

If the helpers provided by Active Record aren't enough you can use the execute function to execute arbitrary SQL.

3.4. Writing your down method

class ExampleMigration < ActiveRecord::Migration
-
-  def self.up
-    create_table :products do |t|
-      t.references :category
-    end
-    #add a foreign key
-    execute "ALTER TABLE products ADD CONSTRAINT fk_products_categories FOREIGN KEY (category_id) REFERENCES categories(id)"
-
-    add_column :users, :home_page_url, :string
-
-    rename_column :users, :email, :email_address
-  end
-
-  def self.down
-    rename_column :users, :email_address, :email
-    remove_column :users, :home_page_url
-    execute "ALTER TABLE products DROP FOREIGN KEY fk_products_categories"
-    drop_table :products
-  end
-end
-

4. Running Migrations

If you specify a target version, Active Record will run the required migrations (up or down) until it has reached the specified version. The -version is the numerical prefix on the migration's filename. For example to migrate to version 20080906120000 run

rake db:migrate VERSION=20080906120000

4.1. Rolling back

rake db:rollback

This will run the down method from the latest migration. If you need to undo several migrations you can provide a STEP parameter:

rake db:rollback STEP=3

will run the down method fron the last 3 migrations.

rake db:migrate:redo STEP=3

Neither of these Rake tasks do anything you could not do with db:migrate, they are simply more convenient since you do not need to explicitly specify the version to migrate to.

Lastly, the db:reset task will drop the database, recreate it and load the current schema into it.

- - - -

- -	This is not the same as running all the migrations - see the section on schema.rb.

4.2. Being Specific

rake db:migrate:up VERSION=20080906120000

4.3. Being talkative

By default migrations tell you exactly what they're doing and how long it took. -A migration creating a table and adding an index might produce output like this

== 20080906170109 CreateProducts: migrating ===================================
--- create_table(:products)
-   -> 0.0021s
--- add_index(:products, :name)
-   -> 0.0026s
-== 20080906170109 CreateProducts: migrated (0.0059s) ==========================

Several methods are provided that allow you to control all this:

-
-suppress_messages suppresses any output generated by its block -
-
-
-say outputs text (the second argument controls whether it is indented or not) -
-
-
-say_with_time outputs text along with how long it took to run its block. If the block returns an integer it assumes it is the number of rows affected. -
-

For example, this migration

class CreateProducts < ActiveRecord::Migration
-  def self.up
-    suppress_messages do
-      create_table :products do |t|
-        t.string :name
-        t.text :description
-        t.timestamps
-      end
-    end
-    say "Created a table"
-    suppress_messages {add_index :products, :name}
-    say "and an index!", true
-    say_with_time 'Waiting for a while' do
-      sleep 10
-      250
-    end
-  end
-
-  def self.down
-    drop_table :products
-  end
-end
-

generates the following output

== 20080906170109 CreateProducts: migrating ===================================
--- Created a table
-   -> and an index!
--- Waiting for a while
-   -> 10.0001s
-   -> 250 rows
-== 20080906170109 CreateProducts: migrated (10.0097s) =========================

If you just want Active Record to shut up then running rake db:migrate VERBOSE=false will suppress any output.

5. Using Models In Your Migrations

class AddPartNumberToProducts < ActiveRecord::Migration
-  class Product < ActiveRecord::Base
-  end
-
-  def self.up
-    ...
-  end
-
-  def self.down
-    ...
-  end
-end
-

The migration has its own minimal copy of the Product model and no longer cares about the Product model defined in the application.

5.1. Dealing with changing models

class AddPartNumberToProducts < ActiveRecord::Migration
-  class Product < ActiveRecord::Base
-  end
-
-  def self.up
-    add_column :product, :part_number, :string
-    Product.reset_column_information
-    ...
-  end
-
-  def self.down
-    ...
-  end
-end
-

6. Schema dumping and you

6.1. What are schema files for?

For example, this is how the test database is created: the current development database is dumped (either to schema.rb or development.sql) and then loaded into the test database.

6.2. Types of schema dumps

There are two ways to dump the schema. This is set in config/environment.rb by the config.active_record.schema_format setting, which may be either :sql or :ruby.

If :ruby is selected then the schema is stored in db/schema.rb. If you look at this file you'll find that it looks an awful lot like one very big migration:

ActiveRecord::Schema.define(:version => 20080906171750) do
-  create_table "authors", :force => true do |t|
-    t.string   "name"
-    t.datetime "created_at"
-    t.datetime "updated_at"
-  end
-
-  create_table "products", :force => true do |t|
-    t.string   "name"
-    t.text     "description"
-    t.datetime "created_at"
-    t.datetime "updated_at"
-    t.string   "part_number"
-  end
-end
-

By definition this will be a perfect copy of the database's structure but this will usually prevent loading the schema into a database other than the one used to create it.

6.3. Schema dumps and source control

Because they are the authoritative source for your database schema, it is strongly recommended that you check them into source control.

7. Active Record and Referential Integrity

8. Changelog

Lighthouse ticket

-
-September 14, 2008: initial version by Frederick Cheung -
-

- -

- - -- cgit v1.2.3

Ruby on Rails

Sustainable productivity for web-application development

Chapters

Action Controller basics

1. What does a controller do?

2. Methods and actions

3. Parameters

3.1. Hash and array parameters

3.2. Routing parameters

4. Session

4.1. Disabling the session

4.2. Accessing the session

4.3. The flash

4.3.1. flash.now

5. Cookies

6. Filters

6.1. After filters and around filters

6.2. Other ways to use filters

7. Verification

8. The request and response objects

8.1. The request

8.1.1. path_parameters, query_parameters and request_parameters

8.2. The response

9. HTTP Basic Authentication

10. Streaming and file downloads

10.1. Sending files

10.2. RESTful downloads

11. Parameter filtering

12. Rescue

12.1. The default 500 and 404 templates

12.2. rescue_from

Ruby on Rails

Sustainable productivity for web-application development

Chapters

Action Controller basics

1. What does a controller do?

2. Methods and actions

3. Parameters

3.1. Hash and array parameters

3.2. Routing parameters

4. Session

4.1. Disabling the session

4.2. Accessing the session

4.3. The flash

4.3.1. flash.now

5. Cookies

6. Filters

6.1. After filters and around filters

6.2. Other ways to use filters

7. Verification

8. The request and response objects

8.1. The request

8.1.1. path_parameters, query_parameters and request_parameters

8.2. The response

9. HTTP Basic Authentication

10. Streaming and file downloads

10.1. Sending files

10.2. RESTful downloads

11. Parameter filtering

12. Rescue

12.1. The default 500 and 404 templates

12.2. rescue_from

Ruby on Rails

Sustainable productivity for web-application development

Chapters

Benchmarking and Profiling Rails

1. Why Benchmark and Profile ?

1.1. What is the difference between benchmarking and profiling ?

2. Using and understanding the log files

3. Helper methods

4. Performance Test Cases

4.1. Modes

4.1.1. Benchmarking

4.1.2. Profiling

4.2. Metrics

4.2.1. Process Time

4.2.2. Memory

4.2.3. Objects

4.2.4. GC Runs

4.2.5. GC Time

12.2. `rescue_from`

12.2. `rescue_from`