diff options
| author | Zachary Scott <e@zzak.io> | 2015-02-16 10:08:01 -0800 |
|---|---|---|
| committer | Zachary Scott <e@zzak.io> | 2015-02-16 10:08:01 -0800 |
| commit | d9f8b58d9802afe62f7bf772fc08683851d49940 (patch) | |
| tree | 15f95e7d65dc27186c1e304a0f425decfca50dd7 | |
| parent | ba8502171e4e0024f7112192c664159bfdab5b82 (diff) | |
| parent | 2a428eda471b8a11bee0cb0afeb87f8798d01e7e (diff) | |
| download | rails-d9f8b58d9802afe62f7bf772fc08683851d49940.tar.gz rails-d9f8b58d9802afe62f7bf772fc08683851d49940.tar.bz2 rails-d9f8b58d9802afe62f7bf772fc08683851d49940.zip | |
Merge pull request #18485 from andreynering/robots.txt
Adding 'Search Engines Indexing' Section [ci skip]
| -rw-r--r-- | guides/CHANGELOG.md | 4 | ||||
| -rw-r--r-- | guides/source/configuring.md | 18 |
2 files changed, 22 insertions, 0 deletions
diff --git a/guides/CHANGELOG.md b/guides/CHANGELOG.md index 99d69d5eda..dd5ca4b395 100644 --- a/guides/CHANGELOG.md +++ b/guides/CHANGELOG.md @@ -6,4 +6,8 @@ *Andrey Nering* +* New section in Configuring: Search Engines Indexing + + *Andrey Nering* + Please check [4-2-stable](https://github.com/rails/rails/blob/4-2-stable/guides/CHANGELOG.md) for previous changes. diff --git a/guides/source/configuring.md b/guides/source/configuring.md index 9c0f2ddc8a..994cc31cff 100644 --- a/guides/source/configuring.md +++ b/guides/source/configuring.md @@ -1049,3 +1049,21 @@ These configuration points are then available through the configuration object: Rails.configuration.x.super_debugger # => true Rails.configuration.x.super_debugger.not_set # => nil ``` + +Search Engines Indexing +----------------------- + +Sometimes, you may want to prevent some pages of your application be visible on search sites like Google, +Bing, Yahoo or Duck Duck Go. The robots that index these sites will first analyse the +`https://your-site.com/robots.txt` file to know what pages it is allowed to index. + +Rails creates this file for you on `/public` folder. By default, it allows search engines to index all +pages of your application. If you want to block indexing on all pages of you application, use this: + +``` +User-agent: * +Disallow: / +``` + +To block just specific pages, it's necessary to use a more complex syntax. Learn it on the +[official documentation](http://www.robotstxt.org/robotstxt.html). |