| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
| |
Adds support for multiple databases to `rails db:schema:cache:dump`
and `rails db:schema:cache:clear`.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since Rails 6.0 will support Ruby 2.4.1 or higher
`# frozen_string_literal: true` magic comment is enough to make string object frozen.
This magic comment is enabled by `Style/FrozenStringLiteralComment` cop.
* Exclude these files not to auto correct false positive `Regexp#freeze`
- 'actionpack/lib/action_dispatch/journey/router/utils.rb'
- 'activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb'
It has been fixed by https://github.com/rubocop-hq/rubocop/pull/6333
Once the newer version of RuboCop released and available at Code Climate these exclude entries should be removed.
* Replace `String#freeze` with `String#-@` manually if explicit frozen string objects are required
- 'actionpack/test/controller/test_case_test.rb'
- 'activemodel/test/cases/type/string_test.rb'
- 'activesupport/lib/active_support/core_ext/string/strip.rb'
- 'activesupport/test/core_ext/string_ext_test.rb'
- 'railties/test/generators/actions_test.rb'
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In Ruby 2.3 or later, `String#+@` is available and `+@` is faster than `dup`.
```ruby
# frozen_string_literal: true
require "bundler/inline"
gemfile(true) do
source "https://rubygems.org"
gem "benchmark-ips"
end
Benchmark.ips do |x|
x.report('+@') { +"" }
x.report('dup') { "".dup }
x.compare!
end
```
```
$ ruby -v benchmark.rb
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
Warming up --------------------------------------
+@ 282.289k i/100ms
dup 187.638k i/100ms
Calculating -------------------------------------
+@ 6.775M (± 3.6%) i/s - 33.875M in 5.006253s
dup 3.320M (± 2.2%) i/s - 16.700M in 5.032125s
Comparison:
+@: 6775299.3 i/s
dup: 3320400.7 i/s - 2.04x slower
```
|
| |
|
|
|
|
| |
psqlrc files can affect the execution of commands in ways that can hold
up execution by blocking or otherwise cause unexpected side effects and
should best be ignored when using psql programmatically.
|
| |
|
|
|
|
|
|
|
|
|
| |
Changes the `configs_for` method from using traditional arguments to
using kwargs. This is so I can add the `include_replicas` kwarg without
having to always include `env_name` and `spec_name` in the method call.
`include_replicas` defaults to false because everywhere internally in
Rails we don't want replicas. `configs_for` is for iterating over
configurations to create / run rake tasks, so we really don't ever need
replicas in that case.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While the three-tier config makes it easier to define databases for
multiple database applications, it quickly became clear to offer full
support for multiple databases we need to change the way the connections
hash was handled.
A three-tier config means that when Rails needed to choose a default
configuration (in the case a user doesn't ask for a specific
configuration) it wasn't clear to Rails which the default was. I
[bandaid fixed this so the rake tasks could work](#32271) but that fix
wasn't correct because it actually doubled up the configuration hashes.
Instead of attemping to manipulate the hashes @tenderlove and I decided
that it made more sense if we converted the hashes to objects so we can
easily ask those object questions. In a three tier config like this:
```
development:
primary:
database: "my_primary_db"
animals:
database; "my_animals_db"
```
We end up with an object like this:
```
@configurations=[
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbded10
@env_name="development",@spec_name="primary",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>,
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbdea90
@env_name="development",@spec_name="animals",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>
]>
```
The configurations setter takes the database configuration set by your
application and turns them into an
`ActiveRecord::DatabaseConfigurations` object that has one getter -
`@configurations` which is an array of all the database objects.
The configurations getter returns this object by default since it acts
like a hash in most of the cases we need. For example if you need to
access the default `development` database we can simply request it as we
did before:
```
ActiveRecord::Base.configurations["development"]
```
This will return primary development database configuration hash:
```
{ "database" => "my_primary_db" }
```
Internally all of Active Record has been converted to use the new
objects. I've built this to be backwards compatible but allow for
accessing the hash if needed for a deprecation period. To get the
original hash instead of the object you can either add `to_h` on the
configurations call or pass `legacy: true` to `configurations.
```
ActiveRecord::Base.configurations.to_h
=> { "development => { "database" => "my_primary_db" } }
ActiveRecord::Base.configurations(legacy: true)
=> { "development => { "database" => "my_primary_db" } }
```
The new configurations object allows us to iterate over the Active
Record configurations without losing the known environment or
specification name for that configuration. You can also select all the
configs for an env or env and spec. With this we can always ask
any object what environment it belongs to:
```
db_configs = ActiveRecord::Base.configurations.configurations_for("development")
=> #<ActiveRecord::DatabaseConfigurations:0x00007fd1acbdf800
@configurations=[
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbded10
@env_name="development",@spec_name="primary",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>,
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbdea90
@env_name="development",@spec_name="animals",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>
]>
db_config.env_name
=> "development"
db_config.spec_name
=> "primary"
db_config.config
=> { "adapter"=>"sqlite3", "database"=>"db/development.sqlite3" }
```
The configurations object is more flexible than the configurations hash
and will allow us to build on top of the connection management in order
to add support for primary/replica connections, sharding, and
constructing queries for associations that live in multiple databases.
|
| |\
| |
| | |
Prevent leaking of user's DB credentials on `rails db:create` failure
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Issue #27852 reports that when `rails db:create` fails, it causes
leaking of user's DB credentials to $stderr.
We print a DB's configuration hash in order to help users more quickly
to figure out what could be wrong with his configuration.
This commit changes message from
"Couldn't create database for #{configuration.inspect}" to
"Couldn't create '#{configuration['database']}' database. Please check your configuration.".
There are two PRs that fixing it #27878, #27879, but they need a bit more work.
I decided help to finish this and added Author of those PRs credit in this commit.
Since it is a security issue, I think we should backport it to
`5-2-stable`, and `5-1-stable`.
Guided by https://edgeguides.rubyonrails.org/maintenance_policy.html#security-issues
Fixes #27852
Closes #27879
Related to #27878
[Alexander Marrs & bogdanvlviv]
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We originally did the whole `load_database_yaml` thing because this test
wasn't cooperating and we needed to finish the namespaced rake tasks for
multiple databases.
However, it turns out that YAML can't eval ERB if you don't tell it it's
ERB so you get Pysch parse errors if you're using multi-line ERB or
ERB with conditionals. It's a hot mess.
After trying a few things and thinking it over we decided that it wasn't
worth bandaiding over, the test needed to be improved. The test was
added in #31135 to test that the env is loaded in these tasks. But it
was blowing up because we were trying to read a database name out of the
configuration - however that's not the purpose of this change. We want
to read environment files in the rake tasks, but not in the config
file.
In this PR we changed the test to test what the PR was actually fixing.
We've also deleted the `load_database_yaml` because it caused more
problems than it was worth. This should fix the issues described in
https://github.com/rails/rails/pull/32274#issuecomment-384161057. We
also had these problems at GitHub.
Co-authored-by: alimi <aibrahim2k2@gmail.com>
|
| |
|
|
|
| |
This commit follows the path we started at commit #ea4f0e2
and continued at PR #33229.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use attr_reader/attr_writer instead of methods
method is 12% slower
Use flat_map over map.flatten(1)
flatten is 66% slower
Use hash[]= instead of hash.merge! with single arguments
merge! is 166% slower
See https://github.com/rails/rails/pull/32337 for more conversation
|
| |
|
|
|
|
| |
This was causing single db applications to have rake tasks named
`db:create:primary`. These tasks are only useful to multiple database
applications so they shouldn't be generated.
|
| |
|
|
|
| |
Pass the spec name to load_schema in order to load from the correct
structure file when there are multiple databases
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Moves the configs_for and DatabaseConfig struct into it's own file. I
was considering doing this in a future refactoring but our set up forced
me to move it now. You see there are `mattr_accessor`'s on the Core
module that have default settings. For example the `schema_format`
defaults to Ruby. So if I call `configs_for` or any methods in the Core
module it will reset the `schema_format` to `:ruby`. By moving it to
it's own class we can keep the logic contained and avoid this
unfortunate issue.
The second change here does a double loop over the yaml files. Bear with
me...
Our tests dictate that we need to load an environment before our rake
tasks because we could have something in an environment that the
database.yml depends on. There are side-effects to this and I think
there's a deeper bug that needs to be fixed but that's for another
issue. The gist of the problem is when I was creating the dynamic rake
tasks if the yaml that that rake task is calling evaluates code (like
erb) that calls the environment configs the code will blow up because
the environment is not loaded yet.
To avoid this issue we added a new method that simply loads the yaml and
does not evaluate the erb or anything in it. We then use that yaml to
create the task name. Inside the task name we can then call
`load_config` and load the real config to actually call the code
internal to the task. I admit, this is gross, but refactoring can't all
be pretty all the time and I'm working hard with `@tenderlove` to
refactor much more of this code to get to a better place re connection
management and rake tasks.
|
| |
|
|
|
|
|
| |
Adds the ability to dump the schema or structure files for mulitple
databases. Loops through the configs for a given env and sets a filename
based on the format, then establishes a connection for that config and
dumps into the file.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`each_current_configuration` is used by create, drop, and other methods
to find the configs for a given environment and returning those to the
method calling them.
The change here allows for the database commands to operate on all the
configs in the environment. Previously we couldn't slice the hashes and
iterate over them becasue they could be two tier or could be three
tier. By using the database config structs we don't need to care whether
we're dealing with a three tier or two tier, we can just parse all the
configs based on the environment.
This makes it possible for us to run `bin/rails db:create` and it will
create all the configs for the dev and test environment ust like it does
for a two tier - it creates the db for dev and test. Now `db:create`
will create `primary` for dev and test, and `animals` for dev and test
if our database.yml looks like:
```
development:
primary:
etc
animals:
etc
test:
primary:
etc
animals:
etc
```
This means that `bin/rails db:create`, `bin/rails db:drop`, and
`bin/rails db:migrate` will operate on the dev and test env for both
primary and animals ds.
|
| |
|
|
|
|
| |
Follow up of b988ecb99ff6c8854e4b74ef8a7ade8d9ef5d954.
This was added for internal usage, it doesn't need to be public.
|
| |
|
|
|
|
|
|
|
|
| |
This reverts commit 16f279ebd474626577ced858e3626ac4535a33df, reversing
changes made to 6c6a30a7c357ce1eafa093d77d2b08684fe50887.
The config can be named anything, not just default (although all
generated apps will be named default). We can't just delete configs that
don't have a database because that will break three-tier configs. Oh
well.
|
| |
|
|
|
|
|
| |
Because of this default configuration we're constantly checking if the
database exists when looping through configurations. This is unnecessary
and we should just delete it before we need to loop through
configurations.
|
| |
|
|
|
| |
Followup to b988ecb, when I cherry-picked from my parallel testing
branch I didn't realize `drop` wasn't part of the commit.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
You could use the `VERBOSE` env var to turn off output for migrations
tasks but you couldn't use it for other tasks.
This change moves the `verbose?` check to a method so we can also use it
in create and drop respectively.
tenderlove and I noticed this as part of the ongoing work in parallel
testing. When the parallel tests boot the app needs to create new
databases for each worker. The output from these is unnecessary but
there was previously no way to turn it off. Now if `VERBOSE=false` is
passes to `bin/rails db:create` the text "Created blah blah db" will no
longer be output.
|
| |
|
|
| |
This was added in #31727, but it is unused.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rails has some support for multiple databases but it can be hard to
handle migrations with those. The easiest way to implement multiple
databases is to contain migrations into their own folder ("db/migrate"
for the primary db and "db/seconddb_migrate" for the second db). Without
this you would need to write code that allowed you to switch connections
in migrations. I can tell you from experience that is not a fun way to
implement multiple databases.
This refactoring is a pre-requisite for implementing other features
related to parallel testing and improved handling for multiple
databases.
The refactoring here moves the class methods from the `Migrator` class
into it's own new class `MigrationContext`. The goal was to move the
`migrations_paths` method off of the `Migrator` class and onto the
connection. This allows users to do the following in their
`database.yml`:
```
development:
adapter: mysql2
username: root
password:
development_seconddb:
adapter: mysql2
username: root
password:
migrations_paths: "db/second_db_migrate"
```
Migrations for the `seconddb` can now be store in the
`db/second_db_migrate` directory. Migrations for the primary database
are stored in `db/migrate`".
The refactoring here drastically reduces the internal API for migrations
since we don't need to pass `migrations_paths` around to every single
method. Additionally this change does not require any Rails applications
to make changes unless they want to use the new public API. All of the
class methods from the `Migrator` class were `nodoc`'d except for the
`migrations_paths` and `migrations_path` getter/setters respectively.
|
| |\
| |
| | |
Simplify implementation of `MySQLDatabaseTasks`
|
| | |
| |
| |
| |
| |
| | |
Don't process MySQL ERROR 1045, raise error instead
Make behavior of `MySQLDatabaseTasks` more consistent with behavior of `PostgreSQLDatabaseTasks`
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
`blog$ bin/rails db:create`
Before:
```
Couldn't create database for {"adapter"=>"mysql2", "encoding"=>"utf42", "pool"=>5,
"username"=>"root", "password"=>nil, "socket"=>"/var/run/mysqld/mysqld.sock",
"database"=>"blog_development"}, {:charset=>"utf42"}
(If you set the charset manually, make sure you have a matching collation)
Created database 'blog_development'
Couldn't create database for {"adapter"=>"mysql2", "encoding"=>"utf42", "pool"=>5,
"username"=>"root", "password"=>nil, "socket"=>"/var/run/mysqld/mysqld.sock",
"database"=>"blog_test"}, {:charset=>"utf42"}
(If you set the charset manually, make sure you have a matching collation)
Created database 'blog_test'
```
After:
```
Unsupported charset: '"utf42"'
Couldn't create database for {"adapter"=>"mysql2", "encoding"=>"utf42", "pool"=>5,
"username"=>"root", "password"=>nil, "socket"=>"/var/run/mysqld/mysqld.sock",
"database"=>"blog_development"}
rails aborted!
Mysql2::Error: Unsupported charset: '"utf42"'
...
(stack trace)
...
bin/rails:4:in `<main>'
Tasks: TOP => db:create
(See full trace by running task with --trace)
```
Closes #29683
Related to #27398
|
| |/
|
|
|
|
| |
Ensure that `bin/rails db:migrate` with specified `VERSION` reverts
all migrations only if `VERSION` is `0`.
Raise error if target migration doesn't exist.
|
| |
|
|
| |
This basically reverts 9d4f79d3d394edb74fa2192e5d9ad7b09ce50c6d
|
| |
|
|
|
|
|
|
|
|
|
|
| |
database connection
When load schema from `structure.sql`, database connection isn't
established. `ActiveRecord::Tasks::DatabaseTasks.load_schema` has to
establish database connection since it executes
```
ActiveRecord::InternalMetadata.create_table
ActiveRecord::InternalMetadata[:environment] = environment
```
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ar_internal_metadata's data for a test database.
Before:
```
$ RAILS_ENV=test rails dbconsole
> SELECT * FROM ar_internal_metadata;
key|value|created_at|updated_at
environment|development|2017-09-11 23:14:10.815679|2017-09-11 23:14:10.815679
```
After:
```
$ RAILS_ENV=test rails dbconsole
> SELECT * FROM ar_internal_metadata;
key|value|created_at|updated_at
environment|test|2017-09-11 23:14:10.815679|2017-09-11 23:14:10.815679
```
Fixes #26731.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Without this, `db:structure:dump` task raises an error as follwing:
```
can't modify frozen String
activerecord/lib/active_record/tasks/sqlite_database_tasks.rb:77:in `run_cmd_error'
activerecord/lib/active_record/tasks/sqlite_database_tasks.rb:72:in `run_cmd'
activerecord/lib/active_record/tasks/sqlite_database_tasks.rb:52:in `structure_dump'
activerecord/lib/active_record/tasks/database_tasks.rb:219:in `structure_dump'
activerecord/lib/active_record/railties/databases.rake:279:in `block (3 levels) in <main>'
railties/lib/rails/commands/rake/rake_command.rb:23:in `block in perform'
railties/lib/rails/commands/rake/rake_command.rb:20:in `perform'
railties/lib/rails/command.rb:48:in `invoke'
railties/lib/rails/commands.rb:18:in `<main>'
```
|
| |\
| |
| | |
Use copy to preserve file permissions
|
| | | |
|
| |/ |
|
| |
|
|
|
|
| |
This was missed when the frozen string literal pragma was added to this
file because the string is only modified when running in the context of
a full Rails app, which wasn't covered by the test suite.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
By making the Rails minitest behave like a standard minitest plugin
we're much more likely to not break when people use other minitest
plugins. Like minitest-focus and pride.
To do this, we need to behave like minitest: require files up front
and then perform the plugin behavior via the at_exit hook.
This also saves us a fair bit of wrangling with test file loading.
Finally, since the environment and warnings options have to be applied
as early as possible, and since minitest loads plugins at_exit, they
have to be moved to the test command.
* Don't expect the root method.
It's likely this worked because we eagerly loaded the Rails minitest plugin
and that somehow defined a root method on `Rails`.
* Assign a backtrace to failed exceptions.
Otherwise Minitest pukes when attempting to filter the backtrace (which
Rails' backtrace cleaner then removes).
Means the exception message test has to be revised too.
This is likely caused by the rails minitest plugin now being loaded for
these tests and assigning a default backtrace cleaner.
|
| |\ |
|
| | |
| |
| |
| |
| | |
This reverts commit 3420a14590c0e6915d8b6c242887f74adb4120f9, reversing
changes made to afb66a5a598ce4ac74ad84b125a5abf046dcf5aa.
|
| | |\
| | |
| | |
| | | |
Enforce frozen string in Rubocop
|
| | | | |
|
| | |\ \
| | |/
| |/|
| | | |
Make ActiveSupport frozen-string-literal friendly.
|
| | | | |
|
| |/ / |
|
| | | |
|
| |/
|
|
|
|
|
| |
- On Mysql, some command line options that affect option-file handling such as `--no-defaults` have to be passed before any other options
- Modified rails to pass them right after the `mysql` command
- Ref https://dev.mysql.com/doc/refman/5.7/en/option-file-options.html and https://bugs.mysql.com/bug.php?id=83386
- Ref #27437
|
