| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
| |
In most cases it works now without explicit require because it's accidentally required through
active_support/core_ext/date_and_time/calculations.rb where we still call `try`,
but that would stop working if we changed the Calculations implementation and remove the require call there.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR proposes moving the schema cache from the connection to the pool
so the connection can ask the pool for the cache. In a future PR our
goal is to be able to read the yaml file from the pool so we can get
rid of the `active_record.check_schema_cache_dump` initializer. This
will fix the issues surrounding dumping the schema cache and mulitple
databases.
Why do we want to get rid of the initializer you ask?
Well I was looking at #34449 and trying to make it work for our usecase
and it revealed A LOT of problems. There are a few issues that I will
fix in remaining PRs with SchemaMigration, but there's a big glaring
issue with this initializer.
When you have an application with multiple databases we'll need to loop
through all the configurations and set the schema cache on those
connections. The problem is on initialization we only have one
connection - the one for Ar::Base. This is fine in a single db
application but not fine in multi-db. If we follow the pattern in #34449
and establish a connection to those other dbs we will end up setting the
cache on the _connection object_ rather than on all connections that
connect for that config.
So even though we looped through the configs and assigned the cache the
cache will not be set (or will be set wrong) once the app is booted
because the connection objects after boot are _different_ than the
connection objects we assigned the cache to.
After trying many different ways to set the schema cache `@tenderlove`
and I came to the conclusion that the initializer is problematic, as is
setting the schema cache twice.
This is part 1 to move the cache to the pool so the cache can read from
the schema cache yaml file instead of setting it when initializing the
app.
To do this we have created a `NullPool` that initializes an empty cache. I
put the `get_schema_cache` and `set_schema_cache` in an `AbstractPool`
so we can share code between `ConnectionPool` and `NullPool` instead of
duplicating code.
Now we only need to set the schema_cache on the pool rather than the
connection. In `discard!` we need to unset the connection from the
schema_cache - we still want the cache just not the connection.
|
| | |
|
| |\
| |
| |
| |
| | |
eileencodes/allow-application-to-change-handler-names
Add ability to change the names of the default handlers
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
When I wrote the `connected_to` and `connects_to` API's I wrote them
with the idea in mind that it didn't really matter what the
handlers/roles were called as long as those connecting to the roles knew
which one wrote and which one read.
With the introduction of the middleware Rails begins to assume it's
`writing` and `reading` and there's no room for other roles. At GitHub
we've been using this method for a long time so we have a ton of legacy
code that uses different handler names `default` and `readonly`. We
could rename all our code but I think this is better for a few reasons:
- Legacy apps that have been using multiple databases for a long time
can have an eaiser time switching.
- If we later find this to cause more issues than it's worth we can
easily deprecate.
- We won't force old apps to rewrite the resolver middleware just to use
a different handler.
Adding the writing_role/reading_role required that I move the code that
creates the first handler for writing to the railtie. If I didn't move
this the core class would assign the handler before I was able to assign
a new one in my configuration and I'd end up with 3 handlers instead of
2.
|
| |/
|
|
|
|
|
|
|
|
| |
Right now we only have one option that's supported, the delay. However I
can see us supporting other options in the future.
This PR refactors the options to get passed into the resolver so whether
you're using middleware or using the config options you can pass options
to the resolver. This will also make it easy to add new options in the
future.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The following PR adds behavior to Rails to allow an application to
automatically switch it's connection from the primary to the replica.
A request will be sent to the replica if:
* The request is a read request (`GET` or `HEAD`)
* AND It's been 2 seconds since the last write to the database (because
we don't want to send a user to a replica if the write hasn't made it
to the replica yet)
A request will be sent to the primary if:
* It's not a GET/HEAD request (ie is a POST, PATCH, etc)
* Has been less than 2 seconds since the last write to the database
The implementation that decides when to switch reads (the 2 seconds) is
"safe" to use in production but not recommended without adequate testing
with your infrastructure. At GitHub in addition to the a 5 second delay
we have a curcuit breaker that checks the replication delay
and will send the query to a replica before the 5 seconds has passed.
This is specific to our application and therefore not something Rails
should be doing for you. You'll need to test and implement more robust
handling of when to switch based on your infrastructure. The auto
switcher in Rails is meant to be a basic implementation / API that acts
as a guide for how to implement autoswitching.
The impementation here is meant to be strict enough that you know how to
implement your own resolver and operations classes but flexible enough
that we're not telling you how to do it.
The middleware is not included automatically and can be installed in
your application with the classes you want to use for the resolver and
operations passed in. If you don't pass any classes into the middleware
the Rails default Resolver and Session classes will be used.
The Resolver decides what parameters define when to
switch, Operations sets timestamps for the Resolver to read from. For
example you may want to use cookies instead of a session so you'd
implement a Resolver::Cookies class and pass that into the middleware
via configuration options.
```
config.active_record.database_selector = { delay: 2.seconds }
config.active_record.database_resolver = MyResolver
config.active_record.database_operations = MyResolver::MyCookies
```
Your classes can inherit from the existing classes and reimplment the
methods (or implement more methods) that you need to do the switching.
You only need to implement methods that you want to change. For example
if you wanted to set the session token for the last read from a replica
you would reimplement the `read_from_replica` method in your resolver
class and implement a method that updates a new timestamp in your
operations class.
|
| | |
|
| |\
| |
| | |
Only define attribute methods from schema cache
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
To define the attribute methods for a model, Active Record needs to know
the schema of the underlying table, which is usually achieved by making
a request to the database. This is undesirable behaviour while the app
is booting, for two reasons: it makes the boot process dependent on the
availability of the database, and it means every new process will make
one query for each table, which can cause issues for large applications.
However, if the application is using the schema cache dump feature, then
the schema cache already contains the necessary information, and we can
define the attribute methods without causing any extra database queries.
|
| |/
|
|
|
| |
`establish_connection` will never raise `ActiveRecord::NoDatabaseError`,
because it doesn't connect to a database; it sets up a connection pool.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The attribute methods for a model are currently defined lazily the first
time that model is instantiated, even when `config.eager_load` is true.
This means the first request to use each model incurs the cost, which
usually involves a database round trip to fetch the schema definition.
By defining the attribute methods for all models while the application
is booting, we move that work out of any individual request. When using
a forking web server, this also reduces the number of times the schema
definition is queried by doing it once in the parent process instead of
from each forked process during their first request.
|
| |
|
|
|
| |
- Moving the `supports_cache_versioning?` check to a class method.
- Shorten the method doc.
- Expand on the error message.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
does not support it
If you are using the "in cache versioning" also known as "recyclable cache keys" the cache store must be aware of this scheme, otherwise you will generate cache entries that never invalidate.
This PR adds a check to the initialization process to ensure that if recyclable cache keys are being used via
```
config.active_record.cache_versioning = true
```
Then the cache store needs to show that it supports this versioning scheme. Cache stores can let Rails know that they support this scheme by adding a method `supports_in_cache_versioning?` and returning true.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In a test app we observed that the query cache was not enabled on the
first request. This was because the query cache hooks are installed on
load and active record is loaded in the middle of the first request.
If we remove the `on_load` from the railtie the query cache hooks will
be installed before the first request, allowing the cache to be enabled
on that first request.
This is ok because query cache doesn't load anything else, only itself
so we're not eager loading all of active record before the first
request, just the query cache hooks.
[Eileen M. Uchitelle & Matthew Draper]
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
sensitive value of database column when call `#inspect`
* Why
Some sensitive data will be exposed in log accidentally by calling `#inspect`, e.g.
```ruby
@account = Account.find params[:id]
payload = { account: @account }
logger.info "payload will be #{ payload }"
```
All the information of `@account` will be exposed in log.
* Solution
Add a class attribute filter_attributes to specify which values of columns shouldn't be exposed.
This attribute equals to `Rails.application.config.filter_parameters` by default.
```ruby
Rails.application.config.filter_parameters += [:credit_card_number]
Account.last.insepct # => #<Account id: 123, credit_card_number: [FILTERED] ...>
```
|
| | |
|
| |
|
|
|
|
|
|
|
| |
As discussed in #33203 rails command already looks for, and runs,
bin/rails if it is present.
We were mixing recommendations within guides and USAGE guidelines,
in some files we recommended using rails, in others bin/rails and
in some cases we even had both options mixed together.
|
| |
|
|
| |
[ci skip]
|
| | |
|
| |\
| |
| | |
Only enable verbose_query_logs in Rails server
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Should fix #31688 unless someone can point me to a better way to achieve
this goal. Essentially David's point was that verbose query logging when
enabled in Rails console tends to make things very noisy.
That's especially true if we display absolute paths to callsites which
sadly is still the case when we detect a caller that isn't part of the
Rails application — think gems.
Discussed this with both @matthewd and @rafaelfranca and went back and
forth between enabling if defined?(Rails::Server) or this implementation
and this one makes more sense for now.
Long term I think it'll make sense to let people override this default
disabling in Rails Console because they might want to use the feature
but for now it feels like the correct default behavior.
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rails has some support for multiple databases but it can be hard to
handle migrations with those. The easiest way to implement multiple
databases is to contain migrations into their own folder ("db/migrate"
for the primary db and "db/seconddb_migrate" for the second db). Without
this you would need to write code that allowed you to switch connections
in migrations. I can tell you from experience that is not a fun way to
implement multiple databases.
This refactoring is a pre-requisite for implementing other features
related to parallel testing and improved handling for multiple
databases.
The refactoring here moves the class methods from the `Migrator` class
into it's own new class `MigrationContext`. The goal was to move the
`migrations_paths` method off of the `Migrator` class and onto the
connection. This allows users to do the following in their
`database.yml`:
```
development:
adapter: mysql2
username: root
password:
development_seconddb:
adapter: mysql2
username: root
password:
migrations_paths: "db/second_db_migrate"
```
Migrations for the `seconddb` can now be store in the
`db/second_db_migrate` directory. Migrations for the primary database
are stored in `db/migrate`".
The refactoring here drastically reduces the internal API for migrations
since we don't need to pass `migrations_paths` around to every single
method. Additionally this change does not require any Rails applications
to make changes unless they want to use the new public API. All of the
class methods from the `Migrator` class were `nodoc`'d except for the
`migrations_paths` and `migrations_path` getter/setters respectively.
|
| |
|
|
|
|
|
| |
This is necessary in order to make the processing dependent on
`Migrator.current_version` work even without database.
Context: https://github.com/rails/rails/pull/31135#issuecomment-348404326
|
| | |
|
| |
|
|
| |
This basically reverts 9d4f79d3d394edb74fa2192e5d9ad7b09ce50c6d
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Abstract boolean serialization has been using 't' and 'f', with MySQL
overriding that to use 1 and 0.
This has the advantage that SQLite natively recognizes 1 and 0 as true
and false, but does not natively recognize 't' and 'f'.
This change in serialization requires a migration of stored boolean data
for SQLite databases, so it's implemented behind a configuration flag
whose default false value is deprecated. The flag itself can be
deprecated in a future version of Rails. While loaded models will give
the correct result for boolean columns without migrating old data,
where() clauses will interact incorrectly with old data.
While working in this area, also change the abstract adapter to use
`"TRUE"` and `"FALSE"` as quoted values and `true` and `false` for
unquoted. These are supported by PostreSQL, and MySQL remains
overriden.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Any connections that were checked out during initialization should be
checked back in before the first request is processed, for two reasons:
- Returning the connection to the pool allows it to be health checked
before it's used again. If the connection dies before the first
request arrives, the health check will replace it with a new one.
- If the thread that initialized Rails is not the same thread that will
be performing work, checking in the connection will allow it to be
reused instead of being stuck to the initialization thread forever.
|
| |
|
|
| |
Actually, private methods cannot be called with `self.`, so it's not just redundant, it's a bad habit in Ruby
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
The current code base is not uniform. After some discussion,
we have chosen to go with double quotes by default.
|
| | |
|
| |
|
|
|
|
|
| |
This reverts commit 43ccebc1db072ba0c96a67de0b3db78fd8fd0973.
This is not fixing the configuration problem since we are assigning to
the ActiveRecord::Base not the configuration. See #24303.
|
| |
|
|
|
|
| |
These should allow external code to run blocks of user code to do
"work", at a similar unit size to a web request, without needing to get
intimate with ActionDipatch.
|
| | |
|
| |
|
|
|
|
|
|
| |
While the option on `ActiveRecord::Base` is always around, we need to
explicitly set it on the config object. Otherwise the recommended
configuration change results in an error.
Fixes #22839
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
When `rails console` or `rails server` are used along with a logger set to output to `STDOUT` then the contents will show up twice. This happens because the logger is extended with `ActiveSupportLogger.broadcast` with a destination of STDOUT even if it is already outputting to `STDOUT`.
Previously PR #22592 attempted to fix this issue, but it ended up causing NoMethodErrors. A better approach than relying on adding a method and flow control is to inspect the log destination directly. For this `ActiveSupport::Logger.logger_outputs_to?` was introduced
```ruby
logger = Logger.new(STDOUT)
ActiveSupport::Logger.logger_outputs_to?(logger, STDOUT)
# => true
```
To accomplish this we must look inside of an instance variable of standard lib's Logger `@logdev`. There is a related Ruby proposal to expose this method in a standard way: https://bugs.ruby-lang.org/issues/11955
|
| |
|
|
| |
Still more to do. Please assist!
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
change AR clear order in ActionDisplatch::Reloader hook
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `db:schema:cache:dump` rake task dumps the database schema structure
to `db/schema_cache.dump`. If this file is present, the schema details
are loaded into the currently checked out connection by a railtie while
Rails is booting, to avoid having to query the database for its schema.
The schema cache dump is only applied to the initial connection used to
boot the application though; other connections from the same pool are
created with an empty schema cache, and still have to load the structure
of each table directly from the database.
With this change, a copy of the schema cache is associated with the
connection pool and applied to connections as they are created.
|
