| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
| |
Honestly I don't think the tests that are fixed by this change should
have been merged. Passing a range or an array to `where` has a special
meaning. We need to solve the problem more concretely without overriding
the behavior that is present for *every* other type.
However, the damage has been done. These changes were in 5.1, so we need
a deprecation cycle to remove it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A common source of bugs and code bloat within Active Record has been the
need for us to maintain the list of bind values separately from the AST
they're associated with. This makes any sort of AST manipulation
incredibly difficult, as any time we want to potentially insert or
remove an AST node, we need to traverse the entire tree to find where
the associated bind parameters are.
With this change, the bind parameters now live on the AST directly.
Active Record does not need to know or care about them until the final
AST traversal for SQL construction. Rather than returning just the SQL,
the Arel collector will now return both the SQL and the bind parameters.
At this point the connection adapter will have all the values that it
had before.
A bit of this code is janky and something I'd like to refactor later. In
particular, I don't like how we're handling associations in the
predicate builder, the special casing of `StatementCache::Substitute` in
`QueryAttribute`, or generally how we're handling bind value replacement
in the statement cache when prepared statements are disabled.
This also mostly reverts #26378, as it moved all the code into a
location that I wanted to delete.
/cc @metaskills @yahonda, this change will affect the adapters
Fixes #29766.
Fixes #29804.
Fixes #26541.
Close #28539.
Close #24769.
Close #26468.
Close #26202.
There are probably other issues/PRs that can be closed because of this
commit, but that's all I could find on the first few pages.
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
If casted value is nil, generated SQL should be `IS NULL`. But currently
it is generated as `= NULL`. To prevent this behavior, avoid making bind
param if casted value is nil.
Fixes #28945.
|
| |
|
|
|
|
|
|
| |
Since `AssociationQueryHandler` and `PolymorphicArrayHandler` has
removed in #28715, only exists `AssociationQueryValue` and
`PolymorphicArrayValue` in these files.
|
|
|
|
| |
association handling
|
| |
|
| |
|
|
|
|
| |
Because `RelationHandler` uses `value.arel`.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently association query is handled as a postprocess. This has two
problems.
1. When `value` is a `Hash`, we need to skip the postprocess using
`next`.
2. `can_be_bound?` should return false if
`table.associated_with?(column_name)` is true (pass to the postprocess).
These are unneeded if preprocessing association query handling.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
| |
This reverts commit 3a1f6fe7b4a70bf0698b0684dd48ac712c6883b6.
This commit takes the code in a direction that I am looking to avoid.
The predicate builder should be purely concerned with AST construction
as it matters to methods like `where`. Things like case sensitivity
should continue to be handled elsewhere.
|
|\
| |
| |
| | |
Extract `PredicateBuilder::CaseSensitiveHandler`
|
| |
| |
| |
| |
| |
| | |
Currently uniqueness validator is coupled with building Arel ASTs.
This commit extracts `PredicateBuilder::CaseSensitiveHandler` for
decouple the building Arel ASTs.
|
|/
|
|
|
|
| |
If handled as an associated predicate even though a table has the
column, will generate invalid SQL by valid column name treated as a
table name.
|
|\
| |
| |
| | |
Make association queries to preparable: Step 1
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Currently association queries cannot be preparable.
```ruby
Post.where(author_id: 1).to_a
# => SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = ? [["author_id", 1]]
Post.where(author: 1).to_a
# => SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 1
```
To make association queries to preparable, it should be handled in
`create_binds_for_hash`. This change is a first step for it.
|
|/
|
|
|
|
|
| |
Currently predicate builder cannot build a predicate for `array|range`
attribute. This commit fixes the issue.
Related #25671.
|
|\
| |
| | |
`ActiveRecord::PredicateBuilder#expand` to be private
|
| |
| |
| |
| | |
This method is not touched from outside.
|
|/ |
|
| |
|
|
|
|
|
| |
The current code base is not uniform. After some discussion,
we have chosen to go with double quotes by default.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In 04ac5655be91f49cd4dfe2838df96213502fb274 I assumed that we would
never want to pass the "table_name.column_name" form to where with a
symbol. However, in Ruby 2.2 and later, you can quote symbols using the
new hash syntax, so it's a semi-reasonable thing to do if we want to
support the dot notation (which I'd rather deprecate, but that would be
too painful of a migration).
Instead we've changed the definition of "this is a table name with a
dot" to when the value associated is a hash. It would make very little
sense to write `where("table_name.column_name": { foo: :bar })` in any
scenario (other than equality for a JSON column which we don't support
through `where` in this way).
Close #24514.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This issue occured because associations now call `where` directly, and a
dot in the key name for `where` means nested tables. For this fix, we
now pass the table name as a symbol, and do not attempt to expand
symbols containing a dot.
This is a temporary fix. I do not think we should support table names
containing a dot, as it has a special meaning in most backends, as well
as most APIs that involve table names. This commit does not include a
test, as I am going to deprecate table names containing dots in the
following commit.
Fixes #24367
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
different types.
When passing in an array of different types of objects to `where`, it would only take into account the class of the first object in the array.
PriceEstimate.where(estimate_of: [Treasure.find(1), Car.find(2)])
# => SELECT "price_estimates".* FROM "price_estimates"
WHERE ("price_estimates"."estimate_of_type" = 'Treasure' AND "price_estimates"."estimate_of_id" IN (1, 2))
This is fixed to properly look for any records matching both type and id:
PriceEstimate.where(estimate_of: [Treasure.find(1), Car.find(2)])
# => SELECT "price_estimates".* FROM "price_estimates"
WHERE (("price_estimates"."estimate_of_type" = 'Treasure' AND "price_estimates"."estimate_of_id" = 1)
OR ("price_estimates"."estimate_of_type" = 'Car' AND "price_estimates"."estimate_of_id" = 2))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a similar case to wanting ot use bind params for limit and
offset. Right now passing a range grows the amount of prepared
statements in an unbounded fashion. We could avoid using prepared
statements in that case, similar to what we do with arrays, but there's
a known number of variants for ranges.
This ends up duplicating some of the logic from Arel for how to handle
potentially infinite ranges, and that behavior may be removed from Arel
in the future.
Fixes #23074
|
|
|
|
| |
ClassMethod, Since commit https://github.com/rails/rails/commit/a3936bbe21f4bff8247f890cacfd0fc882921003 [ci skip]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Stackprof output truncated.
```
TOTAL (pct) SAMPLES (pct) FRAME
23 (4.7%) 12 (2.4%) Hash#transform_keys
11 (2.2%) 11 (2.2%) block in Hash#transform_keys
30 (6.1%) 7 (1.4%) Hash#stringify_keys
```
Benchmark Script:
```
begin
require 'bundler/inline'
rescue LoadError => e
$stderr.puts 'Bundler version 1.10 or later is required. Please update your Bundler'
raise e
end
gemfile(true) do
source 'https://rubygems.org'
gem 'rails', path: '~/rails' # master against ref "f1f0a3f8d99aef8aacfa81ceac3880dcac03ca06"
gem 'arel', github: 'rails/arel', branch: 'master'
gem 'rack', github: 'rack/rack', branch: 'master'
gem 'sass'
gem 'sprockets-rails', github: 'rails/sprockets-rails', branch: 'master'
gem 'sprockets', github: 'rails/sprockets', branch: 'master'
gem 'pg'
gem 'benchmark-ips'
end
require 'active_record'
require 'benchmark/ips'
ActiveRecord::Base.establish_connection('postgres://postgres@localhost:5432/rubybench')
ActiveRecord::Migration.verbose = false
ActiveRecord::Schema.define do
create_table :users, force: true do |t|
t.string :name, :email
t.timestamps null: false
end
end
class User < ActiveRecord::Base; end
attributes = {
name: "Lorem ipsum dolor sit amet, consectetur adipiscing elit.",
email: "foobar@email.com",
}
1000.times { User.create!(attributes) }
Benchmark.ips(5, 3) do |x|
x.report('where with hash') { User.where(name: "Lorem ipsum dolor sit amet, consectetur adipiscing elit.") }
x.report('where with string') { User.where("users.name = ?", "Lorem ipsum dolor sit amet, consectetur adipiscing elit.") }
x.compare!
end
key =
if RUBY_VERSION < '2.2'
:total_allocated_object
else
:total_allocated_objects
end
before = GC.stat[key]
User.where(name: "Lorem ipsum dolor sit amet, consectetur adipiscing elit.")
after = GC.stat[key]
puts "Total Allocated Object: #{after - before}"
```
Before:
```
Calculating -------------------------------------
where with hash 2.796k i/100ms
where with string 4.338k i/100ms
-------------------------------------------------
where with hash 29.177k (± 1.5%) i/s - 148.188k
where with string 47.419k (± 2.8%) i/s - 238.590k
Comparison:
where with string: 47419.0 i/s
where with hash: 29176.6 i/s - 1.63x slower
Total Allocated Object: 85
```
After:
```
Calculating -------------------------------------
where with hash 2.895k i/100ms
where with string 4.416k i/100ms
-------------------------------------------------
where with hash 30.758k (± 2.0%) i/s - 156.330k
where with string 47.708k (± 2.6%) i/s - 238.464k
Comparison:
where with string: 47707.9 i/s
where with hash: 30757.7 i/s - 1.55x slower
Total Allocated Object: 84
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I wrote a utility that helps find areas where you could optimize your program using a frozen string instead of a string literal, it's called [let_it_go](https://github.com/schneems/let_it_go). After going through the output and adding `.freeze` I was able to eliminate the creation of 1,114 string objects on EVERY request to [codetriage](codetriage.com). How does this impact execution?
To look at memory:
```ruby
require 'get_process_mem'
mem = GetProcessMem.new
GC.start
GC.disable
1_114.times { " " }
before = mem.mb
after = mem.mb
GC.enable
puts "Diff: #{after - before} mb"
```
Creating 1,114 string objects results in `Diff: 0.03125 mb` of RAM allocated on every request. Or 1mb every 32 requests.
To look at raw speed:
```ruby
require 'benchmark/ips'
number_of_objects_reduced = 1_114
Benchmark.ips do |x|
x.report("freeze") { number_of_objects_reduced.times { " ".freeze } }
x.report("no-freeze") { number_of_objects_reduced.times { " " } }
end
```
We get the results
```
Calculating -------------------------------------
freeze 1.428k i/100ms
no-freeze 609.000 i/100ms
-------------------------------------------------
freeze 14.363k (± 8.5%) i/s - 71.400k
no-freeze 6.084k (± 8.1%) i/s - 30.450k
```
Now we can do some maths:
```ruby
ips = 6_226k # iterations / 1 second
call_time_before = 1.0 / ips # seconds per iteration
ips = 15_254 # iterations / 1 second
call_time_after = 1.0 / ips # seconds per iteration
diff = call_time_before - call_time_after
number_of_objects_reduced * diff * 100
# => 0.4530373333993266 miliseconds saved per request
```
So we're shaving off 1 second of execution time for every 220 requests.
Is this going to be an insane speed boost to any Rails app: nope. Should we merge it: yep.
p.s. If you know of a method call that doesn't modify a string input such as [String#gsub](https://github.com/schneems/let_it_go/blob/b0e2da69f0cca87ab581022baa43291cdf48638c/lib/let_it_go/core_ext/string.rb#L37) please [give me a pull request to the appropriate file](https://github.com/schneems/let_it_go/blob/b0e2da69f0cca87ab581022baa43291cdf48638c/lib/let_it_go/core_ext/string.rb#L37), or open an issue in LetItGo so we can track and freeze more strings.
Keep those strings Frozen
![](https://www.dropbox.com/s/z4dj9fdsv213r4v/let-it-go.gif?dl=1)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit 524d40591eaa2f4d007409bfad386f6b107492eb, reversing
changes made to 34d3a6095100245283861ef480a54d0643bbee4c.
Reasoning behind the revert are in the PR discussion:
https://github.com/rails/rails/pull/19755
- This means that types can no longer cast to/from `Set`, and reasonably
work with `where` (we already have this problem for `array`/`json`
types on pg)
- This adds precedent for every other `Enumerable`, and we can't target
`Enumerable` directly.
- Calling `to_a` on a `Set` is reasonable.
|
|
|
|
|
|
|
|
|
| |
Previously `#where` used to treat `Set`objects as nil, but now it treats
them as an array:
set = Set.new([1, 2])
Author.where(:id => set)
# => SELECT "authors".* FROM "authors" WHERE "authors"."id" IN (1, 2)
|
|
|
|
|
|
|
|
| |
`bound_attributes` is now used universally across the board, removing
the need for the conversion layer. These changes are mostly mechanical,
with the exception of the log subscriber. Additional, we had to
implement `hash` on the attribute objects, so they could be used as a
key for query caching.
|
|
|
|
|
|
|
|
|
|
|
| |
The column is primarily used for type casting, which we're trying to
separate from the idea of a column. Since what we really need is the
combination of a name, type, and value, let's use the object that we
already have to represent that concept, rather than this tuple. No
consumers of the bind values have been changed, only the producers
(outside of tests which care too much about internals). This is
*finally* possible since the bind values are now produced from a
reasonable number of lcoations.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The bind values can come from four places. `having`, `where`, `joins`,
and `from` when selecting from a subquery that contains binds. These
need to be kept in a specific order, since the clauses will always
appear in that order. Up until recently, they were not.
Additionally, `joins` actually did keep its bind values in a separate
location (presumably because it's the only case that people noticed was
broken). However, this meant that anything accessing just `bind_values`
was broken (which most places were). This is no longer possible, there
is only a single way to access the bind values, and it includes joins in
the proper location. The setter was removed yesterday, so breaking `+=`
cases is not possible.
I'm still not happy that `joins` is putting it's bind values on the
Arel AST, and I'm planning on refactoring it further, but this removes a
ton of bug cases.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This will allow all types which require no additional handling to use
prepared statements. Specifically, this will allow for `true`, `false`,
`Date`, `Time`, and any custom PG type to use prepared statements. This
also revealed another source of nil columns in bind params, and an
inconsistency in their use.
The specific inconsistency comes from a nested query coming from a
through association, where one of the inversed associations is not
bi-directional.
The stop-gap is to simply construct the column at the site it is being
used. This should simply go away on its own once we use `Attribute` to
represent them instead, since we already have all of the information we
need.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Specifically, the issue is relying on `where_unscoping` mutating the
where values. It does not, however, mutate the bind values, which could
cause an error under certain circumstances. This was not exposed by the
tests, since the only place which would have been affected is unscoping
a boolean, which doesn't go through prepared statements. I had a hard
time getting better test coverage to demonstrate the issue.
This in turn, caused `merge` to go through proper logic, and try to
clear out the binds associated with the unscoped relation, which then
exposed a source of `nil` for the columns, as binds weren't expanding
`{ "posts.id" => 1 }` to `{ "posts" => { "id" => 1 } }`. This has been
fixed.
The bulk of `create_binds` needed to be moved to a separate method,
since the dot notation should not be expanded recursively.
I'm pretty sure this removes a subtle quirk that a ton of code in
`Relation::Merger` is working around, and I suspect that code can be
greatly simplified. However, unraveling that rats nest is no small task.
|
|
|
|
|
|
|
|
|
|
|
| |
With the old implementation, the bind values were created, and then we
search the attributes for `Relation` objects, and merge them. This
completely ignores the order that the actual `where` clause will use. If
all non-relation where parameters are before the relations, it will
work. However, if we query on both a relation and a value, with the
value coming second, it breaks. The order of the hash should not affect
the final query (especially since hashes being ordered is an
implementation detail)
|
|
|
|
|
|
| |
I'm looking to introduce a `WhereClause` class to handle most of this
logic, and this method will eventually move over to there. However, this
intermediate refactoring should make that easier to do.
|
|
|
|
|
|
|
| |
This API will require much less consuming code to change to accomodate
the removal of automatic type casting from Arel. As long as the
predicates are constructed using the `arel_table` off of an AR subclass,
there will be no changes that need to happen.
|
|
|
|
|
|
|
|
|
| |
A custom object is required for this, as you cannot build a range
object out of `Arel::Nodes::Quoted` objects. Depends on the changes
introduced in
https://github.com/rails/arel/commit/cf03bd45e39def057a2f63e42a3391b7d750dece
/cc @mrgilman
|
|
|
|
|
|
|
|
|
|
|
| |
As part of the larger refactoring to remove type casting from Arel, we
need to do the casting of values eagerly. The predicate builder is the
closest place that knows about the Active Record class, and can
therefore have the type information.
/cc @mrgilman
[Sean Griffin & Melanie Gilman]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This class cares far too much about the internals of other parts of
Active Record. This is an attempt to break out a meaningful object which
represents the needs of the predicate builder. I'm not fully satisfied
with the name, but the general concept is an object which represents a
table, the associations to/from that table, and the types associated
with it. Many of these exist at the `ActiveRecord::Base` class level,
not as properties of the table itself, hence the need for another
object. Currently it provides these by holding a reference to the class,
but that will likely change in the future. This allows the predicate
builder to remain wholy concerned with building predicates.
/cc @mrgilman
|
|
|
|
|
|
|
|
|
|
| |
I'm attempting to remove `klass` as a dependency of the predicate
builder, in favor of an object that better represents what we're using
it for. The only part of this which doesn't fit nicely into that picture
is the check for an association being polymorphic. Since I'm not yet
sure what that is going to look like, I've moved this logic into another
class in an attempt to separate things that will change from things that
won't.
|
|
|
|
|
|
|
|
|
|
| |
This reduces the number of places which will need to care about single
value or range specific logic as we introduce type casting. The array
handler is only responsible for producing `in` statements.
/cc @mrgilman
[Sean Griffin & Melanie Gilman]
|
|
|
|
|
|
|
|
|
|
|
|
| |
This will allow us to pass the predicate builder into the constructor of
these handlers. The procs had to be changed to objects, because the
`PredicateBuilder` needs to be marshalable. If we ever decide to make
`register_handler` part of the public API, we should come up with a
better solution which allows procs.
/cc @mrgilman
[Sean Griffin & Melanie Gilman]
|