| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since Rails 6.0 will support Ruby 2.4.1 or higher
`# frozen_string_literal: true` magic comment is enough to make string object frozen.
This magic comment is enabled by `Style/FrozenStringLiteralComment` cop.
* Exclude these files not to auto correct false positive `Regexp#freeze`
- 'actionpack/lib/action_dispatch/journey/router/utils.rb'
- 'activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb'
It has been fixed by https://github.com/rubocop-hq/rubocop/pull/6333
Once the newer version of RuboCop released and available at Code Climate these exclude entries should be removed.
* Replace `String#freeze` with `String#-@` manually if explicit frozen string objects are required
- 'actionpack/test/controller/test_case_test.rb'
- 'activemodel/test/cases/type/string_test.rb'
- 'activesupport/lib/active_support/core_ext/string/strip.rb'
- 'activesupport/test/core_ext/string_ext_test.rb'
- 'railties/test/generators/actions_test.rb'
|
|
|
|
|
|
|
|
|
|
|
| |
This is a follow up and/or an alternative of #33844.
Unlike #33844, this would attempt to construct unprepared statement only
when bind params limit (mysql2 65535, pg 65535, sqlite3 249999) is
exceeded.
I only defined 65535 as the limit, not defined 249999 for sqlite3, since
it is an edge case, I'm not excited to add less worth extra code.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since 213796f, bind params are used for IN clause if enabled prepared
statements.
Unfortunately, most adapter modules have a limitation for # of bind
params (mysql2 65535, pg 65535, sqlite3 250000). So if eager loading
large number of records at once, that query couldn't be sent to the
database.
Since eager loading/preloading queries are auto-generated by Active
Record itself, so it should be worked regardless of large number of
records like as before.
Fixes #33702.
|
|
|
|
| |
Context: https://github.com/rails/rails/commit/43ef00e5d7a55ad79bc840276d33cb70f1f5dde5#commitcomment-29256140
|
|
|
|
|
|
|
|
|
|
|
| |
Since #26074, introduced force equality checking to build a predicate
consistently for both `find` and `create` (fixes #27313).
But the assumption that only array/range attribute have subtype was
wrong. We need to make force equality checking more strictly not to
allow serialized attribute.
Fixes #32761.
|
|
|
|
|
| |
Since #32028, Rails 6 requires Ruby 2.3+.
No longer needed workaround for Ruby 2.2 "private attribute?" warning.
|
|
|
|
|
|
|
|
|
| |
Follow up of #31724.
If `composed_of` objects have multiple mappings, array predicate handler
can not correctly handle the expanded condition.
We need to handle it like polymorphic association objects.
|
|
|
|
| |
This basically reverts 9d4f79d3d394edb74fa2192e5d9ad7b09ce50c6d
|
|
|
|
|
|
|
|
| |
I do not want to set the expectation that any enumerable object should
behave this way, but this case in particular comes up frequently enough
that I'm caving on this one.
Fixes #30684.
|
|
|
|
|
|
|
|
|
|
| |
Honestly I don't think the tests that are fixed by this change should
have been merged. Passing a range or an array to `where` has a special
meaning. We need to solve the problem more concretely without overriding
the behavior that is present for *every* other type.
However, the damage has been done. These changes were in 5.1, so we need
a deprecation cycle to remove it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A common source of bugs and code bloat within Active Record has been the
need for us to maintain the list of bind values separately from the AST
they're associated with. This makes any sort of AST manipulation
incredibly difficult, as any time we want to potentially insert or
remove an AST node, we need to traverse the entire tree to find where
the associated bind parameters are.
With this change, the bind parameters now live on the AST directly.
Active Record does not need to know or care about them until the final
AST traversal for SQL construction. Rather than returning just the SQL,
the Arel collector will now return both the SQL and the bind parameters.
At this point the connection adapter will have all the values that it
had before.
A bit of this code is janky and something I'd like to refactor later. In
particular, I don't like how we're handling associations in the
predicate builder, the special casing of `StatementCache::Substitute` in
`QueryAttribute`, or generally how we're handling bind value replacement
in the statement cache when prepared statements are disabled.
This also mostly reverts #26378, as it moved all the code into a
location that I wanted to delete.
/cc @metaskills @yahonda, this change will affect the adapters
Fixes #29766.
Fixes #29804.
Fixes #26541.
Close #28539.
Close #24769.
Close #26468.
Close #26202.
There are probably other issues/PRs that can be closed because of this
commit, but that's all I could find on the first few pages.
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
If casted value is nil, generated SQL should be `IS NULL`. But currently
it is generated as `= NULL`. To prevent this behavior, avoid making bind
param if casted value is nil.
Fixes #28945.
|
| |
|
|
|
|
|
|
| |
Since `AssociationQueryHandler` and `PolymorphicArrayHandler` has
removed in #28715, only exists `AssociationQueryValue` and
`PolymorphicArrayValue` in these files.
|
|
|
|
| |
association handling
|
| |
|
| |
|
|
|
|
| |
Because `RelationHandler` uses `value.arel`.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently association query is handled as a postprocess. This has two
problems.
1. When `value` is a `Hash`, we need to skip the postprocess using
`next`.
2. `can_be_bound?` should return false if
`table.associated_with?(column_name)` is true (pass to the postprocess).
These are unneeded if preprocessing association query handling.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
| |
This reverts commit 3a1f6fe7b4a70bf0698b0684dd48ac712c6883b6.
This commit takes the code in a direction that I am looking to avoid.
The predicate builder should be purely concerned with AST construction
as it matters to methods like `where`. Things like case sensitivity
should continue to be handled elsewhere.
|
|\
| |
| |
| | |
Extract `PredicateBuilder::CaseSensitiveHandler`
|
| |
| |
| |
| |
| |
| | |
Currently uniqueness validator is coupled with building Arel ASTs.
This commit extracts `PredicateBuilder::CaseSensitiveHandler` for
decouple the building Arel ASTs.
|
|/
|
|
|
|
| |
If handled as an associated predicate even though a table has the
column, will generate invalid SQL by valid column name treated as a
table name.
|
|\
| |
| |
| | |
Make association queries to preparable: Step 1
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Currently association queries cannot be preparable.
```ruby
Post.where(author_id: 1).to_a
# => SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = ? [["author_id", 1]]
Post.where(author: 1).to_a
# => SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 1
```
To make association queries to preparable, it should be handled in
`create_binds_for_hash`. This change is a first step for it.
|
|/
|
|
|
|
|
| |
Currently predicate builder cannot build a predicate for `array|range`
attribute. This commit fixes the issue.
Related #25671.
|
|\
| |
| | |
`ActiveRecord::PredicateBuilder#expand` to be private
|
| |
| |
| |
| | |
This method is not touched from outside.
|
|/ |
|
| |
|
|
|
|
|
| |
The current code base is not uniform. After some discussion,
we have chosen to go with double quotes by default.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In 04ac5655be91f49cd4dfe2838df96213502fb274 I assumed that we would
never want to pass the "table_name.column_name" form to where with a
symbol. However, in Ruby 2.2 and later, you can quote symbols using the
new hash syntax, so it's a semi-reasonable thing to do if we want to
support the dot notation (which I'd rather deprecate, but that would be
too painful of a migration).
Instead we've changed the definition of "this is a table name with a
dot" to when the value associated is a hash. It would make very little
sense to write `where("table_name.column_name": { foo: :bar })` in any
scenario (other than equality for a JSON column which we don't support
through `where` in this way).
Close #24514.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This issue occured because associations now call `where` directly, and a
dot in the key name for `where` means nested tables. For this fix, we
now pass the table name as a symbol, and do not attempt to expand
symbols containing a dot.
This is a temporary fix. I do not think we should support table names
containing a dot, as it has a special meaning in most backends, as well
as most APIs that involve table names. This commit does not include a
test, as I am going to deprecate table names containing dots in the
following commit.
Fixes #24367
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
different types.
When passing in an array of different types of objects to `where`, it would only take into account the class of the first object in the array.
PriceEstimate.where(estimate_of: [Treasure.find(1), Car.find(2)])
# => SELECT "price_estimates".* FROM "price_estimates"
WHERE ("price_estimates"."estimate_of_type" = 'Treasure' AND "price_estimates"."estimate_of_id" IN (1, 2))
This is fixed to properly look for any records matching both type and id:
PriceEstimate.where(estimate_of: [Treasure.find(1), Car.find(2)])
# => SELECT "price_estimates".* FROM "price_estimates"
WHERE (("price_estimates"."estimate_of_type" = 'Treasure' AND "price_estimates"."estimate_of_id" = 1)
OR ("price_estimates"."estimate_of_type" = 'Car' AND "price_estimates"."estimate_of_id" = 2))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a similar case to wanting ot use bind params for limit and
offset. Right now passing a range grows the amount of prepared
statements in an unbounded fashion. We could avoid using prepared
statements in that case, similar to what we do with arrays, but there's
a known number of variants for ranges.
This ends up duplicating some of the logic from Arel for how to handle
potentially infinite ranges, and that behavior may be removed from Arel
in the future.
Fixes #23074
|
|
|
|
| |
ClassMethod, Since commit https://github.com/rails/rails/commit/a3936bbe21f4bff8247f890cacfd0fc882921003 [ci skip]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Stackprof output truncated.
```
TOTAL (pct) SAMPLES (pct) FRAME
23 (4.7%) 12 (2.4%) Hash#transform_keys
11 (2.2%) 11 (2.2%) block in Hash#transform_keys
30 (6.1%) 7 (1.4%) Hash#stringify_keys
```
Benchmark Script:
```
begin
require 'bundler/inline'
rescue LoadError => e
$stderr.puts 'Bundler version 1.10 or later is required. Please update your Bundler'
raise e
end
gemfile(true) do
source 'https://rubygems.org'
gem 'rails', path: '~/rails' # master against ref "f1f0a3f8d99aef8aacfa81ceac3880dcac03ca06"
gem 'arel', github: 'rails/arel', branch: 'master'
gem 'rack', github: 'rack/rack', branch: 'master'
gem 'sass'
gem 'sprockets-rails', github: 'rails/sprockets-rails', branch: 'master'
gem 'sprockets', github: 'rails/sprockets', branch: 'master'
gem 'pg'
gem 'benchmark-ips'
end
require 'active_record'
require 'benchmark/ips'
ActiveRecord::Base.establish_connection('postgres://postgres@localhost:5432/rubybench')
ActiveRecord::Migration.verbose = false
ActiveRecord::Schema.define do
create_table :users, force: true do |t|
t.string :name, :email
t.timestamps null: false
end
end
class User < ActiveRecord::Base; end
attributes = {
name: "Lorem ipsum dolor sit amet, consectetur adipiscing elit.",
email: "foobar@email.com",
}
1000.times { User.create!(attributes) }
Benchmark.ips(5, 3) do |x|
x.report('where with hash') { User.where(name: "Lorem ipsum dolor sit amet, consectetur adipiscing elit.") }
x.report('where with string') { User.where("users.name = ?", "Lorem ipsum dolor sit amet, consectetur adipiscing elit.") }
x.compare!
end
key =
if RUBY_VERSION < '2.2'
:total_allocated_object
else
:total_allocated_objects
end
before = GC.stat[key]
User.where(name: "Lorem ipsum dolor sit amet, consectetur adipiscing elit.")
after = GC.stat[key]
puts "Total Allocated Object: #{after - before}"
```
Before:
```
Calculating -------------------------------------
where with hash 2.796k i/100ms
where with string 4.338k i/100ms
-------------------------------------------------
where with hash 29.177k (± 1.5%) i/s - 148.188k
where with string 47.419k (± 2.8%) i/s - 238.590k
Comparison:
where with string: 47419.0 i/s
where with hash: 29176.6 i/s - 1.63x slower
Total Allocated Object: 85
```
After:
```
Calculating -------------------------------------
where with hash 2.895k i/100ms
where with string 4.416k i/100ms
-------------------------------------------------
where with hash 30.758k (± 2.0%) i/s - 156.330k
where with string 47.708k (± 2.6%) i/s - 238.464k
Comparison:
where with string: 47707.9 i/s
where with hash: 30757.7 i/s - 1.55x slower
Total Allocated Object: 84
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I wrote a utility that helps find areas where you could optimize your program using a frozen string instead of a string literal, it's called [let_it_go](https://github.com/schneems/let_it_go). After going through the output and adding `.freeze` I was able to eliminate the creation of 1,114 string objects on EVERY request to [codetriage](codetriage.com). How does this impact execution?
To look at memory:
```ruby
require 'get_process_mem'
mem = GetProcessMem.new
GC.start
GC.disable
1_114.times { " " }
before = mem.mb
after = mem.mb
GC.enable
puts "Diff: #{after - before} mb"
```
Creating 1,114 string objects results in `Diff: 0.03125 mb` of RAM allocated on every request. Or 1mb every 32 requests.
To look at raw speed:
```ruby
require 'benchmark/ips'
number_of_objects_reduced = 1_114
Benchmark.ips do |x|
x.report("freeze") { number_of_objects_reduced.times { " ".freeze } }
x.report("no-freeze") { number_of_objects_reduced.times { " " } }
end
```
We get the results
```
Calculating -------------------------------------
freeze 1.428k i/100ms
no-freeze 609.000 i/100ms
-------------------------------------------------
freeze 14.363k (± 8.5%) i/s - 71.400k
no-freeze 6.084k (± 8.1%) i/s - 30.450k
```
Now we can do some maths:
```ruby
ips = 6_226k # iterations / 1 second
call_time_before = 1.0 / ips # seconds per iteration
ips = 15_254 # iterations / 1 second
call_time_after = 1.0 / ips # seconds per iteration
diff = call_time_before - call_time_after
number_of_objects_reduced * diff * 100
# => 0.4530373333993266 miliseconds saved per request
```
So we're shaving off 1 second of execution time for every 220 requests.
Is this going to be an insane speed boost to any Rails app: nope. Should we merge it: yep.
p.s. If you know of a method call that doesn't modify a string input such as [String#gsub](https://github.com/schneems/let_it_go/blob/b0e2da69f0cca87ab581022baa43291cdf48638c/lib/let_it_go/core_ext/string.rb#L37) please [give me a pull request to the appropriate file](https://github.com/schneems/let_it_go/blob/b0e2da69f0cca87ab581022baa43291cdf48638c/lib/let_it_go/core_ext/string.rb#L37), or open an issue in LetItGo so we can track and freeze more strings.
Keep those strings Frozen
![](https://www.dropbox.com/s/z4dj9fdsv213r4v/let-it-go.gif?dl=1)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit 524d40591eaa2f4d007409bfad386f6b107492eb, reversing
changes made to 34d3a6095100245283861ef480a54d0643bbee4c.
Reasoning behind the revert are in the PR discussion:
https://github.com/rails/rails/pull/19755
- This means that types can no longer cast to/from `Set`, and reasonably
work with `where` (we already have this problem for `array`/`json`
types on pg)
- This adds precedent for every other `Enumerable`, and we can't target
`Enumerable` directly.
- Calling `to_a` on a `Set` is reasonable.
|
|
|
|
|
|
|
|
|
| |
Previously `#where` used to treat `Set`objects as nil, but now it treats
them as an array:
set = Set.new([1, 2])
Author.where(:id => set)
# => SELECT "authors".* FROM "authors" WHERE "authors"."id" IN (1, 2)
|
|
|
|
|
|
|
|
| |
`bound_attributes` is now used universally across the board, removing
the need for the conversion layer. These changes are mostly mechanical,
with the exception of the log subscriber. Additional, we had to
implement `hash` on the attribute objects, so they could be used as a
key for query caching.
|
|
|
|
|
|
|
|
|
|
|
| |
The column is primarily used for type casting, which we're trying to
separate from the idea of a column. Since what we really need is the
combination of a name, type, and value, let's use the object that we
already have to represent that concept, rather than this tuple. No
consumers of the bind values have been changed, only the producers
(outside of tests which care too much about internals). This is
*finally* possible since the bind values are now produced from a
reasonable number of lcoations.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The bind values can come from four places. `having`, `where`, `joins`,
and `from` when selecting from a subquery that contains binds. These
need to be kept in a specific order, since the clauses will always
appear in that order. Up until recently, they were not.
Additionally, `joins` actually did keep its bind values in a separate
location (presumably because it's the only case that people noticed was
broken). However, this meant that anything accessing just `bind_values`
was broken (which most places were). This is no longer possible, there
is only a single way to access the bind values, and it includes joins in
the proper location. The setter was removed yesterday, so breaking `+=`
cases is not possible.
I'm still not happy that `joins` is putting it's bind values on the
Arel AST, and I'm planning on refactoring it further, but this removes a
ton of bug cases.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This will allow all types which require no additional handling to use
prepared statements. Specifically, this will allow for `true`, `false`,
`Date`, `Time`, and any custom PG type to use prepared statements. This
also revealed another source of nil columns in bind params, and an
inconsistency in their use.
The specific inconsistency comes from a nested query coming from a
through association, where one of the inversed associations is not
bi-directional.
The stop-gap is to simply construct the column at the site it is being
used. This should simply go away on its own once we use `Attribute` to
represent them instead, since we already have all of the information we
need.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Specifically, the issue is relying on `where_unscoping` mutating the
where values. It does not, however, mutate the bind values, which could
cause an error under certain circumstances. This was not exposed by the
tests, since the only place which would have been affected is unscoping
a boolean, which doesn't go through prepared statements. I had a hard
time getting better test coverage to demonstrate the issue.
This in turn, caused `merge` to go through proper logic, and try to
clear out the binds associated with the unscoped relation, which then
exposed a source of `nil` for the columns, as binds weren't expanding
`{ "posts.id" => 1 }` to `{ "posts" => { "id" => 1 } }`. This has been
fixed.
The bulk of `create_binds` needed to be moved to a separate method,
since the dot notation should not be expanded recursively.
I'm pretty sure this removes a subtle quirk that a ton of code in
`Relation::Merger` is working around, and I suspect that code can be
greatly simplified. However, unraveling that rats nest is no small task.
|