| Commit message (Collapse) | Author | Age | Files | Lines |
|\
| |
| | |
Reduce unused allocations when casting UUIDs for Postgres
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Using the subscript method `#[]` on a string has several overloads and
rather complex implementation. One of the overloads is the capability to
accept a regular expression and then run a match, then return the
receiver (if it matched) or one of the groups from the MatchData.
The function of the `UUID#cast` method is to cast a UUID to a type and
format acceptable by postgres. Naturally UUIDs are supposed to be
string and of a certain format, but it had been determined that it was
not ideal for the framework to send just any old string to Postgres and
allow the engine to complain when "foobar" or "" was sent, being
obviously of the wrong format for a valid UUID. Therefore this code was
written to facilitate the checking, and if it were not of the correct
format, a `nil` would be returned as is conventional in Rails.
Now, the subscript method will allocate one or more strings on a match
and return one of them, based on the index parameter. However, there
is no need for a new string, as a UUID of the correct format is already
such, and so long as the format was verified then the string supplied is
adequate for consumption by the database.
The subscript method also creates a MatchData object which will never be
used, and so must eventually be garbage collected.
Garbage collection indeed. This innocuous method tends to be called
quite a lot, for example if the primary key of a table is a uuid, then
this method will be called. If the foreign key of a relation is a UUID,
once again this method is called. If that foreign key is belonging to
a has_many relationship with dozens of objects, then again dozens of
UUIDs shall be cast to a dup of themselves, and spawn dozens of
MatchData objects, and so on.
So, for users that:
* Use UUIDs as primary keys
* Use Postgres
* Operate on collections of objects
This accomplishes a significant savings in total allocations, and may
save many garbage collections.
|
|/ |
|
|
|
|
| |
It should be referenced by full qualified name from Active Record.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before:
```
LOG: execute <unnamed>: SELECT t.oid, t.typname
FROM pg_type as t
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool')
LOG: execute <unnamed>: SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype
FROM pg_type as t
LEFT JOIN pg_range as r ON oid = rngtypid
WHERE
t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'interval', 'path', 'line', 'polygon', 'circle', 'lseg', 'box', 'time', 'timestamp', 'numeric')
OR t.typtype IN ('r', 'e', 'd')
OR t.typinput::varchar = 'array_in'
OR t.typelem != 0
LOG: statement: SHOW TIME ZONE
LOG: statement: SELECT 1
LOG: execute <unnamed>: SELECT COUNT(*)
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r','v','m') -- (r)elation/table, (v)iew, (m)aterialized view
AND c.relname = 'accounts'
AND n.nspname = ANY (current_schemas(false))
```
After:
```
LOG: execute <unnamed>: SELECT t.oid, t.typname
FROM pg_type as t
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool')
LOG: execute <unnamed>: SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype
FROM pg_type as t
LEFT JOIN pg_range as r ON oid = rngtypid
WHERE
t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'interval', 'path', 'line', 'polygon', 'circle', 'lseg', 'box', 'time', 'timestamp', 'numeric')
OR t.typtype IN ('r', 'e', 'd')
OR t.typinput::varchar = 'array_in'
OR t.typelem != 0
LOG: statement: SHOW TIME ZONE
LOG: statement: SELECT 1
LOG: execute <unnamed>: SELECT COUNT(*)
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r','v','m') -- (r)elation/table, (v)iew, (m)aterialized view
AND c.relname = 'accounts'
AND n.nspname = ANY (current_schemas(false))
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since Rails 6.0 will support Ruby 2.4.1 or higher
`# frozen_string_literal: true` magic comment is enough to make string object frozen.
This magic comment is enabled by `Style/FrozenStringLiteralComment` cop.
* Exclude these files not to auto correct false positive `Regexp#freeze`
- 'actionpack/lib/action_dispatch/journey/router/utils.rb'
- 'activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb'
It has been fixed by https://github.com/rubocop-hq/rubocop/pull/6333
Once the newer version of RuboCop released and available at Code Climate these exclude entries should be removed.
* Replace `String#freeze` with `String#-@` manually if explicit frozen string objects are required
- 'actionpack/test/controller/test_case_test.rb'
- 'activemodel/test/cases/type/string_test.rb'
- 'activesupport/lib/active_support/core_ext/string/strip.rb'
- 'activesupport/test/core_ext/string_ext_test.rb'
- 'railties/test/generators/actions_test.rb'
|
|\
| |
| | |
PostgreSQL: prepare for pg-1.1
|
| |
| |
| |
| |
| |
| |
| |
| | |
Parsing of malformed array strings without raising an error is deprecated in
pg-1.1. It's therefore necessary to catch parser errors starting with pg-2.0.
See also pg commit:
https://bitbucket.org/ged/ruby-pg/commits/1b081326b346368e70c9c03ee7080e28d6b3a3dc
|
|/ |
|
|
|
|
|
|
|
|
|
|
|
| |
Since #26074, introduced force equality checking to build a predicate
consistently for both `find` and `create` (fixes #27313).
But the assumption that only array/range attribute have subtype was
wrong. We need to make force equality checking more strictly not to
allow serialized attribute.
Fixes #32761.
|
|
|
|
|
|
|
|
|
| |
BC dates are supported by both date and datetime types.
https://www.postgresql.org/docs/current/static/datatype-datetime.html
Since #1097, new datetime allows year zero as 1 BC, but new date does
not. It should be allowed even in new date consistently.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The values infinity and -infinity are supported by both date and
timestamp types.
https://www.postgresql.org/docs/current/static/datatype-datetime.html#DATATYPE-DATETIME-SPECIAL-TABLE
And also, it can not be known whether a value is infinity correctly
unless cast a value.
I've added `QueryAttribute#infinity?` to handle that case.
Closes #27585.
|
| |
|
|
|
|
|
|
|
|
| |
An empty string is an invalid value in Ruby's range class.
So need to handle `Float::INFINITY` as it is and cast it in
`encode_range`.
Fixes #31612
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`BigDecimal.new` has been deprecated in BigDecimal 1.3.3
which will be a default for Ruby 2.5.
Refer https://github.com/ruby/bigdecimal/commit/533737338db915b00dc7168c3602e4b462b23503
```
$ cd rails/activerecord/
$ git grep -l BigDecimal.new | grep \.rb | xargs sed -i -e "s/BigDecimal.new/BigDecimal/g"
```
- Changes made only to Active Record. Will apply the same change to
other module once this commit is merged.
- The following deprecation has not been addressed because it has been
reported at `ActiveRecord::Result.new`. `ActiveRecord::Result.ancestors`
did not show `BigDecimal`.
* Not addressed
```ruby
/path/to/rails/activerecord/lib/active_record/connection_adapters/mysql/database_statements.rb:34:
warning: BigDecimal.new is deprecated
```
* database_statements.rb:34
```ruby
ActiveRecord::Result.new(result.fields, result.to_a) if result
```
* ActiveRecord::Result.ancestors
```ruby
[ActiveRecord::Result,
Enumerable,
ActiveSupport::ToJsonWithActiveSupportEncoder,
Object,
Metaclass::ObjectMethods,
Mocha::ObjectMethods,
PP::ObjectMixin,
ActiveSupport::Dependencies::Loadable,
ActiveSupport::Tryable,
JSON::Ext::Generator::GeneratorMethods::Object,
Kernel,
BasicObject]
```
This commit has been tested with these Ruby and BigDecimal versions
- ruby 2.5 and bigdecimal 1.3.3
```
$ ruby -v
ruby 2.5.0dev (2017-12-14 trunk 61217) [x86_64-linux]
$ gem list |grep bigdecimal
bigdecimal (default: 1.3.3, default: 1.3.2)
```
- ruby 2.4 and bigdecimal 1.3.0
```
$ ruby -v
ruby 2.4.2p198 (2017-09-14 revision 59899) [x86_64-linux-gnu]
$ gem list |grep bigdecimal
bigdecimal (default: 1.3.0)
```
- ruby 2.3 and bigdecimal 1.2.8
```
$ ruby -v
ruby 2.3.5p376 (2017-09-14 revision 59905) [x86_64-linux]
$ gem list |grep -i bigdecimal
bigdecimal (1.2.8)
```
- ruby 2.2 and bigdecimal 1.2.6
```
$ ruby -v
ruby 2.2.8p477 (2017-09-14 revision 59906) [x86_64-linux]
$ gem list |grep bigdecimal
bigdecimal (1.2.6)
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
PostgreSQL 9.1+ introduced range types, and Rails added support for
using this datatype in ActiveRecord. However, the serialization of
`PostgreSQL::OID::Range` was incomplete, because it did not properly
quote the bounds that make up the range. A clear example of this is a
`tsrange`.
Normally, ActiveRecord quotes Date/Time objects to include the
milliseconds. However, the way `PostgreSQL::OID::Range` serialized its
bounds, the milliseconds were dropped. This meant that the value was
incomplete and not equal to the submitted value.
An example of normal timestamps vs. a `tsrange`. Note how the bounds
for the range do not include their milliseconds (they were present in
the ruby Range):
UPDATE "iterations" SET "updated_at" = $1, "range" = $2 WHERE
"iterations"."id" = $3
[["updated_at", "2017-09-23 17:07:01.304864"],
["range", "[2017-09-23 00:00:00 UTC,2017-09-23 23:59:59 UTC]"],
["id", 1234]]
`PostgreSQL::OID::Range` serialized the range by interpolating a
string for the range, which works for most cases, but does not work
for timestamps:
def serialize(value)
if value.is_a?(::Range)
from = type_cast_single_for_database(value.begin)
to = type_cast_single_for_database(value.end)
"[#{from},#{to}#{value.exclude_end? ? ')' : ']'}"
else
super
end
end
(byebug) from = type_cast_single_for_database(value.begin)
2010-01-01 13:30:00 UTC
(byebug) to = type_cast_single_for_database(value.end)
2011-02-02 19:30:00 UTC
(byebug) "[#{from},#{to}#{value.exclude_end? ? ')' : ']'}"
"[2010-01-01 13:30:00 UTC,2011-02-02 19:30:00 UTC)"
@sgrif (the original implementer for Postgres Range support) provided
some feedback about where the quoting should occur:
Yeah, quoting at all is definitely wrong here. I'm not sure what I
was thinking in 02579b5, but what this is doing is definitely in the
wrong place. It should probably just be returning a range of
subtype.serialize(value.begin) and subtype.serialize(value.end), and
letting the adapter handle the rest.
`Postgres::OID::Range` now returns a `Range` object, and
`ActiveRecord::ConnectionAdapters::PostgreSQL::Quoting` can now encode
and quote a `Range`:
def encode_range(range)
"[#{type_cast(range.first)},#{type_cast(range.last)}#{range.exclude_end? ? ')' : ']'}"
end
...
encode_range(range)
#=> "['2010-01-01 13:30:00.670277','2011-02-02 19:30:00.745125')"
This commit includes tests to make sure the milliseconds are
preserved in `tsrange` and `tstzrange` columns
|
|\
| |
| | |
Make `type_map` to private because it is only used in the connection adapter
|
| |
| |
| |
| |
| |
| |
| | |
`type_map` is an internal API and it is only used in the connection
adapter. And also, some type map initializer methods requires passed
`type_map`, but those instances already has `type_map` in itself.
So we don't need explicit passing `type_map` to the initializers.
|
|/ |
|
| |
|
|\
| |
| | |
Prepare AP and AR to be frozen string friendly
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| | |
The uuid validation regex was allowing uuids to have a single leading
curly brace or single trailing curly brace. Saving with such a uuid
would cause Postgres to generate an exception even though the record
seemed valid. With this change, the regex requires both a leading *and*
a trailing curly brace or neither to be valid.
|
|/
|
|
| |
We already have database agnostic `Type::Json` since #29220.
|
|
|
|
|
| |
This reverts commit 3420a14590c0e6915d8b6c242887f74adb4120f9, reversing
changes made to afb66a5a598ce4ac74ad84b125a5abf046dcf5aa.
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
`AbstractJson`
Structured type values sometimes caused representation problems (keys
sort order, spaces, etc). A raw value from the database should be
deserialized (normalized) to prevent the problems.
|
|
|
|
|
| |
The old top level classes PGconn, PGresult and PGError were deprecated
since pg-0.13.0: https://github.com/ged/ruby-pg/blob/master/History.rdoc#v0130-2012-02-09-michael-granger-gedfaeriemudorg
|
|
|
|
| |
Fixes #28285.
|
|
|
|
| |
Closes #27980
|
|
|
|
| |
Closes #27979
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Per the regression commit below, the commit changes the behavior of
`#changed?`to consult the `#changed_in_place?` method on `Type::Value` classes.
Per this change, `PostgreSQL::OID::Hstore` needs to override this method
in order to compare the deserialized forms of the two arguments. In
Ruby, two hashes are considered equal even if their key order is
different. This commit helps to bring that behavior to `Hstore` values.
Fixes regression introduced by 8e633e505880755e7e366ccec2210bbe2b5436e7
Fixes #27502
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In f1a0fa9 we moved backend specific timestamp behavior out of the type
and into the adapter. This was in line with our general attempt to
reduce the number of adapter specific type subclasses. However, on PG,
the array type performs all serialization, including database encoding
in its serialize method.
This means that we have converted the value into a string before
reaching the database, so no adapter specific logic can be applied (and
this also means that timestamp arrays were using the default `.to_s`
method on the given object, which likely meant timestamps were being
ignored in certain cases as well)
Ultimately I want to do a more in depth refactoring which separates
database serializer objects from the active model type objects, to give
us a less awkward API for introducing the attributes API onto Active
Model.
However, in the short term, we follow the solution we've applied
elsewhere for this. Move behavior off of the type and into the adapter,
and use a data object to allow the type to communicate information up
the stack.
Fixes #27514.
|
|
|
|
|
| |
Currently schema dumper does not dump array subtype `precision` and
`scale` options. This commit fixes the issue.
|
| |
|
|\
| |
| | |
use `force_encoding` instread of `encode!` to avoid `UndefinedConversionError`
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
`PG::TextEncoder::Array#encode` returns the encoded value with `ASCII-8BIT`.
But in some cases, trying to convert `ASCII-8BIT` to `UTF-8` cause an error.
```ruby
"{\xE3\x83\x95\xE3\x82\xA1\xE3\x82\xA4\xE3\x83\xAB}".encode!(Encoding::UTF_8)
# => Encoding::UndefinedConversionError: "\xE3" from ASCII-8BIT to UTF-8
```
Should use `force_encoding` to avoid this error.
Follow up to 7ba3a48df5bfdc5e98506bb829f937e03b55a5b3
Ref: https://github.com/rails/rails/pull/23619#issuecomment-189924036
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As reported via #26904, there is a regression in how values for
Postgres' HStore column type are being processed, beginning in Rails 5.
Currently, the way that Active Record checks whether or not values need
to be serialized and put into the correct storage format is whether or
not it is a `Hash` object. Since `ActionController::Parameters` no
longer inherits from `Hash` in Rails 5, this conditional now returns
false. To remedy this, we are now checking to see whether the `value`
parameters being passed in responds to a certain method, and then
calling the `serialize` method, except this time with a real Hash
object. Keeping things DRY!
Fixes #26904.
|
| |
|
|
|
|
| |
Follow up to 99cf7558000090668b137085bfe6bcc06c4571dc.
|
|
|
|
|
|
|
|
|
|
| |
There are some minor changes to the point type as I had forgotten that
this will affect the behavior of `t.point` in migrations and the schema
dumper so we need to handle those as well.
I'll say this again so I can convince myself to come up with a better
structure... TYPES SHOULD NOT CARE ABOUT SCHEMA DUMPING AND WE NEED TO
BETTER SEPARATE THESE.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I still think that this is something that should be handled in the pg
gem, but it's not going to end up happening there so we'll do it here
instead. Once we bump to pg 0.19 we can pass the encoding to the
`encode` method instead.
This issue occurs because C has no concept of encoding (or strings,
really). The bytes that we pass here when sending the value to the
database will always be interpreted as whatever encoding the connection
is currently configured to use. That means that roundtripping to the
database will lose no information
However, after assigning we round trip through our type system without
hitting the database. The only way that we can do the "correct" thin
here would be to actually give a reference to the connection to the
array type and have it check the current value of the connection's
encoding -- which I'm strongly opposed to. We could also pass in the
encoding when it's constructed, but since that can change independently
of the type I'm not a huge fan of that either.
This feels like a reasonable middle ground, where if we have an array of
strings we simply use the encoding of the string we're given.
Fixes #26326.
|
|
|
|
| |
Fixes #26137.
|
| |
|
|
|
|
|
| |
The current code base is not uniform. After some discussion,
we have chosen to go with double quotes by default.
|
|
|
|
| |
as value
|
|
|
|
| |
`IPAddr` is used in `OID::Cidr`.
|
|
|
|
|
|
|
|
| |
p PostgreSQLAdapter::OID::Money.precision
# => 19
p PostgreSQLAdapter::OID::Money.new.precision
# => nil
|