aboutsummaryrefslogtreecommitdiffstats
path: root/activesupport/lib/active_support/inflector/transliterate.rb
Commit message (Collapse)AuthorAgeFilesLines
* Prevent error on transliterate with frozen strings.Cliff Pruitt2019-07-311-0/+1
| | | | ActiveSupport::Inflector.transliterate mutates strings by changing encodings. Prior to this commit passing a frozen string would raise a `FrozenError`. This change duplicates the internal string, if frozen, before transliterating.
* [ci skip] Fix unclosed tags in `Inflector` docsvzvu3k6k2019-07-301-1/+1
|
* Handle GB18030 strings with invalid characters in transliterateCliff Pruitt2019-07-261-6/+14
| | | | | | GB18030 is Unicode compatible and covers all Unicode code points so we can temporarily convert GB18030 strings to UTF-8 to perform the transliteration. After transliterating we want to convert back to GB18030. In all cases of transcoding, we replace invalid or undefined characters with the default replacement character ("?"). This is in line with the behavior of tidy_bytes which is used on the UTF-8 string before transliterating.
* Handle US-ASCII strings with invalid characters in transliterateCliff Pruitt2019-07-261-2/+15
| | | | US-ASCII is a subset of UTF-8 so we can temporarily convert US-ASCII strings to UTF-8 to perform the transliteration. After we've converted characters to ASCII representations, we can set the encoding back to US-ASCII to return the same encoding we accepted.
* Raise errors for ASCII-8BIT encoding in ActiveSupport::Inflector::transliterateCliff Pruitt2019-07-261-0/+4
| | | | Adds ArgumentErrors to `ActiveSupport::Inflector::transliterate` if a string is with ASCII-8BIT which will raise an error in `unicode_normalize`.
* Revert "Make UTF-8 string requirement explicit for ↵Cliff Pruitt2019-07-171-7/+2
| | | | `ActiveSupport::Inflector.transliterate`"
* Make UTF-8 string requirement explicit for `transliterate`Cliff Pruitt2019-07-161-2/+7
| | | | | | It's noted in #34062 that String#parameterize will raise an `Encoding::CompatibilityError` if the string is not UTF-8 encoded. The error is raised as a result of passing the string to `.unicode_normalize`. This PR raises a higher level `ArgumentError` if the provided string is not UTF-8 and updates documentation to note the encoding requirement.
* Update docs for 'parameterize()' [ci skip]Sharang Dashputre2019-03-161-5/+9
|
* Fix bug with parametrize when `locale` is passedSharang Dashputre2019-03-121-1/+1
| | | | Also add tests for parametrize and transliterate
* Add locale option to parameterizeKaan Ozkan2019-03-111-8/+7
| | | | | | Parameterize is triggering I18n#transliterate. This method already accepts a locale. It would be cleaner if similar to other string inflection methods #parameterize also accepted 'locale' as a parameter.
* Deprecate Unicode#normalize and Chars#normalize (#34202)Francesco Rodríguez2018-10-121-3/+3
|
* Add `Style/RedundantFreeze` to remove redudant `.freeze`Yasuo Honda2018-09-291-3/+3
| | | | | | | | | | | | | | | | | | | | | Since Rails 6.0 will support Ruby 2.4.1 or higher `# frozen_string_literal: true` magic comment is enough to make string object frozen. This magic comment is enabled by `Style/FrozenStringLiteralComment` cop. * Exclude these files not to auto correct false positive `Regexp#freeze` - 'actionpack/lib/action_dispatch/journey/router/utils.rb' - 'activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb' It has been fixed by https://github.com/rubocop-hq/rubocop/pull/6333 Once the newer version of RuboCop released and available at Code Climate these exclude entries should be removed. * Replace `String#freeze` with `String#-@` manually if explicit frozen string objects are required - 'actionpack/test/controller/test_case_test.rb' - 'activemodel/test/cases/type/string_test.rb' - 'activesupport/lib/active_support/core_ext/string/strip.rb' - 'activesupport/test/core_ext/string_ext_test.rb' - 'railties/test/generators/actions_test.rb'
* Update incorrect backtick usage in RDoc to teletypeT.J. Schuck2017-11-221-2/+2
| | | [ci skip]
* Fix french spelling mistake Skander2017-11-061-6/+6
| | | | Trés -> Très https://fr.wiktionary.org/wiki/tr%C3%A8s
* [ci skip] show the correct example to demonstrate inflections.Aditya Kapoor2017-11-011-1/+7
|
* [Active Support] require_relative => requireAkira Matsuda2017-10-211-2/+2
| | | | This basically reverts 8da30ad6be34339124ba4cb4e36aea260dda12bc
* Enable `Layout/FirstParameterIndentation` copRyuta Kamizono2017-07-171-3/+4
| | | | | | | We have some indentation cops. But now there is a little inconsistent params indentations. Enable `Layout/FirstParameterIndentation` cop to prevent newly inconsistent indentation added and auto-correct to existing violations.
* [Active Support] `rubocop -a --only Layout/EmptyLineAfterMagicComment`Koichi ITO2017-07-111-0/+1
|
* Use frozen-string-literal in ActiveSupportKir Shatrov2017-07-091-0/+1
|
* [Active Support] require => require_relativeAkira Matsuda2017-07-011-2/+2
|
* Raise ArgumentError if attempting to transliterate anything that is not a stringKevin McPhillips2017-01-161-0/+2
|
* Remove deprecated separator argument from parameterizeAndrew White2016-11-141-5/+1
|
* applies remaining conventions across the projectXavier Noria2016-08-061-1/+0
|
* normalizes indentation and whitespace across the projectXavier Noria2016-08-061-1/+1
|
* modernizes hash syntax in activesupportXavier Noria2016-08-061-1/+1
|
* applies new string literal convention in activesupport/libXavier Noria2016-08-061-4/+4
| | | | | The current code base is not uniform. After some discussion, we have chosen to go with double quotes by default.
* Parameterize with options to preserve case of stringSwaathi K2015-11-071-8/+23
| | | | | | | | | | | | | | | | | | | | | | | | Added test cases Using kwargs instead of three seperate functions Updated parameterize in transliterate.rb Updated parameterize in transliterate.rb Added deprecation warnings and updating RDoc+Guide Misspelled separtor. Fixed. Deprecated test cases and added support to parameterize with keyword parameters Squashing commits. Fixed test cases and added deprecated test cases Small changes to Gemfile.lock and CHANGELOG Update Gemfile.lock
* code gardening in transliterate.rbXavier Noria2015-10-071-2/+5
| | | | | | | | | Saw this while doing a review of a patch: * Normalize case and punctuation across comments. * ascii -> ASCII * Since I was on it, some blank lines that visually add some clarity IMO.
* s/seperator/separator/gAkira Matsuda2015-09-201-3/+3
|
* File encoding is defaulted to utf-8 in Ruby >= 2.1Akira Matsuda2015-09-181-1/+0
|
* Decrease allocations in transliterateschneems2015-07-301-5/+13
| | | | | | We can save a few objects by freezing the `replacement` string. We save a few more by down-casing the string in memory instead of allocating a new one. We save far more objects by checking for the default separator `"-"`, and using pre-generated regular expressions. We will save 209,231 bytes and 1,322 objects.
* Freeze string literals when not mutated.schneems2015-07-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I wrote a utility that helps find areas where you could optimize your program using a frozen string instead of a string literal, it's called [let_it_go](https://github.com/schneems/let_it_go). After going through the output and adding `.freeze` I was able to eliminate the creation of 1,114 string objects on EVERY request to [codetriage](codetriage.com). How does this impact execution? To look at memory: ```ruby require 'get_process_mem' mem = GetProcessMem.new GC.start GC.disable 1_114.times { " " } before = mem.mb after = mem.mb GC.enable puts "Diff: #{after - before} mb" ``` Creating 1,114 string objects results in `Diff: 0.03125 mb` of RAM allocated on every request. Or 1mb every 32 requests. To look at raw speed: ```ruby require 'benchmark/ips' number_of_objects_reduced = 1_114 Benchmark.ips do |x| x.report("freeze") { number_of_objects_reduced.times { " ".freeze } } x.report("no-freeze") { number_of_objects_reduced.times { " " } } end ``` We get the results ``` Calculating ------------------------------------- freeze 1.428k i/100ms no-freeze 609.000 i/100ms ------------------------------------------------- freeze 14.363k (± 8.5%) i/s - 71.400k no-freeze 6.084k (± 8.1%) i/s - 30.450k ``` Now we can do some maths: ```ruby ips = 6_226k # iterations / 1 second call_time_before = 1.0 / ips # seconds per iteration ips = 15_254 # iterations / 1 second call_time_after = 1.0 / ips # seconds per iteration diff = call_time_before - call_time_after number_of_objects_reduced * diff * 100 # => 0.4530373333993266 miliseconds saved per request ``` So we're shaving off 1 second of execution time for every 220 requests. Is this going to be an insane speed boost to any Rails app: nope. Should we merge it: yep. p.s. If you know of a method call that doesn't modify a string input such as [String#gsub](https://github.com/schneems/let_it_go/blob/b0e2da69f0cca87ab581022baa43291cdf48638c/lib/let_it_go/core_ext/string.rb#L37) please [give me a pull request to the appropriate file](https://github.com/schneems/let_it_go/blob/b0e2da69f0cca87ab581022baa43291cdf48638c/lib/let_it_go/core_ext/string.rb#L37), or open an issue in LetItGo so we can track and freeze more strings. Keep those strings Frozen ![](https://www.dropbox.com/s/z4dj9fdsv213r4v/let-it-go.gif?dl=1)
* Fix docs for ActiveSupport::Inflector methodsclaudiob2015-01-031-12/+2
| | | | | | | | | | | | | | | Many methods in ActiveSupport::Inflector were actually documenting the String methods with the same name. For instance the doc for `camelize` said: > If the argument to +camelize+ is set to <tt>:lower</tt>... while it should say: > If the +uppercase_first_letter+ parameter is set to false [ci skip]
* update AS/inflector docs [ci skip]Francesco Rodriguez2012-09-141-5/+6
|
* update some AS code examples to 1.9 hash syntax [ci skip]Francesco Rodriguez2012-09-121-11/+11
|
* Revert "Merge pull request #6354 from lest/patch-1"Carlos Galdino + Rafael Mendonça França2012-05-161-1/+1
| | | | | | | This reverts commit e8feaff60b9c04d34ad234f7d17b5d2ad9cc7a24, reversing changes made to 9adf28c026070afb78b80027521a4ddddd68d697. Reason: This broke the actionmailer tests
* remove unnecessary requireSergey Nartimov2012-05-161-1/+1
| | | | | - access & filters don't use multibyte ext - transliterate requires only AS::Multibyte but not multibyte ext
* removing unnecessary 'examples' noise from activesupportFrancesco Rodriguez2012-05-131-2/+0
|
* require i18n in transliterate so it can run in isolated situationsJack Dempsey2010-09-141-0/+1
|
* Use multibyte proxy class on 1.9, refactor Unicode.Norman Clarke2010-05-211-2/+3
| | | | | | | | | | Makes String#mb_chars on Ruby 1.9 return an instance of ActiveSupport::Multibyte::Chars to work around 1.9's lack of Unicode case folding. Refactors class methods from ActiveSupport::Multibyte::Chars into new Unicode module, adding other related functionality for consistency. [#4594 state:resolved] Signed-off-by: Jeremy Kemper <jeremy@bitsweat.net>
* fixes colon in previous YAML exampleXavier Noria2010-05-031-1/+1
|
* Fix transliteration rule example in docs. [#4526 state:resolved]Norman Clarke2010-05-031-2/+3
| | | | Signed-off-by: Xavier Noria <fxn@hashref.com>
* Delegate Inflector.transliterate to i18n. [#4508 state:resolved]Norman Clarke2010-04-301-38/+56
| | | | | | | Ancillary changes: Moved Chars#normalize into a class method; removed unused UTF_PAT constant. Signed-off-by: José Valim <jose.valim@gmail.com>
* Improve reliability of Inflector.transliterate. [#4374 state:resolved]Norman Clarke2010-04-121-24/+37
| | | | Signed-off-by: Jeremy Kemper <jeremy@bitsweat.net>
* Parameterize should accept malformed utf8 characters [#4323 state:resolved]Kristopher Murata2010-04-041-1/+4
| | | | Signed-off-by: Jeremy Kemper <jeremy@bitsweat.net>
* Any reason we can't just use the slow 1.8 path for transliteration in 1.9?wycats2010-03-171-2/+2
|
* Removed the + sign as an accepted character from the parameterize method, as ↵Stijn Mathysen2010-03-051-1/+1
| | | | | | | | a + sign is interpreted by the browser as a space, possibly resulting in a "ArgumentError: illegal character in key" [#4080 state:committed] Signed-off-by: Jeremy Kemper <jeremy@bitsweat.net>
* Break up inflector to reduce the dependency burden on dependency-les methods ↵Yehuda Katz2009-11-071-0/+61
like constantize.
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-29 16:14:29 +0000