aboutsummaryrefslogtreecommitdiffstats
path: root/activesupport/lib/active_support/multibyte
Commit message (Collapse)AuthorAgeFilesLines
* [Active Support] require_relative => requireAkira Matsuda2017-10-211-5/+5
| | | | This basically reverts 8da30ad6be34339124ba4cb4e36aea260dda12bc
* [Active Support] `rubocop -a --only Layout/EmptyLineAfterMagicComment`Koichi ITO2017-07-112-0/+2
|
* Use frozen-string-literal in ActiveSupportKir Shatrov2017-07-092-0/+2
|
* [Active Support] require => require_relativeAkira Matsuda2017-07-011-5/+5
|
* Define path with __dir__bogdanvlviv2017-05-231-1/+1
| | | | | | ".. with __dir__ we can restore order in the Universe." - by @fxn Related to 5b8738c2df003a96f0e490c43559747618d10f5f
* Fix doc in Multibyte::Chars [ci skip]kenta-s2017-02-171-4/+5
|
* Update Unicode Version to 9.0.0Fumiaki MATSUSHIMA2017-01-281-8/+18
| | | | | | | | | | | | | | | | | | | | | | | | | 9.0.0 was released on June 21, 2016 http://blog.unicode.org/2016/06/announcing-unicode-standard-version-90.html http://www.unicode.org/versions/Unicode9.0.0/ There are some changes about grapheme cluster in Unicode 9.0.0: http://unicode.org/reports/tr29/#Grapheme_Cluster_Boundary_Rules ------------ I noticed that `unpack_graphemes` returns [Other] when the argument is Other ÷ Prepend (it must be [Other, Prepend]). But in [Unicode 8.0.0's Prepend has no characters](http://www.unicode.org/reports/tr29/tr29-27.html#Prepend) so we don't have to backport following patch: ```diff should_break = + if pos == eoc + true ```
* Privatize unneededly protected methods in Active SupportAkira Matsuda2016-12-241-3/+3
|
* No need to nodoc private methodsAkira Matsuda2016-12-241-1/+1
|
* Merge pull request #26905 from bogdanvlviv/docsAndrew White2016-11-131-1/+1
|\ | | | | Add missing `+` around a some literals.
| * Add missing `+` around a some literals.bogdanvlviv2016-10-271-1/+1
| | | | | | | | | | | | Mainly around `nil` [ci skip]
* | Add more rubocop rules about whitespacesRafael Mendonça França2016-10-291-14/+14
|/
* Remove dead constantsFumiaki MATSUSHIMA2016-09-061-30/+0
| | | | It seems that we forgot to remove some codes on https://github.com/rails/rails/commit/7ab47751068c6480e7e44fc9265a7e690dd4af3b
* fixes remaining RuboCop issues [Vipul A M, Xavier Noria]Xavier Noria2016-09-011-12/+12
|
* Add three new rubocop rulesRafael Mendonça França2016-08-162-2/+2
| | | | | | | | Style/SpaceBeforeBlockBraces Style/SpaceInsideBlockBraces Style/SpaceInsideHashLiteralBraces Fix all violations in the repository.
* applies remaining conventions across the projectXavier Noria2016-08-061-10/+9
|
* normalizes indentation and whitespace across the projectXavier Noria2016-08-061-17/+17
|
* modernizes hash syntax in activesupportXavier Noria2016-08-061-1/+1
|
* applies new string literal convention in activesupport/libXavier Noria2016-08-062-19/+19
| | | | | The current code base is not uniform. After some discussion, we have chosen to go with double quotes by default.
* systematic revision of =~ usage in ASXavier Noria2016-07-221-1/+2
| | | | | Where appropriate prefer the more concise Regexp#match?, String#include?, String#start_with?, and String#end_with?
* Merge pull request #12877 from aroben/extended-graphemesRafael França2015-12-311-13/+38
|\ | | | | Support extended grapheme clusters and UAX 29
| * Support extended grapheme clusters and UAX 29Adam Roben2013-11-131-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | http://www.unicode.org/reports/tr29/tr29-21.html is the version of UAX 29 that corresponds to Unicode 6.2.0. Unicode.unpack_graphemes now implements all the rules listed there, including the ones for extended grapheme clusters. I added a new optional test, test/multibyte_grapheme_break_conformance.rb, that is heavily based on test/multibyte_normalization_conformance.rb, which runs the Unicode test suite.
| * Refactor Unicode.unpack_graphemes slightlyAdam Roben2013-11-131-13/+23
| | | | | | | | This will make it easier to add the rest of the rules listed in UAX 29.
* | Update #20737 to address feedbackSean Griffin2015-10-201-2/+5
| | | | | | | | | | | | | | Given that this pull request affects a mutable value, we need to test for and document the affects on the receiver in this case. Additionally, this pull request was missing a CHANGELOG entry.
* | Fixed slice! behavior: return nil for out-of-bound parametersGourav Tiwari2015-10-201-1/+2
| |
* | [ci skip] default_normalization_form accessing from UnicodeGaurav Sharma2015-09-291-1/+1
| |
* | File encoding is defaulted to utf-8 in Ruby >= 2.1Akira Matsuda2015-09-182-2/+0
| |
* | Update Unicode Version to 8.0.0Anshul Sharma2015-09-041-1/+1
| |
* | replace each with each_key when only the key is neededAaron Lasseigne2015-08-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using each_key is faster and more intention revealing. Calculating ------------------------------------- each 31.378k i/100ms each_key 33.790k i/100ms ------------------------------------------------- each 450.225k (± 7.0%) i/s - 2.259M each_key 494.459k (± 6.3%) i/s - 2.467M Comparison: each_key: 494459.4 i/s each: 450225.1 i/s - 1.10x slower
* | String#freeze optimizationsschneems2015-07-301-1/+1
| |
* | Freeze string literals when not mutated.schneems2015-07-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I wrote a utility that helps find areas where you could optimize your program using a frozen string instead of a string literal, it's called [let_it_go](https://github.com/schneems/let_it_go). After going through the output and adding `.freeze` I was able to eliminate the creation of 1,114 string objects on EVERY request to [codetriage](codetriage.com). How does this impact execution? To look at memory: ```ruby require 'get_process_mem' mem = GetProcessMem.new GC.start GC.disable 1_114.times { " " } before = mem.mb after = mem.mb GC.enable puts "Diff: #{after - before} mb" ``` Creating 1,114 string objects results in `Diff: 0.03125 mb` of RAM allocated on every request. Or 1mb every 32 requests. To look at raw speed: ```ruby require 'benchmark/ips' number_of_objects_reduced = 1_114 Benchmark.ips do |x| x.report("freeze") { number_of_objects_reduced.times { " ".freeze } } x.report("no-freeze") { number_of_objects_reduced.times { " " } } end ``` We get the results ``` Calculating ------------------------------------- freeze 1.428k i/100ms no-freeze 609.000 i/100ms ------------------------------------------------- freeze 14.363k (± 8.5%) i/s - 71.400k no-freeze 6.084k (± 8.1%) i/s - 30.450k ``` Now we can do some maths: ```ruby ips = 6_226k # iterations / 1 second call_time_before = 1.0 / ips # seconds per iteration ips = 15_254 # iterations / 1 second call_time_after = 1.0 / ips # seconds per iteration diff = call_time_before - call_time_after number_of_objects_reduced * diff * 100 # => 0.4530373333993266 miliseconds saved per request ``` So we're shaving off 1 second of execution time for every 220 requests. Is this going to be an insane speed boost to any Rails app: nope. Should we merge it: yep. p.s. If you know of a method call that doesn't modify a string input such as [String#gsub](https://github.com/schneems/let_it_go/blob/b0e2da69f0cca87ab581022baa43291cdf48638c/lib/let_it_go/core_ext/string.rb#L37) please [give me a pull request to the appropriate file](https://github.com/schneems/let_it_go/blob/b0e2da69f0cca87ab581022baa43291cdf48638c/lib/let_it_go/core_ext/string.rb#L37), or open an issue in LetItGo so we can track and freeze more strings. Keep those strings Frozen ![](https://www.dropbox.com/s/z4dj9fdsv213r4v/let-it-go.gif?dl=1)
* | Merge pull request #20297 from gouravtiwari/patch-9Claudio B.2015-05-261-0/+6
|\ \ | | | | | | Added multibyte slice! example to doc [ci skip]
| * | Added multibyte slice! example to doc [ci skip]Gourav Tiwari2015-05-261-0/+6
| | |
* | | Remove redundant 'like' from doc of slice! method [ci skip]Mehmet Emin İNAÇ2015-05-261-1/+1
|/ /
* | String already respond_to scrub at Ruby 2.2Rafael Mendonça França2015-01-041-2/+1
| |
* | Update to Unicode 7.0.0Benjamin Fleischer2014-11-151-1/+1
| | | | | | | | | | | | | | | | 7.0.0 was released on June 16, 2014 http://unicode-inc.blogspot.com.ar/2014/10/unicode-version-70-complete-text-of.html ruby bin/generate_tables
* | As of Unicode 6.3, Mongolian Vowel Separator is not whitespaceMatthew Draper2014-09-151-1/+0
| | | | | | | | | | Ruby 2.2 knows this, and no longer matches it with [[:space:]], so it's not a good candidate for testing String#squish.
* | Preload UnicodeDatabase outside the loopAkira Matsuda2014-08-181-0/+1
| | | | | | | | | | | | This fixes random multibyte_chars_test fail under Ruby 1.9.3. I don't know why the tests fail. And I really don't know why this fixes. Maybe we need some more investigation...
* | formatAkira Matsuda2014-08-181-2/+1
| |
* | Prevent using String#scrub on RubiniusRobin Dupret2014-07-301-1/+2
| | | | | | | | | | | | Rubinius' has built-in support for String#scrub but it doesn't have yet support for ASCII-incompatible chars so for now, we should rely on the old implementation of #tidy_bytes.
* | Fix tidy_bytes for JRubyJustin Coyne2014-02-101-3/+3
| | | | | | | | | | The previous implementation was broken because JRuby (1.7.10) doesn't have a code converter for UTF-8 to UTF8-MAC.
* | use feature detection to decide which implementation to useAaron Patterson2014-02-081-1/+1
| | | | | | | | Decouple the code from the particular Ruby version.
* | Update to Unicode 6.3.0Norman Clarke2013-12-271-1/+1
| | | | | | | | | | | | 6.3.0 was released on September 30, 2013. http://unicode-inc.blogspot.com.ar/2013/09/announcing-unicode-standard-version-63.html
* | Use String#scrub when available to tidy bytesNorman Clarke2013-12-261-35/+35
|/
* Initializing Codepoint object with default valuesHitendra Singh2013-09-201-0/+7
|
* Drying up method_missing codeHitendra Singh2013-09-201-2/+1
|
* compatability => compatibilityVipul A M2013-05-261-3/+3
|
* Use ruby's Encoding support for tidy_bytesBurke Libbey2013-05-081-39/+19
| | | | | | | | | | The previous implementation was quite slow. This leverages some of the transcoding abilities built into Ruby 1.9 instead. It is roughly 96% faster. The roundtrip through UTF_8_MAC here is because ruby won't let you transcode from UTF_8 to UTF_8. I chose the closest encoding I could find as an intermediate.
* Update to latest Unicode data.Norman Clarke2013-02-101-1/+1
| | | | Release notes at: http://www.unicode.org/versions/Unicode6.2.0/
* Revert "Use flat_map { } instead of map {}.flatten"Santiago Pastorino2012-10-051-2/+2
| | | | | | | | | | | This reverts commit abf8de85519141496a6773310964ec03f6106f3f. We should take a deeper look to those cases flat_map doesn't do deep flattening. irb(main):002:0> [[[1,3], [1,2]]].map{|i| i}.flatten => [1, 3, 1, 2] irb(main):003:0> [[[1,3], [1,2]]].flat_map{|i| i} => [[1, 3], [1, 2]]
generated by cgit v1.2.3 (git 2.25.1) at 2024-10-30 01:48:33 +0000