aboutsummaryrefslogtreecommitdiffstats
path: root/actiontext
diff options
context:
space:
mode:
authorCliff Pruitt <cliff.pruitt@cliffpruitt.com>2019-07-26 11:02:40 -0400
committerCliff Pruitt <cliff.pruitt@cliffpruitt.com>2019-07-26 12:18:01 -0400
commit05633d02d8ac8aa1289c0a01872e13e9b2449cd5 (patch)
tree952cb2760ca0877ce37b9449f930850ef67b5469 /actiontext
parent369daa530dd9db7bbba79b8c75012e0fa84c9f48 (diff)
downloadrails-05633d02d8ac8aa1289c0a01872e13e9b2449cd5.tar.gz
rails-05633d02d8ac8aa1289c0a01872e13e9b2449cd5.tar.bz2
rails-05633d02d8ac8aa1289c0a01872e13e9b2449cd5.zip
Handle GB18030 strings with invalid characters in transliterate
GB18030 is Unicode compatible and covers all Unicode code points so we can temporarily convert GB18030 strings to UTF-8 to perform the transliteration. After transliterating we want to convert back to GB18030. In all cases of transcoding, we replace invalid or undefined characters with the default replacement character ("?"). This is in line with the behavior of tidy_bytes which is used on the UTF-8 string before transliterating.
Diffstat (limited to 'actiontext')
0 files changed, 0 insertions, 0 deletions