Fix Encoding::CompatibilityError when public path is UTF-8

In #5337 we forced the path encoding to ASCII-8BIT to prevent static
file handling from blowing up before an application has had chance to
deal with possibly invalid urls. However this has a negative side
effect of making it an incompatible encoding if the application's
public path has UTF-8 characters in it.

To work around the problem we check to see if the path has a valid
encoding once it has been unescaped. If it is not valid then we can
return early since it will not match any file anyway.

Fixes #13518

8 files changed, 41 insertions, 6 deletions

diff --git a/actionpack/CHANGELOG.md b/actionpack/CHANGELOG.md
index 3e3df19a84..2490282d6e 100644
--- a/actionpack/CHANGELOG.md
+++ b/actionpack/CHANGELOG.md
@@ -1,3 +1,18 @@
+*   Fix `Encoding::CompatibilityError` when public path is UTF-8
+
+    In #5337 we forced the path encoding to ASCII-8BIT to prevent static file handling
+    from blowing up before an application has had chance to deal with possibly invalid
+    urls. However this has a negative side effect of making it an incompatible encoding
+    if the application's public path has UTF-8 characters in it.
+
+    To work around the problem we check to see if the path has a valid encoding once
+    it has been unescaped. If it is not valid then we can return early since it will
+    not match any file anyway.
+
+    Fixes #13518
+
+    *Andrew White*
+
 *   `ActionController::Parameters#permit!` permits hashes in array values.
 
     *Xavier Noria*
diff --git a/actionpack/lib/action_dispatch/middleware/static.rb b/actionpack/lib/action_dispatch/middleware/static.rb
index c6a7d9c415..2764584fe9 100644
--- a/actionpack/lib/action_dispatch/middleware/static.rb
+++ b/actionpack/lib/action_dispatch/middleware/static.rb
@@ -11,9 +11,10 @@ module ActionDispatch
     end
 
     def match?(path)
-      path = path.dup
+      path = unescape_path(path)
+      return false unless path.valid_encoding?
 
-      full_path = path.empty? ? @root : File.join(@root, escape_glob_chars(unescape_path(path)))
+      full_path = path.empty? ? @root : File.join(@root, escape_glob_chars(path))
       paths = "#{full_path}#{ext}"
 
       matches = Dir[paths]
@@ -40,7 +41,6 @@ module ActionDispatch
     end
 
     def escape_glob_chars(path)
-      path.force_encoding('binary') if path.respond_to? :force_encoding
       path.gsub(/[*?{}\[\]]/, "\\\\\\&")
     end
   end
diff --git a/actionpack/test/dispatch/static_test.rb b/actionpack/test/dispatch/static_test.rb
index acccbcb2e6..d83461e52f 100644
--- a/actionpack/test/dispatch/static_test.rb
+++ b/actionpack/test/dispatch/static_test.rb
@@ -137,7 +137,7 @@ module StaticTests
     end
 
     def with_static_file(file)
-      path = "#{FIXTURE_LOAD_PATH}/public" + file
+      path = "#{FIXTURE_LOAD_PATH}/#{public_path}" + file
       File.open(path, "wb+") { |f| f.write(file) }
       yield file
     ensure
@@ -149,11 +149,24 @@ class StaticTest < ActiveSupport::TestCase
   DummyApp = lambda { |env|
     [200, {"Content-Type" => "text/plain"}, ["Hello, World!"]]
   }
-  App = ActionDispatch::Static.new(DummyApp, "#{FIXTURE_LOAD_PATH}/public", "public, max-age=60")
 
   def setup
-    @app = App
+    @app = ActionDispatch::Static.new(DummyApp, "#{FIXTURE_LOAD_PATH}/public", "public, max-age=60")
+  end
+
+  def public_path
+    "public"
   end
 
   include StaticTests
 end
+
+class StaticEncodingTest < StaticTest
+  def setup
+    @app = ActionDispatch::Static.new(DummyApp, "#{FIXTURE_LOAD_PATH}/公共", "public, max-age=60")
+  end
+
+  def public_path
+    "公共"
+  end
+end
diff --git a/actionpack/test/fixtures/公共/foo/bar.html b/actionpack/test/fixtures/公共/foo/bar.html
new file mode 100644
index 0000000000..9a35646205
--- /dev/null
+++ b/actionpack/test/fixtures/公共/foo/bar.html
@@ -0,0 +1 @@
+/foo/bar.html
\ No newline at end of file
diff --git a/actionpack/test/fixtures/公共/foo/baz.css b/actionpack/test/fixtures/公共/foo/baz.css
new file mode 100644
index 0000000000..b5173fbef2
--- /dev/null
+++ b/actionpack/test/fixtures/公共/foo/baz.css
@@ -0,0 +1,3 @@
+body {
+background: #000;
+}
diff --git a/actionpack/test/fixtures/公共/foo/index.html b/actionpack/test/fixtures/公共/foo/index.html
new file mode 100644
index 0000000000..497a2e898f
--- /dev/null
+++ b/actionpack/test/fixtures/公共/foo/index.html
@@ -0,0 +1 @@
+/foo/index.html
\ No newline at end of file
diff --git a/actionpack/test/fixtures/公共/foo/こんにちは.html b/actionpack/test/fixtures/公共/foo/こんにちは.html
new file mode 100644
index 0000000000..1df9166522
--- /dev/null
+++ b/actionpack/test/fixtures/公共/foo/こんにちは.html
@@ -0,0 +1 @@
+means hello in Japanese
diff --git a/actionpack/test/fixtures/公共/index.html b/actionpack/test/fixtures/公共/index.html
new file mode 100644
index 0000000000..525950ba6b
--- /dev/null
+++ b/actionpack/test/fixtures/公共/index.html
@@ -0,0 +1 @@
+/index.html
\ No newline at end of file