| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
| |
after `/' operator"
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since 06ab7b27ea1c1ab357085439abacdb464f6742bf,
`GCSServiceTest#test_signed_URL_response_headers` is broken.
https://travis-ci.org/rails/rails/jobs/460454477#L7084-L7087
This seems to be due to lack of `content_type` at upload.
This is solved by specifying `conten_type`.
However, since the same content is also tested with `test_upload_with_content_type`,
it will be duplicated content, so I think that can remove `test_signed_URL_response_headers`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Force content-type to binary on service urls for relevant content types
We have a list of content types that must be forcibly served as binary,
but in practice this only means to serve them as attachment always. We
should also set the Content-Type to the configured binary type.
As a bonus: add text/cache-manifest to the list of content types to be
served as binary by default.
* Store content-disposition and content-type in GCS
Forcing these in the service_url when serving the file works fine for S3
and Azure, since these services include params in the signature.
However, GCS specifically excludes response-content-disposition and
response-content-type from the signature, which means an attacker can
modify these and have files that should be served as text/plain attachments
served as inline HTML for example. This makes our attempt to force
specific files to be served as binary and as attachment can be easily
bypassed.
The only way this can be forced in GCS is by storing
content-disposition and content-type in the object metadata.
* Update GCS object metadata after identifying blob
In some cases we create the blob and upload the data before identifying
the content-type, which means we can't store that in GCS right when
uploading. In these, after creating the attachment, we enqueue a job to
identify the blob, and set the content-type.
In other cases, files are uploaded to the storage service via direct
upload link. We create the blob before the direct upload, which happens
independently from the blob creation itself. We then mark the blob as
identified, but we have already the content-type we need without having
put it in the service.
In these two cases, then, we need to update the metadata in the GCS
service.
* Include content-type and disposition in the verified key for disk service
This prevents an attacker from modifying these params in the service
signed URL, which is particularly important when we want to force them
to have specific values for security reasons.
* Allow only a list of specific content types to be served inline
This is different from the content types that must be served as binary
in the sense that any content type not in this list will be always
served as attachment but with its original content type. Only types in
this list are allowed to be served either inline or as attachment.
Apart from forcing this in the service URL, for GCS we need to store the
disposition in the metadata.
Fix CVE-2018-16477.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In Ruby 2.3 or later, `String#+@` is available and `+@` is faster than `dup`.
```ruby
# frozen_string_literal: true
require "bundler/inline"
gemfile(true) do
source "https://rubygems.org"
gem "benchmark-ips"
end
Benchmark.ips do |x|
x.report('+@') { +"" }
x.report('dup') { "".dup }
x.compare!
end
```
```
$ ruby -v benchmark.rb
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
Warming up --------------------------------------
+@ 282.289k i/100ms
dup 187.638k i/100ms
Calculating -------------------------------------
+@ 6.775M (± 3.6%) i/s - 33.875M in 5.006253s
dup 3.320M (± 2.2%) i/s - 16.700M in 5.032125s
Comparison:
+@: 6775299.3 i/s
dup: 3320400.7 i/s - 2.04x slower
```
|
| |
|
|
|
|
|
|
|
|
|
| |
`ActiveStorage::Blob#download` and `ActiveStorage::Blob#open` raise
`ActiveStorage::FileNotFoundError` when the corresponding file is missing
from the storage service. Services translate service-specific missing
object exceptions (e.g. `Google::Cloud::NotFoundError` for the GCS service
and `Errno::ENOENT` for the disk service) into
`ActiveStorage::FileNotFoundError`.
|
| |
|
|
|
|
| |
Closes #32530.
|
|
|
|
| |
Sidestep Google Cloud Storage's restrictive per-object rate limit.
|
|
|
|
| |
Ref: https://github.com/ruby/ruby/blob/cc77a811298bd7ab1c422f7f999b93c858327da9/lib/net/http/response.rb#L138-L155
|
|\
| |
| | |
Add ActiveStorage::Service::DiskService#url_for_direct_upload test
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| |
| | |
Trying to pass the current request down to the service so that it can
create full urls instead of paths makes the API messy so use a model
based on ActiveSupport::CurrentAttributes to provide the current host
to services that need it (primarily the disk service).
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If an explicit AWS key pair and/or region is not provided in
config/storage.yml, attempt to use environment variables, shared
credentials, or IAM role credentials. Order of precedence is
determined by the AWS SDK[1].
[1]: https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The Active Storage service for Azure Storage has an option called `path`
that is ambiguous in meaning. It needs to be set to the primary blob
storage endpoint but that can be determined from the blobs client anyway.
To simplify the configuration this commit removes the `path` option and
gets the endpoint from the blobs client instead.
Closes #32225.
|
|/
|
|
| |
Fixes #32129.
|
|
|
|
| |
Fix customizing the download Content-Type for a directly-uploaded blob via a signed URL. See e8286ee.
|
|
|
|
|
| |
Only us-east-1 gives URLs like bucket.s3.amazonaws.com whereas other
regions have URLs like s3-eu-west-1.amazonaws.com/ubxd-rails
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `keyfile` was renamed to `credentials` in `google-cloud-storage` 1.8.0.
https://github.com/GoogleCloudPlatform/google-cloud-ruby/blob/master/google-cloud-storage/CHANGELOG.md#180--2017-11-14
Although `keyfile` can still be used, but it looks like deprecate.
https://github.com/GoogleCloudPlatform/google-cloud-ruby/blob/ddf7b2a856d676316525eb581c1a4cc83ca6097b/google-cloud-storage/lib/google/cloud/storage.rb#L589...L590
Therefore, I think that should use `credentials` in newly generated
applications.
Ref: https://github.com/GoogleCloudPlatform/google-cloud-ruby/issues/1802
|
| |
|
|
|
| |
Closes #31073.
|
|
|
|
| |
https://travis-ci.org/rails/rails/jobs/281044755#L5582-L5586
|
| |
|
|
|
|
| |
activestorage/test/service/configurations.example.yml
|
|
|
|
| |
`content_type` parameter is before `disposition` parameter.
|
| |
|
| |
|
| |
|
| |
|
| |
|
|\
| |
| |
| | |
Support content_type in AzureStorageService#url
|
|/
|
|
|
|
|
|
| |
Add in the content_type option, which is defined as part of the generic
ActiveStorage::Service class.
Without this option, attempts to generate a URL for an attached file
fail with "ArgumentError (unknown keyword: content_type)"
|
|
|
|
|
|
|
|
|
|
| |
This fixes following warnings:
```
test/models/variant_test.rb:11: warning: ambiguous first argument; put parentheses or a space even after `/' operator
lib/active_storage/attached/macros.rb:63: warning: instance variable @active_storage_attached_highlights not initialized
lib/active_storage/attached/macros.rb:25: warning: instance variable @active_storage_attached_avatar not initialized
```
|
| |
|
|
|
|
|
|
|
|
|
| |
If an AWS bucket name includes a `.` (e.g. `bucket.name`), then the canonical
URL for an object will start with "https://s3.amazonaws.com/bucket.name/"
and not with "https://bucket.name.s3.amazonaws.com/".
The URL tests have now been separated into two separate asserts, to ensure
that both the "s3.amazonaws.com" and the "bucket.name" components are included,
but not specifically in that order.
|
|
|
|
|
|
|
|
|
|
|
|
| |
Running `rubocop activestorage` before this commit resulted in 20 offenses.
This commit only fixes:
- Trailing whitespace detected
- Space inside } missing
- Put one space between the method name and the first argument.
The other offenses are left since they are intentional according to
@georgeclaghorn (https://github.com/rails/rails/pull/30061#issuecomment-319999190)
|
|
|
|
|
|
|
|
|
|
|
|
| |
```
% be rubocop -a --only Style/StringLiterals activestorage
Inspecting 74 files
........................................CCCCCCCCCC.C........CC.......C.C..
(snip)
74 files inspected, 31 offenses detected, 31 offenses corrected
```
|
| |
|
|
|
|
|
|
|
| |
"httparty" is only added in #30020 to write two tests to make PUT requests
against S3 and GCS.
The same requests can be made with net/http, removing a dependency from the Gemfile.
|
| |
|
|
git-subtree-dir: activestorage
git-subtree-mainline: 0d58e7e478e79c2d6b2a39a4444d2a17a903b2a6
git-subtree-split: 3f4a7218a4a4923a0e7ce1b2eb0d2888ce30da58
|