BlobKey, MirrorService and Sidekiq 8 are incompatible
Bug report for Cloudinary Ruby SDK
Describe the bug in a sentence or two.
When using the Cloudinary ActiveStorage service in a mirror with S3 and Sidekiq 8 as a job queue, I am getting errors of the form
Job arguments to ActiveStorage::MirrorJob must be native JSON types, but "1xfiunhqeexf8h3pkdxucnby5eou" is a ActiveStorage::BlobKey
Issue Type (Can be multiple)
- [ ] Build - Cannot install or import the SDK
- [ ] Performance - Performance issues
- [X] Behaviour - Functions are not working as expected (such as generate URL)
- [ ] Documentation - Inconsistency between the docs and behaviour
- [ ] Other (Specify)
Steps to reproduce
- Use Sidekiq 8 with strict_args enabled (default)
- Define an S3 service and a Cloudinary Service in storage.yml
- Define a mirror as
mirror:
service: Mirror
primary: amazon
mirrors:
- cloudinary
- Attempt to upload a file
- ActiveStorage will attempt to enqueue ActiveStorage::MirrorJob which takes
keyas an arg. But because the Cloudinary gem is redefining the key as aBlobKeywe get the above error from Sidekiq because BlobKey is not a native JSON type.
Error screenshots or Stack Trace (if applicable)
lib/active_storage/service/mirror_service.rb:55
A minimal repro can be achieved with the following:
> s = ActiveStorage::Service::MirrorService.new(primary: nil, mirrors: [])
=> #<ActiveStorage::Service::MirrorService:0x0000ffff69a5d2f0 @mirrors=[], @primary=nil>
> k = ActiveStorage::BlobKey.new(key: "test")
=> "test"
> s.mirror_later(k, checksum: nil)
INFO 2025-05-15T14:47:39.511Z pid=955 tid=25v: Sidekiq 8.0.3 connecting to Redis with options {size: 10, pool_name: "internal", url: "redis://redis:6379/0"}
Failed enqueuing ActiveStorage::MirrorJob to Sidekiq(default): ArgumentError (Job arguments to ActiveStorage::MirrorJob must be native JSON types, but "test" is a ActiveStorage::BlobKey.
See https://github.com/sidekiq/sidekiq/wiki/Best-Practices
To disable this error, add `Sidekiq.strict_args!(false)` to your initializer.
This monkey patch to ActiveStorage can address the issue, but I dont know whether rails would want this in their core code when the expect the key to be a string anyway
module ActiveStorage
class Service::MirrorService < Service
def mirror_later(key, checksum:) # :nodoc:
ActiveStorage::MirrorJob.perform_later key.to_s, checksum: checksum
end
end
end
Operating System
- [X] Linux
- [ ] Windows
- [ ] macOS
- [ ] All
Environment and Libraries (fill in the version numbers)
- Cloudinary Ruby SDK version - 2.3.0
- Ruby Version - 3.4.3
- Rails Version - 7.2.2.1
- Other Libraries - ActiveStorage 7.2.2.1
@adampope we need to research thsi but did you try this:
To disable this error, add Sidekiq.strict_args!(false) to your initializer.
@tommyg-cld I did not. That would definitely work as it restores the old behaviour but goes against the default and recommended way to use Sidekiq.
From the sidekiq 7.0 release notes
Strict argument checking is now enabled by default. Sidekiq will raise an error if you pass a Symbol or generic Ruby object into perform_async because they do not serialize correctly into JSON. This has long been a source of frustration and debugging time for users, better to be upfront about this design decision. From the beginning, Sidekiq has used plain JSON as its job format, for interoperability with all programming languages and environments.
My monkey patch has limited the scope of the fix to Cloudinary/Active Storage, whereas disabling strict args could allow more errors in across the code base.
@adampope we'll investigate further and update you