apalis icon indicating copy to clipboard operation
apalis copied to clipboard

Adding `RateLimitLayer` results in a compilation error

Open IgnisDa opened this issue 10 months ago • 11 comments

I am not sure why the error happens, but as soon as I remove the RateLimitLayer, the error disappears.

image

Here is what the code looks like:

           .register_with_count(3, {
                WorkerBuilder::new("perform_application_job")
                    .layer(ApalisRateLimitLayer::new(
                        rate_limit_num,
                        Duration::new(5, 0),
                    ))
                    .layer(ApalisTraceLayer::new())
                    .data(importer_service_1.clone())
                    .data(exporter_service_1.clone())
                    .data(media_service_4.clone())
                    .data(exercise_service_1.clone())
                    .with_storage(perform_application_job_storage.clone())
                    .build_fn(perform_application_job)
            })

IgnisDa avatar Apr 21 '24 17:04 IgnisDa

Tried changing the order of layers, count from 3 -> 1. Did not work.

IgnisDa avatar Apr 21 '24 17:04 IgnisDa

https://github.com/IgnisDa/ryot/blob/b9019dfa1f5948c071857a076ac7e14146d592d8/apps/backend/src/main.rs#L310-L322 are the lines if you are interested.

IgnisDa avatar Apr 21 '24 17:04 IgnisDa

Could you try to check clone and check service clone? https://docs.rs/tower/latest/tower/builder/struct.ServiceBuilder.html#method.check_clone

You can access the service builder using WorkerBuilder.chain

geofmureithi avatar Apr 22 '24 05:04 geofmureithi

@geofmureithi I am not sure what I need to do with the methods you gave. I added them to my code, it didn't change the error.

IgnisDa avatar Apr 22 '24 05:04 IgnisDa

Have you checked this issue? https://github.com/geofmureithi/apalis/discussions/256

geofmureithi avatar Apr 22 '24 05:04 geofmureithi

See also https://github.com/tokio-rs/axum/discussions/987#discussioncomment-2678595

The issue is that from 0.5 apalis needs the service to be clone yet tower rate limit is not clone.

The suggested option is to include BufferLayer infront of the rate limit one.

geofmureithi avatar Apr 22 '24 06:04 geofmureithi

The suggested option is to include BufferLayer infront of the rate limit one.

I did that, still does not work. The error has changed: image

IgnisDa avatar Apr 22 '24 06:04 IgnisDa

Seems the issue here is with tower. Can you try using .chain Can you also try using a custom implementation eg https://medium.com/@khalludi123/creating-a-rate-limiter-middleware-using-tower-for-axum-rust-be1d65fbeca

geofmureithi avatar Apr 22 '24 06:04 geofmureithi

Got it working with:

            .register_with_count(
                1,
                WorkerBuilder::new("perform_application_job")
                    .data(importer_service_1.clone())
                    .data(exporter_service_1.clone())
                    .data(media_service_4.clone())
                    .data(exercise_service_1.clone())
                    .with_storage(perform_application_job_storage.clone())
                    .chain(|s| {
                        s.layer(BufferLayer::new(1024))
                            .layer(ApalisRateLimitLayer::new(
                                rate_limit_num,
                                Duration::new(5, 0),
                            ))
                            .layer(ApalisTraceLayer::new())
                    })
                    .build_fn(perform_application_job),
            )

Should I keep this issue open? (Since I had to use a workaround).

IgnisDa avatar Apr 22 '24 06:04 IgnisDa

I will check the problem then close it.

geofmureithi avatar Apr 22 '24 07:04 geofmureithi

Cool! Thanks for your guidance.

IgnisDa avatar Apr 22 '24 07:04 IgnisDa

@geofmureithi Any updates on this?

IgnisDa avatar May 24 '24 11:05 IgnisDa

@geofmureithi Sorry for the ping. Were you able to find the problem?

IgnisDa avatar Jun 29 '24 17:06 IgnisDa

Currently I have the register method able to work without clone. The issue though is when using register_with_count, I am trying to figure something. Will update in upcoming 0.6 releases.

geofmureithi avatar Jul 02 '24 20:07 geofmureithi

This has been fixed by #348

geofmureithi avatar Jul 04 '24 18:07 geofmureithi

@geofmureithi Thanks. Can you cut a new release?

IgnisDa avatar Jul 05 '24 03:07 IgnisDa

There is v0.6.0-rc.1

geofmureithi avatar Jul 05 '24 04:07 geofmureithi