hawkbit icon indicating copy to clipboard operation
hawkbit copied to clipboard

hawkbit-ddi-resource: do not log range requests

Open zyga opened this issue 2 years ago • 5 comments

HawkBit creates ActionStatus entries for each GET request associated with an artefact of an assigned software module. Amongst others, this severely breaks RAUC block-based adaptive updates [1], which issue a flurry of range requests for small chunks, so that the whole image does not have to be streamed in case of incremental system changes.

[1] https://rauc.readthedocs.io/en/latest/advanced.html#block-based-adaptive-update-block-hash-index

Fixes: https://github.com/eclipse/hawkbit/issues/1249

zyga avatar Mar 08 '23 13:03 zyga

Can one of the admins verify this patch?

hawkbit-bot avatar Mar 08 '23 13:03 hawkbit-bot

NOTE: this is failing in tests that observe download progress.

zyga avatar Mar 08 '23 13:03 zyga

Nice job!

I dont think its important to have these range requests logged anyway, its a thing between client and server and it should always be followed up with another action status, no matter what happens. For client integration debugging purposes its much easier to just add a logger on the fly. As it is right now, this provides 0 benefits for normal users.

Edit: I hit submit too early.

I do think that some sort of "Download Progress" is needed, but if the client is streaming it would be much better if the client was in charge of emitting these actions, as it might not be a consecutive download anyway and downloading 10% of the update package might even be 90% of the download progress on the client side.

MartB avatar Mar 08 '23 16:03 MartB

I can look at removing the database entries while keeping the progress callback intact. If you have any suggestions I'm happy to learn.

zyga avatar Mar 09 '23 09:03 zyga

@MartB I've pushed a small update. I think making the current download progress event work is futile, with my limited knowledge of spring, it seems that the underlying class requires an non-null object and in this case it was always the ActionStatus instance that was tied to a database entry.

zyga avatar Sep 28 '23 13:09 zyga

has conflicts. is this logging related change needed?

avgustinmm avatar Jan 28 '25 07:01 avgustinmm

@avgustinmm Definitely. - In fact, I would not classify this as a logging related change but as a fix for a functional issue.

As outlined by @zyga in the original issue description, the problem is not that the log elements are useless or that there are a lot of them to look through. The problem is that right now, HawkBit simply cannot be used for range-request streamed updates, because too many activity items are generated for the job.

~~This makes HawkBit unusable for devices that rely on this technique to download an update for which they don't have the space or memory to cache it before installation.~~

{
   "exceptionClass":"org.eclipse.hawkbit.repository.exception.AssignmentQuotaExceededException",
   "errorCode":"hawkbit.server.error.quota.tooManyEntries",
   "message":"Quota exceeded: Cannot assign 1 more ActionStatus entities to Action '26021'. The maximum is 1000."
}

Edit: As per the docs, I think setting hawkbit.server.security.dos.maxStatusEntriesPerAction=-1 would allow HawkBit to work in this case. The database would just become huge very quickly.

cybrox avatar Jan 30 '25 16:01 cybrox

I'm not actively involved with hawkbit but I would love to see this fixed. It would unlock simpler installations to use rauc progressive updates.

zyga avatar Jan 30 '25 16:01 zyga

@zyga, @cybrox, thanks for your input. I agree this over detailed logging in case of range requests seems little bit useless and potentially bad.

I just extended this PR idea with skipping also action checks (could be too many, with no big sense) - #2261. This should cover this PR, so I'm closing it. If the #2261 doesn't solve the issue - please open an issue for hawkbit.

Thanks for your contribution in hawkBit.

avgustinmm avatar Jan 31 '25 11:01 avgustinmm