Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limitations on Deployment HTTP Range Request Download #1249

Open
Bastian-Krause opened this issue Apr 21, 2022 · 6 comments · May be fixed by #1331
Open

Limitations on Deployment HTTP Range Request Download #1249

Bastian-Krause opened this issue Apr 21, 2022 · 6 comments · May be fixed by #1331

Comments

@Bastian-Krause
Copy link

Hi,

we are using hawkBit with RAUC in rauc-hawkbit-updater. In a recent PR, streaming installation of update bundles is implemented: rauc-hawkbit-updater passes the authentication header and the hawkBit deployment download URL to RAUC. RAUC performs HTTP Range Requests (~128 KB range per request, due to NBD reads per squashfs block, see RAUC's streaming docs) while installing the update bundle.

While testing this, I hit these hawkBit errors:

Registered DOS attack! Client [0:0:0:0:0:0:0:1] is above configured READ request threshold (200)!
Cannot assign 1 ActionStatus entities to Action '1' because of the configured quota limit 1000. Currently, there are 1000 ActionStatus entities assigned.

That means hawkBit's default configuration limits HTTP Range Requests effectively to ~1000 requests per action and 200 requests per second. For the time being, I disabled both limitations via:

--hawkbit.server.security.dos.filter.enabled=false
--hawkbit.server.security.dos.maxStatusEntriesPerAction=-1

While this works, I am not so sure if this is the desired approach for our use case. What do you think?

Furthermore, one ActionStatus entity per Range Request is generated, meaning a medium sized update bundle of 150 MB generates >1000 ActionStatuses when requested in 128 KB chunks. Could you elaborate why hawkBit needs to generate an ActionStatus per Range Request? I could not find reasoning in the commits/PRs. Could these Range Request ActionStatuses be dropped maybe? Or made configurable somehow?

@Bastian-Krause
Copy link
Author

Can anyone comment on the questions above?

@Bastian-Krause
Copy link
Author

A comment on this issue would still be very welcome.

I'd be willing to create a PR to change the ActionStatus behavior for HTTP Range Requests, but only if you tell me whether to drop ActionStatuses for this case entirely or make them configurable.

@MartB
Copy link
Contributor

MartB commented Oct 31, 2022

@Bastian-Krause Is this still a limitation?
Would be great to get a comment from the responsible maintainers!

I don't see any issue with dropping the Range Request ActionStatuses so I will change this should I end up deploying hawkbit.

@Bastian-Krause
Copy link
Author

@MartB I did not hear back since I opened this issue and I did not see any changes relevant to these issues. A comment would be welcome.

@zyga
Copy link
Contributor

zyga commented Nov 23, 2022

I've also hit this limitation. I think logging action status makes no sense.

In general I would also say that having hawkbit serve the range requests would be best avoided. Ideally there would be a way to provide an URL to a CDN service, to keep the application server out of the loop.

zyga added a commit to zyga/hawkbit that referenced this issue Mar 8, 2023
When HawkBit is used to stream chunked deltas through RAUC adaptive updates,
adding a database entry for each 4KB block that is requested makes no sense.

Fixes: eclipse-hawkbit#1249

Signed-off-by: Zygmunt Krynicki <[email protected]>
@zyga zyga linked a pull request Mar 8, 2023 that will close this issue
@zyga
Copy link
Contributor

zyga commented Mar 8, 2023

@stefbehl would it be okay to simply disable logging of range requests?

I had a quick stab at this but it would need some extra tweaks to tests. Please see #1331

zyga added a commit to zyga/hawkbit that referenced this issue Sep 28, 2023
HawkBit creates ActionStatus entries for each GET request associated with an
artifact of an assigned software module. Amongst others, this severely breaks
RAUC block-based adaptive updates [1], which issue a flurry of range requests
for small chunks, so that the whole image does not have to be streamed in case
of incremental system changes.

[1] https://rauc.readthedocs.io/en/latest/advanced.html#block-based-adaptive-update-block-hash-index

Fixes: eclipse-hawkbit#1249

Signed-off-by: Zygmunt Krynicki <[email protected]>
zyga added a commit to zyga/hawkbit that referenced this issue Sep 28, 2023
HawkBit creates ActionStatus entries for each GET request associated with an
artefact of an assigned software module. Amongst others, this severely breaks
RAUC block-based adaptive updates [1], which issue a flurry of range requests
for small chunks, so that the whole image does not have to be streamed in case
of incremental system changes.

[1] https://rauc.readthedocs.io/en/latest/advanced.html#block-based-adaptive-update-block-hash-index

Fixes: eclipse-hawkbit#1249

Signed-off-by: Zygmunt Krynicki <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants