Add http & https asset sources

[mrchantey]

Objective

• Fixes Feature: Upstream bevy_web_asset, allowing assets loaded via http #16307

Solution

• Add http & https asset sources

Testing

• Verify visually by running cargo run --example http_source

---

Showcase

Bevy now supports assets loaded via url!

Add the http_source and optionally http_source_cache features.

  // Simply use a url where you would normally use an asset folder relative path
  let handle = asset_server.load("https://raw.githubusercontent.com/bevyengine/bevy/refs/heads/main/assets/branding/bevy_bird_dark.png");
  commands.spawn(Sprite::from_image(handle));

[mrchantey]

It looks like surf and http-cache-surf have failed licence, vunerability & unmaintaned checks. Does that mean they're not allowed as first party dependencies?

[BenjaminBrienen]

It's possible to add an exception for the licenses, but unmaintained and vulnerable is pretty bad.

[mnmaita]

This could be done with something like reqwest maybe to avoid using the unmaintained crates. I successfully ran the example with it but my code is probably bad 😅. I can still push a PR as a showcase in case it serves as inspiration.

[MalekiRe]

Yep this looks awesome to me! Also I wasn't aware of the surf crate; I'll have to keep it in mind for any bevy dashboards I make in the future

[mrchantey]

I can still push a PR as a showcase in case it serves as inspiration.

@mnmaita actually that would be great, I've replaced surf and http-cache-surf with reqwest line for line but am a bit stuck at the moment with reqwest demanding to be run inside a tokio runtime.

// crates/bevy_asset/src/http_source.rs#134

    let client = reqwest_middleware::ClientBuilder::new(Client::new())
        .with(Cache(HttpCache {
            mode: CacheMode::Default,
            manager: CACacheManager::default(),
            options: HttpCacheOptions::default(),
        }))
        .build();

    let response = ContinuousPoll(client.get(str_path).send())
        .await

[jf908]

FWIW surf has a bug where any erroring http response >8kb stops every subsequent request from succeeding and is unlikely to ever be fixed which made me unable to use bevy_web_asset so I would personally appreciate a switch 😄.

If you can't get reqwest to work perhaps hyper would suffice since it's already in the dep tree for BRP? Perhaps too low level for this.

[mrchantey]

The council of bevy async wizards has spoken and we're going with ureq. It doesn't currently have a http-cache wrapper but I've opened an issue and the author might look into creating one when they have time. I guess its worth waiting a bit to see if ureq will get first party support before rolling our own http-cache wrapper, in which case this pr is ready for review :)

A few notes:

• We're still failing check-advisories & check-licences
• I used gh codespaces so wasnt able to cargo run --example http_source, can somebody please try running that?

[github-actions]

You added a new example but didn't add metadata for it. Please update the root Cargo.toml file.

[jf908]

• I used gh codespaces so wasnt able to cargo run --example http_source, can somebody please try running that?

Tested on Windows 11. The example prints a 404 error when the .meta file fails to load and doesn't show the sprite.

If I configure AssetPlugin to AssetMetaCheck::Never, it runs correctly.

I believe the asset is still supposed to load successfully even if the .meta file 404s so I think this is a bug?

[BenjaminBrienen]

We're still failing check-advisories & check-licences
I used gh codespaces

You can ignore that. It isn't a required check. Once those 2 small issues above are fixed, this should be in a pretty good state, I think.

[mockersf]

a decision on authorising licenses unicode 3.0 and MPL 2.0 need to be taken before merging this PR

[alice-i-cecile]

a decision on authorising licenses unicode 3.0 and MPL 2.0 need to be taken before merging this PR

Previous Unicode licenses are already allowed and it is extremely permissive.

MPL 2.0 is already in tree under off-by-default feature flags for various audio formats. This doesn't strike me as any different, and MPL is not dangerously viral to use as a dependency in the same way as e.g. AGPL.

[mockersf]

then they should be added to the license check config, otherwise every other pr after will have failed ci

[mrchantey]

Last license issue is ring (ureq > rustls dep) which has a custom license:

error[unlicensed]: ring = 0.17.8 is unlicensed
  ┌─ registry+https://github.com/rust-lang/crates.io-index#[email protected]:2:9
  │
2 │ name = "ring"
  │         ━━━━ a valid license expression could not be retrieved for the crate
3 │ version = "0.17.8"
4 │ license = ""
  │            ─ license expression was not specified
5 │ license-files = [
6 │     { path = "/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/LICENSE", hash = 0xbd0eed23, score = 0.66, license = "OpenSSL" },
  │                                                                                                                                  ──── low confidence in the license text

[mockersf]

could you try to remove the dependency on ring by disabling default feature also on the dev-dependency to ureq?

[mockersf]

ring comes from rustls support, which is enabled by default with the tls feature

This means that we force our users to use rustls, but also that someone has to properly review that license

[mockersf]

from the license:

3. All advertising materials mentioning features or use of this
   software must display the following acknowledgment:
   "This product includes software developed by the OpenSSL Project
   for use in the OpenSSL Toolkit. (http://www.openssl.org/)"

Seems kind of a pain. I think it means we won't be able to talk about the engine being able to download things from https without copying that

[alice-i-cecile]

Looking at the ring license, it appears to be mostly MIT.

However:

• 3. All advertising materials mentioning features or use of this software
• must display the following acknowledgement:
• "This product includes cryptographic software written by

• Eric Young ([email protected])"
  

This clause is super annoying. It looks like it's a very old amateur license from 1995 or so. The practical risk is low, but we should not allow this license. Can we change features around to avoid ring?

[alice-i-cecile]

See briansmith/ring#1827 for a tracking issue for the upstream licensing stuff. We very much shouldn't try to wait on that.

[mrchantey]

From what i can gather on these ureq issues, the new rustls default is aws-lc-rs instead of ring. It seems to have better licensing but worse devx, requiring non-rust dependencies for windows builds.
algesten/ureq#751
algesten/ureq#765

[alice-i-cecile]

Bah, okay. Can we ship without https support by default and leave the choice up to the end user? I don't particularly care about encrypting e.g. example assets TBH.

[mrchantey]

Sounds good, i guess most users of this feature will need it because just about everything is https these days.

The workaround is in the Cargo.toml dev-dependencies because the http_source example uses https://raw.githubusercontent/.../bevy_bird_dark.png.

We can add tls when ring sorts its license out and in the meantime adding https suppport is a one-liner:

# enable bevy https asset loader
ureq = { version = "*", features = ["tls"] }

check-licenses

check-licenses still isnt happy, i think it cant distinguish a dev-dependency when its the same crate that we're toggling a feature on:

 ├ ring v0.17.8
    ├── rustls v0.23.20
    │   └── ureq v2.12.1
    │       ├── (dev) bevy v0.15.0-dev
    │       └── bevy_asset v0.15.0-dev

[alice-i-cecile]

Once there's a note or two about the license in the docs I'm happy to merge this.

[mockersf]

I don't think it's worth it to merge this without https support

Nowadays everything is https. hiding the license behind a not enabled feature is not a fix, because for pretty much all use cases it will need to be enabled and the license issue solved