A rust implementation of the car mirror protocol that can be compiled to Wasm to run in the browser. Car mirror is used to transfer IPLD from one computer to another over the internet over various transports, but most notably HTTP(s). It tries to do so with deduplication and minimal communcation round-trips.
The main storage abstraction that rs-car-mirror is built upon is a BlockStore implementation from the wnfs-common crate.
- Crates
- Usage and Installation
- Testing the Project
- Benchmarking the Project
- Setting-up car-mirror-wasm
- Contributing
- Getting Help
- External Resources
- License
- car-mirror: The sans-io implementation of the car mirror protocol
- car-mirror-axum: Utilities for and an implementation of a car mirror HTTP server.
- car-mirror-reqwest: Utilities for running car mirror protocol requests against a car mirror HTTP server.
- car-mirror-wasm: (Browser-flavoured) Wasm bindings to the client parts of car-mirror.
- car-mirror-benches: Benchmarks. Not a published crate.
This is the dependency graph between these crates:
flowchart TD
car-mirror-wasm --> car-mirror
car-mirror-reqwest --> car-mirror
car-mirror-axum --> car-mirror
car-mirror-benches --> car-mirror
This is an example of running both a local car-mirror axum server with some test data as well as running requests against it using car-mirror-reqwest:
use anyhow::Result;
use car_mirror::{cache::NoCache, common::Config};
use car_mirror_reqwest::RequestBuilderExt;
use reqwest::Client;
use wnfs_common::{BlockStore, MemoryBlockStore, CODEC_RAW};
#[tokio::main]
async fn main() -> Result<()> {
// Start a car-mirror axum webserver:
tokio::spawn(car_mirror_axum::serve(MemoryBlockStore::new()));
// Generate some test IPLD data:
let store = MemoryBlockStore::new();
let data = b"Hello, world!".to_vec();
let root = store.put_block(data, CODEC_RAW).await?;
// Run the car mirror push protocol to upload the IPLD data:
let client = Client::new();
client
.post(format!("http://localhost:3344/dag/push/{root}"))
.run_car_mirror_push(root, &store, &NoCache)
.await?;
let store = MemoryBlockStore::new(); // clear out data
// Download the same data again:
client
.post(format!("http://localhost:3344/dag/pull/{root}"))
.run_car_mirror_pull(root, &Config::default(), &store, &NoCache)
.await?;
assert!(store.has_block(&root).await?);
Ok(())
}cargo add car-mirror-reqwestif you want to do client requests with thereqwestcrate against a car-mirror server over HTTP.cargo add car-mirror-axumif you want to implement anaxumversion0.7server to serve data via car-mirror over HTTP.cargo add car-mirroras a lower-level library for clients/servers/peers that want to talk car-mirror over other protocol types.
npm i car-mirror-wasm
This monorepo contains the car-mirror-wasm bindings to the client parts of car-mirror.
These bindings are really low-level and only have rudimentary typescript types.
Take a look at the test utilities to see how to use the low-level functions together with e.g. the fetch API.
-
Run tests for the whole workspace:
cargo testOr just the
car-mirrorcrate:cargo test -p car-mirror -
To test
car-mirror-wasm:cd car-mirror-wasm && npm test
For benchmarking and measuring performance, this workspaces provides
a Rust-specific benchmarking package leveraging criterion and a
test_utils feature flag for integrating proptest within the
suite for working with strategies and sampling from randomly
generated values.
-
Run benchmarks
cargo bench -p car-mirror-benches
Note: Currently, this workspace only supports Rust-native benchmarking, as
wasm-bindgen support for criterion is still an open issue.
However, with some extra work, benchmarks can be compiled to wasi and
run with wasmer/wasmtime or in the brower with
webassembly.sh. Please catch-up with wasm support for criterion on their
user-guide.
🎈 We're thankful for any feedback and help in improving our project! We have a contributing guide to help you get involved. We also adhere to our Code of Conduct.
This repository contains a Nix flake that initiates both the Rust toolchain set in rust-toolchain.toml and a pre-commit hook. It also installs helpful cargo binaries for development. Please install nix and direnv to get started.
Run nix develop or direnv allow to load the devShell flake output,
according to your preference.
For formatting Rust in particular, we automatically format on nightly, as it
uses specific nightly features we recommend by default.
This library recommends using pre-commit for running pre-commit hooks. Please run this before every commit and/or push.
- If you are doing interim commits locally, and for some reason if you don't
want pre-commit hooks to fire, you can run
git commit -a -m "Your message here" --no-verify.
- We recommend leveraging cargo-watch, cargo-expand and irust for Rust development.
- We recommend using cargo-udeps for removing unused dependencies before commits and pull-requests.
This project lightly follows the Conventional Commits
convention to help explain
commit history and tie in with our release process. The full specification
can be found here. We recommend prefixing your commits with
a type of fix, feat, docs, ci, refactor, etc..., structured like so:
<type>[optional scope]: <description>
[optional body]
[optional footer(s)]
For usage questions, usecases, or issues reach out to us in our Discord channel.
We would be happy to try to answer your question or try opening a new issue on Github.
These are references to specifications, talks and presentations, etc.
This project is licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.