New Project Proposal - SPK/SPFS

[rydrman]

Project description

SPK is a software package and runtime system for building, distributing and executing software environments in a studio setting. The website summarizes the top-level goals of the project well:

Driven by the unique requirements of the film, vfx, and animation industries, SPK has a few primary goals:

• Package Compatibility Beyond Version Numbers
• Recipe and Source Publication
• Fast, Dynamic Build and Runtime Environments
• Reliable and Natural Definition of Platforms and Constraints

Additionally, SPK is built on top of a technology called SPFS, which lends a few superpowers to the whole system:

• Per-process, isolated software runtimes
• A single, consistent file path for all software at runtime
• File-level de-duplication of package data
• Efficient sync, transfer and localization of software

SPK/SPFS was started as an internal project at Imageworks in 2019, and was adopted by ILM for some workflows starting in 2022. These two companies are using SPK and SPFS to modernize their software build and distribution processes in order to keep up with the increasing size and complexity of the environments required for visual effects and animation work. We believe that these tools represent a new era for software environments in the industry and should be designed, built and discussed by the community as a whole in the most accessible forums as possible.

Sponsor from TAC

Cary Phillips

Proposed Project Stage

Sandbox

Please explain how this project is aligned with the mission of the Academy Software Foundation?

SPK and SPFS continue to see interest from studios and companies outside of ILM and Imageworks, but the current project meetings are not entirely neutral or accessible, despite the desire to be so. This is one of the areas that we hope can be better aligned with the help of the ASWF.

As a tool for defining how to build and run software in complex environments, SPK and SPFS align directly as a possible solution to some of the ASWFs goals. Specifically, the goal to “Streamline development for build and runtime environments through the sharing of open source build configurations, scripts and recipes” could be at least partially achieved through the many common package recipes that already exist in our repository.

At a high level, SPK has similar use cases to the Rez project, and the two intentionally share workflows related to publishing and running environments. Although SPK can be seen as an alternative to Rez, the two projects are not meant to be in competition and could mutually benefit from collaboration within the ASWF.

What is the project’s license for code contributions and methodology for code contributions?

Apache License 2.0
DCO only, no CLA required

What tool or platform is utilized for source control (GitHub, etc.), and what is the location

(e.g., URL)?

github: https://github.com/spkenv/spk

What are the external dependencies of the project, and what are the licenses of those dependencies?

This is a report of the immediate dependencies for our rust code as reported by the cargo-license command line tool. It lists each crate that we depend on by name, the current version that we are using and the license attached to its code.

name	version	license
anyhow	1.0.75	Apache-2.0 OR MIT
arc-swap	1.6.0	Apache-2.0 OR MIT
async-compression	0.3.15	Apache-2.0 OR MIT
async-recursion	1.0.5	Apache-2.0 OR MIT
async-stream	0.3.5	MIT
async-trait	0.1.74	Apache-2.0 OR MIT
bytes	1.5.0	MIT
cached	0.48.1	MIT
caps	0.5.5	Apache-2.0 OR MIT
chrono	0.4.34	Apache-2.0 OR MIT
clap	4.5.0	Apache-2.0 OR MIT
clap_complete	4.4.3	Apache-2.0 OR MIT
close-err	1.0.2	MIT
colored	2.0.4	MPL-2.0
config	0.14.0	Apache-2.0 OR MIT
console	0.15.8	MIT
crossterm	0.28.1	MIT
ctrlc	3.4.1	Apache-2.0 OR MIT
dashmap	5.5.3	MIT
data-encoding	2.4.0	MIT
derive_builder	0.20.0	Apache-2.0 OR MIT
dirs	5.0.1	Apache-2.0 OR MIT
dunce	1.0.4	Apache-2.0 OR CC0-1.0 OR MIT-0
dyn-clone	1.0.14	Apache-2.0 OR MIT
enum_dispatch	0.3.13	Apache-2.0 OR MIT
faccess	0.2.4	MIT
flatbuffers	23.5.26	Apache-2.0
format_serde_error	0.3.0	MIT
fuser	0.14.0	MIT
futures	0.3.29	Apache-2.0 OR MIT
futures-core	0.3.29	Apache-2.0 OR MIT
gitignore	1.0.8	Apache-2.0 OR MIT
glob	0.3.1	Apache-2.0 OR MIT
hyper	0.14.27	MIT
hyper	1.4.1	MIT
ignore	0.4.20	MIT OR Unlicense
indexmap	1.9.3	Apache-2.0 OR MIT
indexmap	2.2.3	Apache-2.0 OR MIT
indicatif	0.17.8	MIT
is_default_derive_macro	0.42.0
itertools	0.10.5	Apache-2.0 OR MIT
itertools	0.11.0	Apache-2.0 OR MIT
itertools	0.12.0	Apache-2.0 OR MIT
libc	0.2.155	Apache-2.0 OR MIT
miette	7.0.0	Apache-2.0
nix	0.27.1	MIT
nom	7.1.3	MIT
nom-supreme	0.8.0	MPL-2.0
nonempty	0.8.1	MIT
num-bigint	0.4.4	Apache-2.0 OR MIT
num-format	0.4.4	Apache-2.0 OR MIT
num_cpus	1.16.0	Apache-2.0 OR MIT
number_prefix	0.4.0	MIT
once_cell	1.19.0	Apache-2.0 OR MIT
paste	1.0.14	Apache-2.0 OR MIT
pin-project-lite	0.2.13	Apache-2.0 OR MIT
priority-queue	1.3.2	LGPL-3.0 OR MPL-2.0
procfs	0.16.0	Apache-2.0 OR MIT
prost	0.12.3	Apache-2.0
question	0.2.2	MIT
quote	1.0.35	Apache-2.0 OR MIT
rand	0.8.5	Apache-2.0 OR MIT
regex	1.10.2	Apache-2.0 OR MIT
relative-path	1.9.0	Apache-2.0 OR MIT
ring	0.17.7
rstest	0.18.2	Apache-2.0 OR MIT
semver	1.0.20	Apache-2.0 OR MIT
sentry	0.34.0	Apache-2.0
sentry-tracing	0.34.0	Apache-2.0
serde	1.0.197	Apache-2.0 OR MIT
serde_json	1.0.108	Apache-2.0 OR MIT
serde_qs	0.10.1	Apache-2.0 OR MIT
serde_yaml	0.8.26	Apache-2.0 OR MIT
serde_yaml	0.9.27	Apache-2.0 OR MIT
shellexpand	3.1.0	Apache-2.0 OR MIT
signal-hook	0.3.17	Apache-2.0 OR MIT
spdx	0.10.6	Apache-2.0 OR MIT
static_assertions	1.1.0	Apache-2.0 OR MIT
strip-ansi-escapes	0.1.1	Apache-2.0 OR MIT
strum	0.26.3	MIT
syn	1.0.109	Apache-2.0 OR MIT
syn	2.0.48	Apache-2.0 OR MIT
sys-info	0.9.1	MIT
syslog-tracing	0.2.0	MIT
tap	1.0.1	MIT
tar	0.4.40	Apache-2.0 OR MIT
tempfile	3.8.1	Apache-2.0 OR MIT
tera	1.19.1	MIT
thiserror	1.0.57	Apache-2.0 OR MIT
time	0.3.36	Apache-2.0 OR MIT
tokio	1.39.2	MIT
tokio-retry	0.3.0	MIT
tokio-stream	0.1.14	MIT
tokio-util	0.7.10	MIT
tonic	0.11.0	MIT
tower	0.4.13	MIT
tracing	0.1.40	MIT
tracing-capture	0.1.0	Apache-2.0 OR MIT
tracing-subscriber	0.3.17	MIT
ulid	1.1.0	MIT
unix_mode	0.1.4	Apache-2.0
url	2.4.1	Apache-2.0 OR MIT
uuid	0.8.2	Apache-2.0 OR MIT
uuid	1.5.0	Apache-2.0 OR MIT
variantly	0.4.0	MIT
walkdir	2.4.0	MIT OR Unlicense
whoami	1.5.1	Apache-2.0 OR BSL-1.0 OR MIT
windows	0.51.1	Apache-2.0 OR MIT
windows	0.52.0	Apache-2.0 OR MIT
winfsp	0.9.4+winfsp-2.0	GPL-3.0
winfsp-sys	0.2.2+winfsp-2.0	GPL-3.0

What roles does the project have (e.g., maintainers, committers?) Who are the current

core committers of the project, or which can a list of committers be found?

No distinction in roles so far, aside from repo maintainers (push access) vs external contributors via pull requests

The full list of contributors can be found here: https://github.com/spkenv/spk/graphs/contributors. Notably, Ryan Bottriell (ILM), J Robert Ray (SPI) and David Gilligan-Cook (SPI) are the most active contributors.

What mailing lists are currently used by the project?

None

What tool or platform is leveraged by the project for issue tracking?

github issues on the project itself

Does the project have an OpenSSF Best Practices Badge? Do you foresee any

challenges in obtaining one?

yes, it does

What is the project’s website? Is there a wiki?

https://spkenv.dev/ (documentation is generated from markdown in our github repo, anyone can contribute)

What social media accounts are used by the project?

None

What is the project’s release methodology and cadence?

No particular cadence so far, releases are cut from the main branch when a sufficient amount of new features and development efforts have been merged.

Are any trademarks, registered or unregistered, leveraged by the project? Have any

trademark registrations been filed by the project or any third party anywhere in the world?

No

[jmertic]

Thank you @rydrman for this submission. We will follow up directly with you the Technical Charter and Project Contribution Agreement, with a goal to try and get this presentation in next week's TAC meeting if we can get this all completed quickly. Thanks!