Skip to content

Conversation

@ReeceHumphreys
Copy link
Contributor

@ReeceHumphreys ReeceHumphreys commented Nov 24, 2025

  • Tickets addressed: N/A
  • Review: By commit
  • Merge strategy: Merge (no squash)

Description

In an effort to reduce the size of basilisks wheels and shift to a more sustainable long term architecture it is desirable to not ship support data files inside the wheels. As such, we need a mechanism to fetch and cache files on demand as they are requested.

If you clone the repo most of the data is still provided directly. As before, some data such as JPL Spice data was downloaded during the build. With this branch such Spice data will now also be downloaded on demand.

To accomplish this, the way data files are loaded and used in bsk has changed drastically and now leverages Pooch to handle fetching the files. This is a library that has been battle tested by many other popular Python scientific libraries for similar reasons.

As such, this PR does the following:

  • Removes supportData files from sdist and wheels
  • Adds a Pooch-based fetcher for all supportData categories
  • Introduces a generated registry with MD5 hashes for versioned files
  • Implements fallbacks for external (non-versioned) NAIF (JPL Spice) kernels
  • Updates Denton GEO model and related modules to use the new fetch API
  • Adds local-repo detection so editable installs use on-disk supportData
  • Updates documentation to explain the new fetch/caching workflow
  • Regenerates supportData registry and updates data enums accordingly
  • update bskLargeData command to prefetch all dynamic data components

Verification

CI/CD and local testing of newly built wheels verify that they do not contain the supportData files and can fetch the files using the new API.

Documentation

  • bskLargeData docs minorly updated.
  • New documentation page explaining how support data is now handled
  • updated release notes

Future work

N/A

@ReeceHumphreys ReeceHumphreys moved this to 🏗 In progress in Basilisk Nov 24, 2025
@ReeceHumphreys ReeceHumphreys self-assigned this Nov 24, 2025
@ReeceHumphreys ReeceHumphreys force-pushed the feature/pooch branch 4 times, most recently from ad9ee1d to da91cfe Compare December 1, 2025 05:21
@ReeceHumphreys ReeceHumphreys changed the title Load supportData files from GitHub using Pooch Migrate supportData to on-demand fetching with Pooch Dec 1, 2025
@ReeceHumphreys ReeceHumphreys moved this from 🏗 In progress to 👀 In review in Basilisk Dec 1, 2025
@ReeceHumphreys ReeceHumphreys marked this pull request as ready for review December 1, 2025 15:48
@ReeceHumphreys ReeceHumphreys requested a review from a team as a code owner December 1, 2025 15:48
@ReeceHumphreys ReeceHumphreys force-pushed the feature/pooch branch 8 times, most recently from 6457e97 to 275cdaf Compare December 19, 2025 18:35
@schaubh
Copy link
Contributor

schaubh commented Dec 20, 2025

Local data in basilisk/supportData is ignored in a cloned repo and required data is downloaded anyway.

@ReeceHumphreys ReeceHumphreys force-pushed the feature/pooch branch 8 times, most recently from 2d124e2 to c61e2ba Compare December 24, 2025 08:52
@ReeceHumphreys
Copy link
Contributor Author

ReeceHumphreys commented Jan 2, 2026

Updated as requested, there are two new more substantial commits 5ff0a8b and e82b6c1:

  • 5ff0a8b is similar to what we did for the Denton model where now you specify the configuration file in python.

  • e82b6c1 is similar in idea but loads all of the expected kernel files automatically when initializing a SpiceInterface object in python. This is again similar to the idea of what is required for Denton. However, since there are quite a few files for the SpiceInterface object to be loaded and they are always expected to be there, we can just auto load them for the users!

Copy link
Contributor

@schaubh schaubh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor items including:

  • WMM model needs updated TeX documentation on how to set path
  • restore the ISC license info in a *.I file. Please don't remove these!

@ReeceHumphreys ReeceHumphreys merged commit 7871d7b into develop Jan 3, 2026
22 of 23 checks passed
@ReeceHumphreys ReeceHumphreys deleted the feature/pooch branch January 3, 2026 15:39
@github-project-automation github-project-automation bot moved this from 👀 In review to ✅ Done in Basilisk Jan 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants