Proposal: Add new litestream VFS extension to PyPi, NPM RubyGems for easier installation

[asg017]

Hey! Big fan of Litestream, absolutely love the new Litestream VFS.

I want to make it easier for folks to install and use the new Litestream SQLite extension. I've written and published about a dozen SQLite extension in the past, so I've experience how hard it can be to get users to install SQLite extensions. Loadable .dylib/.so files are simple, but many developers only trust their programming language package managers.

Proposal — Litestream packages for Python, Node.js, and Ruby

My sqlite-dist project can package pre-compiled SQLite extensions for various package managers, like pip for Python, npm for JavaScript, and gem for Ruby. It has a few other "targets" like Github releases/Datasette/sqlite-utils, but I don't think those are necessary for Litestream yet.

sqlite-dist is a CLI that's run during the release process, which would likely end up in Github Actions like the .github/workflows/release.yml workflow.

Given a directory of pre-compiled SQLite extensions and a sqlite-dist.toml config file, sqlite-dist can generate Python wheels (.whl), npm packages (.zip), and gems (.gem). These files can then be uploaded to PyPi/npm/RubyGems, allowing users to directly pip install/npm install/gem install them.

For example, Python developers would be able to run:

pip install litestream

import sqlite3
import litestream

# load the VFS extension in a temporary connection
db = sqlite3.connect(":memory:")
db.enable_load_extension(True)
litestream.load(db)

db = sqlite3.connect('file:///my.db?vfs=litestream', uri=True)
db.execute("PRAGMA litestream_time = '5 minutes ago'")

And for Node.js developers (or Deno, Bun, etc.):

npm install litestream

import { DatabaseSync } from "node:sqlite";
import * as litestream from "litestream";

# load the VFS extension in a temporary connection
let db = new DatabaseSync(":memory:", { allowExtension: true });
litestream.load(db);

db = new DatabaseSync('file:///my.db?vfs=litestream')
db.exec("PRAGMA litestream_time = '5 minutes ago'")

And finally for Ruby developers:

gem install litestream

require 'sqlite3'
require 'litestream'

db = SQLite3::Database.new(':memory:')
db.enable_load_extension(true)
Litestream.load(db)

db = SQLite3::Database.new('file:///my.db?vfs=litestream')
db.execute("PRAGMA litestream_time = '5 minutes ago'")

Implementation

Would be happy to put up a PR for this, which would look something like:

1. Add a new sqlite-dist.toml config file, which would look like this from sqlite-vec. For Litestream, we'd only need to define the pip, npm, and gem targets.
2. Add a new step to release.yml:

• Download + run sqlite-dist on pre-compiled Litestream extension for macos/linux
• Publish generated assets to pip/npm/rubygems, using uv publish/npm publish/gem push

Overall it won't be a lot of new code, though we would need to figure out API keys for pip/npm/gem.

Risks

• You all would have to maintain/own pip/npm/gem packages, as well as field any bug reports about the new installation methods
• sqlite-dist is a one-man show, which I haven't touched in a while (but I plan to make a few upgrades to before implementing here). graft is the only other project that I know that uses sqlite-dist, but otherwise it's just me.

Open questions

• What package names should we take for pip/npm/gem? I think litestream is available on all of them, but since it's just the VFS extension we're publishing, should it be litestream-vfs instead?
• Who should own/have admin rights over the packages themselves?