MVP of self-hosting

[ddimaria]

BUGS

• Self host deployment Table error
• Don't pass token to S3 @ddimaria
• Hide Examples @ddimaria
• Localhost doesn't work (later)
• License key shown as revoked incorrectly (can we replicate this?)

TODO

• Remove self-hosting directory (in favor of new self-hosting repo
• Deploy self-hosting portal on the production domain
• Update all references to new domain (selfhost.quadratichq.com)
• Add cloud formation template for production deployment
• Update URLs to self-hosting portal production domain
• Usurp self-hosting checks for quadratic production @ddimaria
• Set ENV vars for preview (@davidkircos)
• API Code Review @jimniels
• Client Code Review @davidfig
• Set ENV vars for production(@davidkircos)
• Verify AWS Credentials in production have necessary permissions for creating image repos and pushing.
• Update portal to link to different self hosting options
• After start print where to access Quadratic. Quadratic Selfhost ASCII Art + "Open Quadratic at [Link]" @ddimaria

After merge

• Update repo names to point to prod images https://github.com/quadratichq/quadratic-selfhost/blob/main/docker-compose.yml

Self hosting auth alternative (Ory Kratos support)

• Add Kratos to docker compose and configure
• Add kratos-selfservice-ui-node to docker compose and configure
• Implement AUTH_TYPE in quadratic-api and switch between auth0 and ory based on the selection
  • Mirror auth0 functionality in ory
• Implement AUTH_TYPE in quadratic-client and switch between auth0 and ory based on the selection
  • Mirror auth0 functionality in ory
• Host jwks.json in the quadratic-client

Simple file storage

• Implement file-system storage in quadratic-rust-shared and quadratic-files
• Implement file-system storage in quadratic-api and quadratic-client

On the API side, you denote the storage method as an env var: STORAGE_TYPE=file-system # s3 or file-system. There is a storage abstraction within API that can switch between S3 and FileSystem. The abstracted functions are:

• getFileUrl()
• getPresignedFileUrl() (reuse ENCRYPTION_KEY for uuid encryption in presigned URLs)
• uploadFile()
• uploadMiddleware() (express middleware for S3 and FileSystem

On the File Service side, the same env var STORAGE_TYPE is used to turn on a few new routes:

• GET /storage/:key Get any file in storage, validated by JWT
• POST /storage/:key Upload any file in storage, validated by JWT
• GET /storage/presigned/:key Get any file in storage via a presigned key, validated by key decryption

Environment Variables to Add:

quadratic-api

STORAGE_TYPE=s3 # s3 or file-system
QUADRATIC_FILE_URI=

AUTH_TYPE=ory # auth0 or ory
ORY_JWKS_URI='http://localhost:3000/.well-known/jwks.json'
ORY_ADMIN_HOST=http://0.0.0.0:4434

LICENSE_KEY="LICENSE_KEY"
LICENSE_API_URI=https://selfhost.quadratic-preview.com

quadratic-client

VITE_AUTH_TYPE=ory # auth0 or ory
VITE_ORY_HOST=http://localhost:4433

quadratic-files

STORAGE_TYPE=s3 # s3 or file-system
STORAGE_DIR=
STORAGE_ENCRYPTION_KEYS=

[qa-wolf]

QA Wolf here! As you write new code it's important that your test coverage is keeping up.
Click here to request test coverage for this PR!

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
quadratic	✅ Ready (Inspect)	Visit Preview	Nov 5, 2024 8:02pm

[davidkircos]

--weak-refs is not in Vercel / Amplify build

[davidkircos]

npm run compile is not in Vercel / Amplify build

[davidkircos]

We can't do this with s3!

[davidkircos]

[qa-wolf]

E2E tests

QA 2 blocking bugs

Ran	Status	Preview	Started	Run time	Est. dev time saved
83 workflows	Done (Details)		Nov 5, 2024 at 9:18 PM (UTC)	21 minutes	~36.17 hrs

✅ 76 passed

• Data Actions

• Zoom Options

• Import Files

• Auto Focus after Closing Menus

• API Calls

• Read JavaScript Output within Formula

• Read Python Output within Formula

• Sheet Actions

• Time Formatting

• Log In

• Right Click Actions

• Resize Column width with Fill

• Open and Use Formula Editor

• Types: Series and Data-Frames

• Date Formatting

• Open Formula Editor

• Rename Team

• Command Palette

• Cell Reference - Javascript

• Cell Formatting

• Upload Large File

• Edit Share File Permissions

• Code Editor Displays Return

• Code Snippets

• Key Actions

• Share File - Spreadsheet

• Create Example

• Open Python Editor

• Can View Team Member Cannot Edit Files

• Basic Formula Creation

• Help Actions

• SQL - Create a Connection, Add Data to Database, Query Database

• Docs page

• File Actions - Dashboard

• Number Formatting

• Discard Changes

• Connection goes down in Multiplayer Session

• Can Edit Members are Able to Invite

• Spills

• AI Assistant

• View Actions

• SQL - Reference Data in Formula and Python

• Log Out

• Formulas

• Drag Relative References

• File Actions

• Custom time and date formatting

• Text Wrap, Horizontal and Vertical Alignment

• Dashboard Views - My Files

• Switch between Python and Formula

• Create File for Team

• Can Edit Member are Able to Change Permissions

• Make Changes while Network is off

• Create New File

• Multi-Delete Columns

• Multi-Delete Rows

• Invite Member to Team

• Can Stop Code Execution

• Cell Reference - Python

• Insert and Delete Rows

• Insert and Delete Columns

• Filter Files by Name - My Files

• Create a Team

• Charts

• Can View Members are Unable to Invite Members

• Formatting

• Can Edit Team Member Can Edit Files

• Members Can Leave Team

• Dashboard Views - Shared with me

• User Can See Other Users on File

• Action Visibility

• Mouse Visibility

• Filter Files by Name - Shared with me

• Code Cell Outlines

• Auto-Complete

• Switching Tabs Persists Cursor

Preexisting bugs

2 Blocking bugs

If you are aware of any of these bugs, you can set their priority to low and prevent them from causing a run failure.
View all blocking bugs

• Intermittent "NaN" Output in Code Modal When Using Queried SQL Data with JavaScript

• Intermittent license revoke error

1 Non-blocking bug

• Cells Evaluate to Error When Running All Code Cells

. . . . . . . .

Re-run your E2E tests