-
Notifications
You must be signed in to change notification settings - Fork 1
build(native-image): add cross-platform GraalVM native-image workflow #57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
ebbb8a6
build(native-image): add cross-platform GraalVM native-image workflow
avelino 0768058
feat: improve native image build process
avelino f7eb275
feat: add Dockerfile and update GraalVM native-image build for ChronDB
avelino b8196e5
Refactor: Comprehensive update to documentation and Docker build
avelino c659bd8
Update CI/CD documentation and Dockerfile base image version
avelino ca2eb71
feat(ci): add workflow to publish Docker image to GHCR on push to main
avelino File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| AGENT.md |
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| AGENT.md |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| name: GraalVM Native Images | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| build-native: | ||
| runs-on: ${{ matrix.os }} | ||
| name: "Build native image on ${{ matrix.os }}" | ||
| strategy: | ||
| matrix: | ||
| os: ["macos-13", "ubuntu-22.04"] | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - uses: DeLaGuardo/setup-clojure@13.4 | ||
| with: | ||
| cli: 1.12.3.1577 | ||
| github-token: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Build JAR artifact | ||
| run: clojure -M:build -- --uberjar | ||
|
|
||
| - uses: graalvm/setup-graalvm@v1 | ||
| with: | ||
| version: "22.3.2" | ||
| java-version: "17" | ||
| distribution: "graalvm" | ||
| components: "native-image" | ||
| github-token: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Build native binary | ||
| working-directory: target | ||
| run: | | ||
| native-image @native-image-args -H:Name=chrondb_${{ runner.os }} | ||
|
|
||
| - uses: actions/upload-artifact@v4.6.2 | ||
| with: | ||
| name: chrondb_${{ runner.os }} | ||
| path: target/chrondb_${{ runner.os }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| name: Publish Docker Image | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
|
|
||
| env: | ||
| REGISTRY: ghcr.io | ||
|
|
||
| jobs: | ||
| build-and-push: | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| packages: write | ||
|
|
||
| steps: | ||
| - name: Checkout source | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Log in to GitHub Container Registry | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.repository_owner }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Docker metadata | ||
| id: meta | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: ${{ env.REGISTRY }}/${{ github.repository }} | ||
| tags: type=raw,value=latest | ||
|
|
||
| - name: Build and push image | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| push: true | ||
| context: . | ||
| tags: ${{ steps.meta.outputs.tags }} | ||
| labels: ${{ steps.meta.outputs.labels }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,139 @@ | ||
| # ChronDB Agent Guide | ||
|
|
||
| ## Project Overview | ||
|
|
||
| ChronDB is a chronological key/value database implemented in Clojure and backed by a Git-like storage engine. Every write produces an immutable commit, enabling full history, time-travel queries, and branching semantics across multiple protocols (HTTP REST, Redis, PostgreSQL). | ||
|
|
||
| ## Key Features | ||
|
|
||
| - Native Clojure API for JVM applications | ||
| - REST API for HTTP interfaces and automation | ||
| - Redis protocol compatibility for drop-in clients | ||
| - PostgreSQL wire protocol for SQL access | ||
| - Git-inspired versioning system with immutable history | ||
| - Document-oriented storage with temporal queries | ||
|
|
||
| ## Code Style and Standards | ||
|
|
||
| ### Clojure Conventions | ||
|
|
||
| - Use idiomatic Clojure naming (`kebab-case` for vars, namespaces as `chrondb.*`) | ||
| - Prefer pure functions and data transformations | ||
| - Add docstrings for public vars and functions | ||
| - Follow indentation as enforced by `cljfmt` | ||
| - Use descriptive logging that reflects chronological operations | ||
|
|
||
| ### Project Structure | ||
|
|
||
| - Application sources: `src/` | ||
| - Developer tooling: `dev/` | ||
| - Tests: `test/` | ||
| - Native image build helpers: `dev/chrondb/native_image.clj` | ||
| - Docker assets: `Dockerfile` | ||
| - Configuration examples: `config.example.edn` | ||
|
|
||
| ## Development Guidelines | ||
|
|
||
| ### Database Operations | ||
|
|
||
| - Preserve chronological ordering and causal relationships | ||
| - Handle repository creation and concurrency with JGit primitives | ||
| - Surface meaningful errors when Git operations fail (lock contention, missing refs) | ||
| - Avoid destructive operations; prefer new commits over in-place mutation | ||
|
|
||
| ### API Development | ||
|
|
||
| - Keep REST, Redis, and SQL protocols behaviorally consistent | ||
| - Validate inputs early and return protocol-appropriate errors | ||
| - Document new endpoints in `docs/` | ||
| - Maintain backwards-compatible defaults unless a migration path exists | ||
|
|
||
| ### Testing | ||
|
|
||
| - Cover time-travel and branching scenarios | ||
| - Include integration tests that exercise Git-backed storage | ||
| - Add regression tests for concurrency and conflict resolution | ||
| - Prefer property-based tests when sequence ordering matters | ||
|
|
||
| ### Git Architecture Integration | ||
|
|
||
| - Understand repository layout under `data/` | ||
| - Ensure new features respect branch isolation and merge semantics | ||
| - Use commit metadata to encode chronological context (timestamps, authors) | ||
| - Reference <https://www.moclojer.com/blog/git-as-database-harnessing-hidden-power-internals-chronological-data-storage/> for architectural details | ||
|
|
||
| ## Specific Instructions | ||
|
|
||
| ### When Generating Code | ||
|
|
||
| 1. Respect chronological data guarantees (no rewrites of immutable commits) | ||
| 2. Fail fast with actionable error messages and structured logs | ||
| 3. Leverage `transduce`, `reduce`, or pipelines for efficient data flows | ||
| 4. Guard shared mutable state; prefer atoms with swap! semantics | ||
| 5. Profile Git-heavy operations when adding loops or recursion | ||
|
|
||
| ### When Reviewing Code | ||
|
|
||
| 1. Confirm time-based queries return correct snapshots | ||
| 2. Check API parity across protocols | ||
| 3. Demand unit and integration coverage for new behaviors | ||
| 4. Inspect for Git storage performance regressions | ||
| 5. Enforce Clojure idioms and docstring completeness | ||
|
|
||
| ### Dependencies | ||
|
|
||
| - Manage dependencies via `deps.edn` | ||
| - Verify compatibility with GPLv3 license requirements | ||
| - Avoid adding dependencies that conflict with GraalVM native image constraints | ||
| - Document new libraries and rationale in PR descriptions | ||
|
|
||
| ## Documentation | ||
|
|
||
| - Update `README.md` and `docs/` for major user-facing changes | ||
| - Record new API endpoints and wire protocol behaviors | ||
| - Provide runnable examples where possible (e.g., `docs/examples-*.md`) | ||
| - Keep developer-focused instructions in this `AGENT.md` | ||
|
|
||
| ## CI/CD Workflows | ||
|
|
||
| - `publish-docker.yml` publishes the Docker image to GitHub Container Registry on every push to `main`. The workflow authenticates with `${{ secrets.GITHUB_TOKEN }}`, builds using the project `Dockerfile`, and pushes the `latest` tag to `ghcr.io/<owner>/<repo>`. | ||
|
|
||
| ## Native Image Support | ||
|
|
||
| - Generate the uberjar and GraalVM metadata with `clojure -M:build -- --uberjar` | ||
| - The `chrondb.native-image` module creates: | ||
| - `target/native-image-args` | ||
| - `target/filter.json` | ||
| - `target/native-config/` | ||
| - Build the native binary locally with `native-image @target/native-image-args -o target/chrondb_local` | ||
| - Always include `--features=clj_easy.graal_build_time.InitClojureClasses` | ||
| - Do not reintroduce `-H:+UnlockExperimentalVMOptions`; critical JGit classes are already marked for build-time init | ||
| - Avoid running the native image build in parallel with services on ports `3000`, `6379`, or `5432` | ||
| - Use `.github/workflows/build-native-image.yml` as a reference for the full pipeline and smoke tests | ||
|
|
||
| ### Advanced Native Image Topics | ||
|
|
||
| - **Profile-guided optimization (PGO)**: Capture profile data by running the JVM artifact with representative workloads and feed it to `native-image` via `--pgo`. Keep PGO configs alongside `target/native-image-args`. | ||
| - **Closed-world assumptions**: Audit reflection, dynamic class loading, and proxies. Add explicit configuration to `target/native-config/` using `clj-easy/graal-build-time` helpers when new reflective code paths are introduced. | ||
| - **Heap configuration**: Tune `--initial-heap-size` and `--max-heap-size` in `target/native-image-args` when targeting constrained environments. Document rationale and expected memory footprint. | ||
| - **Monitoring hooks**: When adding logging bridges or metrics, ensure required classes are initialized at build time or registered via `native-image` resources to avoid startup penalties. | ||
| - **Determinism**: Native builds must remain reproducible. Capture environment-sensitive values (timestamps, random seeds) explicitly during the `:build` task to prevent binary drift between runs. | ||
|
|
||
| ## Docker Image Documentation | ||
|
|
||
| - Builder image: `ghcr.io/graalvm/native-image:ol9-java17-22.3.2` | ||
| - Runtime base: `debian:12-slim` | ||
| - Build steps: | ||
| - Install Clojure via official installer script | ||
| - Run `clojure -M:build -- --uberjar` to produce `target/chrondb.jar` | ||
| - Execute `native-image @target/native-image-args -H:Name=chrondb` | ||
| - Runtime layout: | ||
| - Binary copied to `/usr/local/bin/chrondb` | ||
| - Resources copied to `/app/resources` | ||
| - Writable data directory at `/app/data` | ||
| - Exposed ports: `3000` (HTTP), `6379` (Redis), `5432` (PostgreSQL) | ||
| - Non-root user `chrondb` (UID/GID managed in Dockerfile) with home at `/app` | ||
| - Usage: | ||
| - Build: `docker build -t moclojer/native-image-builder .` | ||
| - Run: `docker run --rm -p 3000:3000 moclojer/native-image-builder` | ||
| - Persist data: `docker run --rm -v $(pwd)/data:/app/data moclojer/native-image-builder` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,24 +1,33 @@ | ||
| # Change Log | ||
|
|
||
| All notable changes to this project will be documented in this file. This change log follows the conventions of [keepachangelog.com](http://keepachangelog.com/). | ||
|
|
||
| ## [Unreleased] | ||
|
|
||
| ### Changed | ||
| - Add a new arity to `make-widget-async` to provide a different widget shape. | ||
|
|
||
| - Add support for building ChronDB as a GraalVM native binary. | ||
|
|
||
| ## [0.1.1] - 2019-12-13 | ||
|
|
||
| ### Changed | ||
|
|
||
| - Documentation on how to make the widgets. | ||
|
|
||
| ### Removed | ||
|
|
||
| - `make-widget-sync` - we're all async, all the time. | ||
|
|
||
| ### Fixed | ||
|
|
||
| - Fixed widget maker to keep working when daylight savings switches over. | ||
|
|
||
| ## 0.1.0 - 2019-12-13 | ||
|
|
||
| ### Added | ||
|
|
||
| - Files from the new template. | ||
| - Widget maker public API - `make-widget-sync`. | ||
|
|
||
| [Unreleased]: https://github.com/your-name/chrondb/compare/0.1.1...HEAD | ||
| [0.1.1]: https://github.com/your-name/chrondb/compare/0.1.0...0.1.1 | ||
| [Unreleased]: https://github.com/moclojer/chrondb/compare/0.1.1...HEAD | ||
| [0.1.1]: https://github.com/moclojer/chrondb/compare/0.1.0...0.1.1 |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't ignore META-INF globally — it will hide GraalVM configs.
META-INF/matches in any directory and would excluderesources/META-INF/native-image/**(e.g., reflect-config.json, resource-config.json) requested by the native-image work. Remove it. While here, minor cleanups:target/native/is redundant (already covered bytarget/).chrondb_*to repo root to avoid unintended ignores in subdirs.Apply within these lines:
📝 Committable suggestion
🤖 Prompt for AI Agents