Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .cursorrules
94 changes: 0 additions & 94 deletions .github/copilot-instructions.md

This file was deleted.

1 change: 1 addition & 0 deletions .github/copilot-instructions.md
40 changes: 40 additions & 0 deletions .github/workflows/build-native-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: GraalVM Native Images

on:
workflow_dispatch:

jobs:
build-native:
runs-on: ${{ matrix.os }}
name: "Build native image on ${{ matrix.os }}"
strategy:
matrix:
os: ["macos-13", "ubuntu-22.04"]
steps:
- uses: actions/checkout@v4

- uses: DeLaGuardo/setup-clojure@13.4
with:
cli: 1.12.3.1577
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Build JAR artifact
run: clojure -M:build -- --uberjar

- uses: graalvm/setup-graalvm@v1
with:
version: "22.3.2"
java-version: "17"
distribution: "graalvm"
components: "native-image"
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Build native binary
working-directory: target
run: |
native-image @native-image-args -H:Name=chrondb_${{ runner.os }}

- uses: actions/upload-artifact@v4.6.2
with:
name: chrondb_${{ runner.os }}
path: target/chrondb_${{ runner.os }}
45 changes: 45 additions & 0 deletions .github/workflows/publish-docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Publish Docker Image

on:
push:
branches:
- main

env:
REGISTRY: ghcr.io

jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write

steps:
- name: Checkout source
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ github.repository }}
tags: type=raw,value=latest

- name: Build and push image
uses: docker/build-push-action@v5
with:
push: true
context: .
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,16 @@ target/
data/
test-*
benchmark_results_*.txt
report/
.native-image/
chrondb_*
META-INF/
target/native/
Comment on lines +21 to +25
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Don't ignore META-INF globally — it will hide GraalVM configs.

META-INF/ matches in any directory and would exclude resources/META-INF/native-image/** (e.g., reflect-config.json, resource-config.json) requested by the native-image work. Remove it. While here, minor cleanups:

  • target/native/ is redundant (already covered by target/).
  • Scope chrondb_* to repo root to avoid unintended ignores in subdirs.

Apply within these lines:

 report/
 .native-image/
-chrondb_*
-META-INF/
-target/native/
+/chrondb_*
+# Do not ignore META-INF globally; needed for native-image configs
+# target/native/ is already covered by target/
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
report/
.native-image/
chrondb_*
META-INF/
target/native/
report/
.native-image/
/chrondb_*
# Do not ignore META-INF globally; needed for native-image configs
# target/native/ is already covered by target/
🤖 Prompt for AI Agents
In .gitignore around lines 21 to 25, remove the global "META-INF/" entry (it
hides GraalVM native-image configs under resources/META-INF/native-image/),
delete the redundant "target/native/" line (target/ already covers it), and
change the unconstrained "chrondb_*" to be repo-root scoped (prepend a leading
slash: "/chrondb_*") so it doesn't ignore similarly named files in
subdirectories; keep the other entries (report/ and .native-image/) as-is and
apply these edits within the specified lines.


# OS
.DS_Store
Thumbs.db
.envrc

# doc codox
docs/api/
docs/api/
139 changes: 139 additions & 0 deletions AGENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# ChronDB Agent Guide

## Project Overview

ChronDB is a chronological key/value database implemented in Clojure and backed by a Git-like storage engine. Every write produces an immutable commit, enabling full history, time-travel queries, and branching semantics across multiple protocols (HTTP REST, Redis, PostgreSQL).

## Key Features

- Native Clojure API for JVM applications
- REST API for HTTP interfaces and automation
- Redis protocol compatibility for drop-in clients
- PostgreSQL wire protocol for SQL access
- Git-inspired versioning system with immutable history
- Document-oriented storage with temporal queries

## Code Style and Standards

### Clojure Conventions

- Use idiomatic Clojure naming (`kebab-case` for vars, namespaces as `chrondb.*`)
- Prefer pure functions and data transformations
- Add docstrings for public vars and functions
- Follow indentation as enforced by `cljfmt`
- Use descriptive logging that reflects chronological operations

### Project Structure

- Application sources: `src/`
- Developer tooling: `dev/`
- Tests: `test/`
- Native image build helpers: `dev/chrondb/native_image.clj`
- Docker assets: `Dockerfile`
- Configuration examples: `config.example.edn`

## Development Guidelines

### Database Operations

- Preserve chronological ordering and causal relationships
- Handle repository creation and concurrency with JGit primitives
- Surface meaningful errors when Git operations fail (lock contention, missing refs)
- Avoid destructive operations; prefer new commits over in-place mutation

### API Development

- Keep REST, Redis, and SQL protocols behaviorally consistent
- Validate inputs early and return protocol-appropriate errors
- Document new endpoints in `docs/`
- Maintain backwards-compatible defaults unless a migration path exists

### Testing

- Cover time-travel and branching scenarios
- Include integration tests that exercise Git-backed storage
- Add regression tests for concurrency and conflict resolution
- Prefer property-based tests when sequence ordering matters

### Git Architecture Integration

- Understand repository layout under `data/`
- Ensure new features respect branch isolation and merge semantics
- Use commit metadata to encode chronological context (timestamps, authors)
- Reference <https://www.moclojer.com/blog/git-as-database-harnessing-hidden-power-internals-chronological-data-storage/> for architectural details

## Specific Instructions

### When Generating Code

1. Respect chronological data guarantees (no rewrites of immutable commits)
2. Fail fast with actionable error messages and structured logs
3. Leverage `transduce`, `reduce`, or pipelines for efficient data flows
4. Guard shared mutable state; prefer atoms with swap! semantics
5. Profile Git-heavy operations when adding loops or recursion

### When Reviewing Code

1. Confirm time-based queries return correct snapshots
2. Check API parity across protocols
3. Demand unit and integration coverage for new behaviors
4. Inspect for Git storage performance regressions
5. Enforce Clojure idioms and docstring completeness

### Dependencies

- Manage dependencies via `deps.edn`
- Verify compatibility with GPLv3 license requirements
- Avoid adding dependencies that conflict with GraalVM native image constraints
- Document new libraries and rationale in PR descriptions

## Documentation

- Update `README.md` and `docs/` for major user-facing changes
- Record new API endpoints and wire protocol behaviors
- Provide runnable examples where possible (e.g., `docs/examples-*.md`)
- Keep developer-focused instructions in this `AGENT.md`

## CI/CD Workflows

- `publish-docker.yml` publishes the Docker image to GitHub Container Registry on every push to `main`. The workflow authenticates with `${{ secrets.GITHUB_TOKEN }}`, builds using the project `Dockerfile`, and pushes the `latest` tag to `ghcr.io/<owner>/<repo>`.

## Native Image Support

- Generate the uberjar and GraalVM metadata with `clojure -M:build -- --uberjar`
- The `chrondb.native-image` module creates:
- `target/native-image-args`
- `target/filter.json`
- `target/native-config/`
- Build the native binary locally with `native-image @target/native-image-args -o target/chrondb_local`
- Always include `--features=clj_easy.graal_build_time.InitClojureClasses`
- Do not reintroduce `-H:+UnlockExperimentalVMOptions`; critical JGit classes are already marked for build-time init
- Avoid running the native image build in parallel with services on ports `3000`, `6379`, or `5432`
- Use `.github/workflows/build-native-image.yml` as a reference for the full pipeline and smoke tests

### Advanced Native Image Topics

- **Profile-guided optimization (PGO)**: Capture profile data by running the JVM artifact with representative workloads and feed it to `native-image` via `--pgo`. Keep PGO configs alongside `target/native-image-args`.
- **Closed-world assumptions**: Audit reflection, dynamic class loading, and proxies. Add explicit configuration to `target/native-config/` using `clj-easy/graal-build-time` helpers when new reflective code paths are introduced.
- **Heap configuration**: Tune `--initial-heap-size` and `--max-heap-size` in `target/native-image-args` when targeting constrained environments. Document rationale and expected memory footprint.
- **Monitoring hooks**: When adding logging bridges or metrics, ensure required classes are initialized at build time or registered via `native-image` resources to avoid startup penalties.
- **Determinism**: Native builds must remain reproducible. Capture environment-sensitive values (timestamps, random seeds) explicitly during the `:build` task to prevent binary drift between runs.

## Docker Image Documentation

- Builder image: `ghcr.io/graalvm/native-image:ol9-java17-22.3.2`
- Runtime base: `debian:12-slim`
- Build steps:
- Install Clojure via official installer script
- Run `clojure -M:build -- --uberjar` to produce `target/chrondb.jar`
- Execute `native-image @target/native-image-args -H:Name=chrondb`
- Runtime layout:
- Binary copied to `/usr/local/bin/chrondb`
- Resources copied to `/app/resources`
- Writable data directory at `/app/data`
- Exposed ports: `3000` (HTTP), `6379` (Redis), `5432` (PostgreSQL)
- Non-root user `chrondb` (UID/GID managed in Dockerfile) with home at `/app`
- Usage:
- Build: `docker build -t moclojer/native-image-builder .`
- Run: `docker run --rm -p 3000:3000 moclojer/native-image-builder`
- Persist data: `docker run --rm -v $(pwd)/data:/app/data moclojer/native-image-builder`
15 changes: 12 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,33 @@
# Change Log

All notable changes to this project will be documented in this file. This change log follows the conventions of [keepachangelog.com](http://keepachangelog.com/).

## [Unreleased]

### Changed
- Add a new arity to `make-widget-async` to provide a different widget shape.

- Add support for building ChronDB as a GraalVM native binary.

## [0.1.1] - 2019-12-13

### Changed

- Documentation on how to make the widgets.

### Removed

- `make-widget-sync` - we're all async, all the time.

### Fixed

- Fixed widget maker to keep working when daylight savings switches over.

## 0.1.0 - 2019-12-13

### Added

- Files from the new template.
- Widget maker public API - `make-widget-sync`.

[Unreleased]: https://github.com/your-name/chrondb/compare/0.1.1...HEAD
[0.1.1]: https://github.com/your-name/chrondb/compare/0.1.0...0.1.1
[Unreleased]: https://github.com/moclojer/chrondb/compare/0.1.1...HEAD
[0.1.1]: https://github.com/moclojer/chrondb/compare/0.1.0...0.1.1
Loading
Loading