AGNTCY Slim v0.7.0 Release

[msardara]

SLIM v0.7.0 Release Announcement 🚀

We're excited to announce the release of SLIM (Secure Low-Latency Interactive Messaging) v0.7.0! This major release delivers a cleaned and more Pythonic API (drop of all Py* prefixes, addition of completion handles for reliable delivery, simplified session creation), major reliability and security enhancements (graceful session draining, backoff retries, SPIRE-based mTLS identity, HMAC tokens, client credential flow, strengthened MLS ), infrastructure and tooling upgrades (SQLite controller persistence, reworked slimctl` CLI commands, improved logging, container security scanning), Homebrew packaging for slimctl.

🎯 Key Highlights

• Graceful session draining (coordinated teardown to prevent message loss).
• Completion handles (awaitable delivery/finalization tokens) for publish, invite, remove, create_session, delete_session — enables true end-to-end delivery confirmation.
• Backoff retry mechanism for controller (#939).
• Reworked Python binding surface: all Py* prefixes removed (clean, conventional naming).
• Restructured session layer (new standalone session crate) and unified multicast & point-to-point handling (#141bc311).
• SPIRE-based mTLS & identity integration (unified TLS configuration, flexible metadata) (#646, #892).
• SPIRE token provider exposed to Python bindings (#604f076a, #5e8abef8).
• Expanded SharedSecret auth to HMAC tokens (#858).
• Identity provider client credential flow (#464).
• Messaging Layer Security (MLS) enhancements:
  • Identity claims integration & PoP (Proof-of-Possession) enforcement (#885).
  • Async MLS operations (#877).
• SQLite persistence for the Controller (#706916a8) replacing previous in‑memory or simplistic persistence.
• Route listing capability (controller northbound API expansion) (#871).
• Reworked slimctl CLI command/subcommand structure for a more intuitive operational experience #933
• Original MsgID propagation to all response messages (#891) improving correlation/tracing.
• Support for Homebrew packaging / cask updates for slimctl (#e6688b82, #922).
• Logging configuration improvements & richer diagnostics.
• Control-plane connection drop handling & recovery (#901, #e967fdb2).
• Option in slimrpc to reuse an existing local app for server (#775).

📜 Documentation

• Updated examples to reflect new Python API (no Py* prefixes; completion handle usage).
• Added PEP 561 compliance (py.typed) for Python type checkers.
• Inline docstring expansions for identity provider / verifier variants.

⚠ Breaking Changes

• Python API surface refactor (see Migration Guide):
  • All PyX → X (e.g., PyName → Name, PySessionConfiguration → SessionConfiguration).
  • Slim.new(...) async factory removed; use synchronous constructor Slim(Name, provider, verifier).
  • create_session signature changed: destination Name passed separately; configuration variants no longer carry peer_name / channel_name.
  • Publish / invite / remove / delete_session now return or produce CompletionHandle objects; callers can optionally await for reliable delivery semantics.
  • Top-level binding functions (connect, publish, invite, etc.) are no longer exported via slim_bindings.__all__; operations go through Slim / Session.
  • Shared secret length expectations increased (examples now use secrets >32 bytes for HMAC robustness).
• IdentityProvider / IdentityVerifier trait initialization supports async init in backend (custom implementations may need adjustment).
• Session draining operations may alter timing of shutdown flows.
• When session_delete is called on the session initiator, all remote peers are notified that the session has been closed, and the session is also closed on their end.

🔧 Infrastructure & Tooling

• GitHub Actions:
  • Docker build caching (#908).
  • Task binary caching (#906).
  • Container security scanning workflow with automated issue creation (#880, #881).
  • Coverage enhancements for Python bindings (#e45ad473) and southbound API (#927).
• Virtual environment usage for stub generation (#918).
• Added config subcommand to slimctl (#818).
• Reworked slimctl` CLI command/subcommand structure for a more intuitive operational experience #933
• Improved command/subcommand clarity in slimctl (#933).
• Use Go-based SQLite library for controller (#aa7518ef).

🛡 Security & Hardening

• SPIRE-based mTLS identity and certificate bootstrapping (#892, #3b0a7eae).
• MLS strengthening: PoP, identity claims (#885).
• Expanded auth token flexibility (custom claims, HMAC secret tokens) (#879, #858).
• Container image vulnerability scans integrated (Trivy summaries).
• Enforced proper audience / issuer validation in identity flows.
• Added robust connection drop recovery (#901) preventing stale security contexts.

Compatibility Notes

• Python users must refactor imports and calling conventions (see Migration Guide).
• Shared secrets used with HMAC tokens must now be sufficiently long (≥32 bytes recommended).
• Reliable message delivery requires awaiting completion handles (non-awaited publishes may appear “fire-and-forget” but can silently fail).
• Controller persistence changes (SQLite) may require migration strategy for existing controller state.
• New TLS modes (SPIRE) require updated config files (see data-plane/config)

Python API Migration Guide (0.6.x → 0.7.0)

Overview of Changes

The Python bindings underwent a structural cleanup:

• All Py* prefixes removed.
• Synchronous constructor replaces async factory.
• Destination Name now passed separately when creating sessions.
• Operations return CompletionHandle objects for reliable end-to-end confirmation.
• Identity provider / verifier API uses nested classes (pattern matching friendly).
• Public API surface trimmed; only high-level types exported.

Old vs New Imports

Before:

from slim_bindings import (
    PyName,
    PyIdentityProvider,
    PyIdentityVerifier,
    PySessionConfiguration,
    PySessionType,
    PySession,
    connect, publish, invite, create_session,
)

After:

from slim_bindings import (
    Name,
    IdentityProvider,
    IdentityVerifier,
    SessionConfiguration,
    SessionType,
    Session,
    Slim,
    CompletionHandle,
)

SLIM instance construction

Before:

local = PyName("org", "ns", "app")
provider = PyIdentityProvider.SharedSecret(identity="app", shared_secret="secret")
verifier = PyIdentityVerifier.SharedSecret(identity="app", shared_secret="secret")
slim = await Slim.new(local, provider, verifier)

After:

local = Name("org", "ns", "app")
provider = IdentityProvider.SharedSecret(identity="app", shared_secret="a_long_secret_string_____32+chars")
verifier = IdentityVerifier.SharedSecret(identity="app", shared_secret="a_long_secret_string_____32+chars")
slim = Slim(local, provider, verifier)
await slim.connect({"endpoint": "...", "tls": {"insecure": True}})

Session Creation

Before:

remote = PyName("org", "ns", "peer")
config = PySessionConfiguration.PointToPoint(
    peer_name=remote,
    max_retries=5,
    timeout=datetime.timedelta(seconds=5),
    mls_enabled=False,
)
session = await slim.create_session(config)

After:

remote = Name("org", "ns", "peer")
config = SessionConfiguration.PointToPoint(
    max_retries=5,
    timeout=datetime.timedelta(seconds=5),
    mls_enabled=False,
)
session, handle = await slim.create_session(remote, config)
await handle  # Ensure underlying session fully initialized (route + handshake)

Publishing & Delivery Acknowledgment

Before:

await session.publish(b"payload", payload_type="msg")

After:

# Step 1: schedule send → returns CompletionHandle
delivery = session.publish(b"payload", payload_type="msg")
# Step 2: await final ack / completion
ack = await delivery
print("Delivered")

Inviting / Removing Participants (Group Sessions)

Before:

await session.invite(invitee_name)
await session.remove(invitee_name)

After:

invite_handle = await session.invite(invitee_name)
await invite_handle  # Ensure control message delivered
remove_handle = await session.remove(invitee_name)
await remove_handle

Deleting Sessions

Before:

await slim.delete_session(session)

After:

delete_handle = await slim.delete_session(session)
await delete_handle  # Guarantees remote teardown propagated

Listening for sessions

Before:

sess = await slim.listen_for_session()

After:

sess = await slim.listen_for_session(timeout=datetime.timedelta(minutes=5))

Error Handling

• Timeouts can occur at two stages:
  1. Scheduling send (rare unless internal queue saturation).
  2. Awaiting completion handle (delivery ack window expiration).
• For reliability, you can wrap await handle in a await asyncio.wait_for(handle, timeout=30)

Shared Secret Length

For HMAC-backed tokens, use secrets ≥32 bytes. Examples updated:

• Old: "secret"
• New: "abcde-12345-fedcb-67890-deadc" (illustrative; choose random high-entropy value).

Removed / Changed APIs

• Removed: Slim.new, set_default_session_config (configure per-session explicitly).
• Removed from public namespace: low-level functions like publish, connect, invite.
• Added: CompletionHandle, App (low-level handle backing Slim).

Migration Checklist

1. Dependencies:

   • Pin slim-bindings>=0.7.0.

2. Imports:

   • Replace all PyX imports with new names.
   • Remove direct function imports (use methods).

3. Construction:

   • Replace await Slim.new(...) with Slim(...); await slim.connect(config).

4. Session Creation:

   • Remove peer_name / channel_name from configuration.
   • Pass destination Name separately.

5. Message Sending:

   • Store return value from publish / publish_to.
   • Await completion handle for reliable delivery.

6. Group Management:

   • Await invite/remove handles.

7. Deletion:

   • Await handle from delete_session.

8. Auth:

   • Update SharedSecret secrets to sufficiently long HMAC-friendly values.

9. Error Handling:

   • Wrap completion awaits in asyncio.wait_for()
   • Use ack object (when provided) for tracing/correlation.

Practical Example (Point-to-Point Session Upgrade)

Before:

remote = PyName("org", "ns", "peer")
config = PySessionConfiguration.PointToPoint(peer_name=remote, timeout=datetime.timedelta(seconds=5))
session = await slim.create_session(config)
await session.publish(b"hi")

After:

remote = Name("org", "ns", "peer")
config = SessionConfiguration.PointToPoint(timeout=datetime.timedelta(seconds=5), max_retries=5)
session, handle = await slim.create_session(remote, config)
await handle  # Ensure session established
delivery = session.publish(b"hi")
ack = await delivery
print("Message sent")

Final Notes

The 0.7.0 release focuses on reliability (completion handles + backoff), security (SPIRE + MLS enhancements), and developer ergonomics (Python API cleanup & typing). Adopting the migration steps above should be straightforward — prioritize updating session creation and publish flows first, then refine identity and routing patterns.

Special thanks to all the contributors who made this release happen! @micpapal @lpuskas @sancyx @muscariello @Tehsmash @janosSarusiKis @gomba999 @hackeramitkumar