Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
218 changes: 216 additions & 2 deletions docs/advanced-guide/using-publisher-subscriber/page.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ scaled and maintained according to its own requirement.
## Design choice

In GoFr application if a user wants to use the Publisher-Subscriber design, it supports several message brokers,
including Apache Kafka, Google PubSub, MQTT, and NATS JetStream.
including Apache Kafka, Google PubSub, MQTT, NATS JetStream, and Redis Pub/Sub.
The initialization of the PubSub is done in an IoC container which handles the PubSub client dependency.
With this, the control lies with the framework and thus promotes modularity, testability, and re-usability.
Users can do publish and subscribe to multiple topics in a single application, by providing the topic name.
Expand Down Expand Up @@ -332,6 +332,220 @@ docker run -d \
When subscribing or publishing using NATS JetStream, make sure to use the appropriate subject name that matches your stream configuration.
For more information on setting up and using NATS JetStream, refer to the official NATS documentation.

### Redis Pub/Sub

Redis Pub/Sub is a lightweight messaging system. GoFr supports two modes:
1. **Streams Mode** (Default): Uses Redis Streams for persistent messaging with consumer groups and acknowledgments.
2. **PubSub Mode**: Standard Redis Pub/Sub (fire-and-forget, no persistence).

#### Configs

{% table %}
- Name
- Description
- Required
- Default
- Example
- Valid format

---

- `PUBSUB_BACKEND`
- Using Redis Pub/Sub as message broker.
- `+`
-
- `REDIS`
- Not empty string

---

- `REDIS_PUBSUB_MODE`
- Operation mode: `pubsub` or `streams`.
- `-`
- `streams`
- `pubsub`
- `pubsub`, `streams`

---

- `REDIS_STREAMS_CONSUMER_GROUP`
- Consumer group name (Required for streams mode).
- `+` (if mode=streams)
-
- `my-group`
- String

---

- `REDIS_STREAMS_CONSUMER_NAME`
- Unique consumer name (Optional).
- `-`
- `hostname`
- `my-consumer`
- String

---

- `REDIS_STREAMS_BLOCK_TIMEOUT`
- Blocking duration for reading new messages.
- `-`
- `5s`
- `2s`
- Duration

---

- `REDIS_STREAMS_MAXLEN`
- Maximum length of the stream (approximate).
- `-`
- `0` (unlimited)
- `1000`
- Integer

---

- `REDIS_HOST`
- Hostname of the Redis server.
- `+`
- `localhost`
- `redis.example.com`
- String

---

- `REDIS_PORT`
- Port of the Redis server.
- `-`
- `6379`
- `6380`
- Integer

---

- `REDIS_USER`
- Username for Redis authentication (if required).
- `-`
- `""`
- `myuser`
- String

---

- `REDIS_PASSWORD`
- Password for Redis authentication (if required).
- `-`
- `""`
- `mypassword`
- String

---

- `REDIS_DB`
- Database number to use (0-15).
- `-`
- `0`
- `1`
- Integer (0-15)

---

- `REDIS_TLS_ENABLED`
- Enable TLS for Redis connections.
- `-`
- `false`
- `true`
- Boolean

---

- `REDIS_TLS_CA_CERT`
- Path to the TLS CA certificate file (or PEM-encoded certificate string).
- `-`
- `""`
- `/path/to/ca.pem`
- Path or PEM string

---

- `REDIS_TLS_CERT`
- Path to the TLS certificate file (or PEM-encoded certificate string).
- `-`
- `""`
- `/path/to/cert.pem`
- Path or PEM string

---

- `REDIS_TLS_KEY`
- Path to the TLS key file (or PEM-encoded key string).
- `-`
- `""`
- `/path/to/key.pem`
- Path or PEM string

{% /table %}

```dotenv
PUBSUB_BACKEND=REDIS
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_USER=myuser
REDIS_PASSWORD=mypassword
REDIS_DB=0
REDIS_TLS_ENABLED=true
REDIS_TLS_CA_CERT=/path/to/ca.pem
REDIS_TLS_CERT=/path/to/cert.pem
REDIS_TLS_KEY=/path/to/key.pem

# Streams mode (default) - requires consumer group
REDIS_STREAMS_CONSUMER_GROUP=my-group
REDIS_STREAMS_CONSUMER_NAME=my-consumer
REDIS_STREAMS_BLOCK_TIMEOUT=5s
REDIS_STREAMS_MAXLEN=1000

# To use PubSub mode instead, set:
# REDIS_PUBSUB_MODE=pubsub
```

#### Docker setup

```shell
docker run -d \
--name redis \
-p 6379:6379 \
redis:7-alpine
```

For Redis with password authentication:

```shell
docker run -d \
--name redis \
-p 6379:6379 \
redis:7-alpine redis-server --requirepass mypassword
```

For Redis with TLS:

```shell
docker run -d \
--name redis \
-p 6379:6379 \
-v /path/to/certs:/tls \
redis:7-alpine redis-server \
--tls-port 6380 \
--port 0 \
--tls-cert-file /tls/redis.crt \
--tls-key-file /tls/redis.key \
--tls-ca-cert-file /tls/ca.crt
```

> **Note**: Redis Pub/Sub uses channels (topics) that are created automatically on first publish/subscribe.
> Channels cannot be explicitly created or deleted - they exist as long as there are active subscriptions.

> **Note**: By default, Redis Pub/Sub uses Streams mode which provides persistence and at-least-once delivery semantics with consumer groups and acknowledgments.
> Use `REDIS_PUBSUB_MODE=pubsub` for fire-and-forget messaging with at-most-once delivery semantics (messages are not persisted).

### Azure Event Hubs
GoFr supports Event Hubs starting gofr version v1.22.0.

Expand Down Expand Up @@ -416,7 +630,7 @@ func (ctx *gofr.Context) error
```

`Subscribe` method of GoFr App will continuously read a message from the configured `PUBSUB_BACKEND` which
can be either `KAFKA` or `GOOGLE` as of now. These can be configured in the configs folder under `.env`
can be `KAFKA`, `GOOGLE`, `MQTT`, `NATS`, `REDIS`, or `AZURE_EVENTHUB`. These can be configured in the configs folder under `.env`

> The returned error determines which messages are to be committed and which ones are to be consumed again.

Expand Down
Loading
Loading