Skip to content
/ openfaas-wal-listener Public

Example repo that shows how to use wal-listener to trigger your OpenFaaS functions

7 stars 1 fork Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

LucasRoesler/openfaas-wal-listener

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
postgres
postgres
 
 
wal_listener
wal_listener
 
 
Makefile
Makefile
 
 
README.md
README.md
 
 
docker-compose.yaml
docker-compose.yaml
 
 

Repository files navigation

Event Driven OpenFaaS with Postgres

This is an sample of how to create an event driven application using

  1. OpenFaaS faasd
  2. Postgres
  3. NATS
  4. my fork of wal-listener

The goal is that we can call OpenFaaS functions in response to Create, Update, and Delete events in the db.

Install

  1. install faasd

  2. Copy the new docker-compose

    make install 
# or manually run 
# sudo cp -rf postgres /var/lib/faasd
 # sudo mkdir -p /var/lib/faasd/postgres/pgdata
 # sudo mkdir -p /var/lib/faasd/postgresql/run
 # sudo cp -rf wal_listener /var/lib/faasd
 # sudo cp docker-compose.yaml /var/lib/faasd
 # sudo chown -R 1000:1000 /var/lib/faasd/postgres

  3. Restart faasd

    make restart
# or manually: sudo systemctl restart faasd faasd-provider

  4. Deploy the same receiver function

    faas-cli deploy --name receive-event --image theaxer/receive-message:latest --fprocess='./handler' --annotation topic="sample_app"

    You can then verify that the connector is sending events to tne receiver fucntion by manually publishing the topic (you need to install the natscli)

    $ nats pub sample_app "manual push"
15:21:39 Published 11 bytes to "sample_app"
$ faas-cli logs receive-event
WARNING! Communication is not secure, please consider using HTTPS. Letsencrypt.org offers free SSL/TLS certificates.
2021-01-24T14:19:35Z 2021/01/24 14:19:35 POST / - 200 OK - ContentLength: 27
2021-01-24T14:19:35Z 2021/01/24 14:19:35 stderr: 2021/01/24 14:19:35 received "manual push"
2021-01-24T14:21:17Z 2021/01/24 14:21:17 SIGTERM received.. shutting down server in 10s
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Removing lock-file : /tmp/.lock
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Error scanning stdout: read |0: file already closed
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Error scanning stderr: read |0: file already closed
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Forked function has terminated: signal: terminated
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Started logging stderr from function.
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Started logging stdout from function.
2021-01-24T14:21:17Z 2021/01/24 14:21:17 OperationalMode: http
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Timeouts: read: 10s, write: 10s hard: 10s.
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Listening on port: 8080
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Writing lock-file to: /tmp/.lock
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Metrics listening on port: 8081
2021-01-24T14:21:17Z Forking - ./handler []
2021-01-24T14:21:39Z 2021/01/24 14:21:39 POST / - 200 OK - ContentLength: 27
2021-01-24T14:21:39Z 2021/01/24 14:21:39 stderr: 2021/01/24 14:21:39 received "manual push"

Verifying db events are sent

  1. Generate some changes in the db

    make init-db
# psql -U postgres -h localhost -d app -f postgres/init_app.sql

  2. You can then see the logs as wal-listener reacts to the inserts by using

    make logs service=wal-listener

  3. Finally, see that our receiver function was invoked

    $ faas-cli logs receive-event
WARNING! Communication is not secure, please consider using HTTPS. Letsencrypt.org offers free SSL/TLS certificates.
2021-01-24T14:19:35Z 2021/01/24 14:19:35 POST / - 200 OK - ContentLength: 27
2021-01-24T14:19:35Z 2021/01/24 14:19:35 stderr: 2021/01/24 14:19:35 received "manual push"
2021-01-24T14:21:17Z 2021/01/24 14:21:17 SIGTERM received.. shutting down server in 10s
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Removing lock-file : /tmp/.lock
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Error scanning stdout: read |0: file already closed
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Error scanning stderr: read |0: file already closed
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Forked function has terminated: signal: terminated
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Started logging stderr from function.
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Started logging stdout from function.
2021-01-24T14:21:17Z 2021/01/24 14:21:17 OperationalMode: http
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Timeouts: read: 10s, write: 10s hard: 10s.
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Listening on port: 8080
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Writing lock-file to: /tmp/.lock
2021-01-24T14:21:17Z 2021/01/24 14:21:17 Metrics listening on port: 8081
2021-01-24T14:21:17Z Forking - ./handler []
2021-01-24T14:21:39Z 2021/01/24 14:21:39 POST / - 200 OK - ContentLength: 27
2021-01-24T14:21:39Z 2021/01/24 14:21:39 stderr: 2021/01/24 14:21:39 received "manual push"
2021-01-24T15:04:25Z 2021/01/24 15:04:25 POST / - 200 OK - ContentLength: 386
2021-01-24T15:04:25Z 2021/01/24 15:04:25 stderr: 2021/01/24 15:04:25 received "{\"id\":\"92c431a7-9027-4656-9d39-f52ee90d5dd6\",\"schema\":\"public\",\"table\":\"users\",\"action\":\"UPDATE\",\"data\":{\"created_at\":\"2021-01-23 16:50:53.543372+00\",\"updated_at\":\"2021-01-23 16:50:53.543372+00\",\"name\":\"John Wick\",\"email\":\"[email protected]\",\"id\":\"99a9c7bf-8f31-4c30-998e-9740f87bdaa0\"},\"commitTime\":\"2021-01-24T15:04:25.58604Z\"}"
2021-01-24T15:04:25Z 2021/01/24 15:04:25 POST / - 200 OK - ContentLength: 383
2021-01-24T15:04:25Z 2021/01/24 15:04:25 stderr: 2021/01/24 15:04:25 received "{\"id\":\"80a8d33a-0ae5-4aed-825d-1a7f9e24adfb\",\"schema\":\"public\",\"table\":\"users\",\"action\":\"UPDATE\",\"data\":{\"name\":\"James Bond\",\"email\":\"[email protected]\",\"id\":\"5a6a9672-722a-40c9-8f10-17cf527a6b41\",\"created_at\":\"2021-01-23 16:50:53.543372+00\",\"updated_at\":\"2021-01-23 16:50:53.543372+00\"},\"commitTime\":\"2021-01-24T15:04:25.58604Z\"}"
2021-01-24T15:04:25Z 2021/01/24 15:04:25 POST / - 200 OK - ContentLength: 384
2021-01-24T15:04:25Z 2021/01/24 15:04:25 stderr: 2021/01/24 15:04:25 received "{\"id\":\"326ea179-dddb-4c04-bd56-9f02bdec9aaa\",\"schema\":\"public\",\"table\":\"users\",\"action\":\"UPDATE\",\"data\":{\"created_at\":\"2021-01-23 16:50:53.543372+00\",\"updated_at\":\"2021-01-23 16:50:53.543372+00\",\"name\":\"Elim Garak\",\"email\":\"[email protected]\",\"id\":\"b1f3cc48-6366-4bff-b0c5-3f5beed02f44\"},\"commitTime\":\"2021-01-24T15:04:25.58604Z\"}"
2021-01-24T15:04:25Z 2021/01/24 15:04:25 POST / - 200 OK - ContentLength: 392
2021-01-24T15:04:25Z 2021/01/24 15:04:25 stderr: 2021/01/24 15:04:25 received "{\"id\":\"15755624-195a-4617-8d60-6b7cb748caf9\",\"schema\":\"public\",\"table\":\"users\",\"action\":\"UPDATE\",\"data\":{\"id\":\"3879dc32-78cd-456d-8a64-fe7fab540a7f\",\"created_at\":\"2021-01-23 16:50:53.543372+00\",\"updated_at\":\"2021-01-23 16:50:53.543372+00\",\"name\":\"Sarah Walker\",\"email\":\"[email protected]\"},\"commitTime\":\"2021-01-24T15:04:25.58604Z\"}"
2021-01-24T15:05:00Z 2021/01/24 15:05:00 POST / - 200 OK - ContentLength: 387
2021-01-24T15:05:00Z 2021/01/24 15:05:00 stderr: 2021/01/24 15:05:00 received "{\"id\":\"38f4ae55-86a1-466a-8ebd-c2f5a1054439\",\"schema\":\"public\",\"table\":\"users\",\"action\":\"UPDATE\",\"data\":{\"updated_at\":\"2021-01-23 16:50:53.543372+00\",\"name\":\"John Wick\",\"email\":\"[email protected]\",\"id\":\"99a9c7bf-8f31-4c30-998e-9740f87bdaa0\",\"created_at\":\"2021-01-23 16:50:53.543372+00\"},\"commitTime\":\"2021-01-24T15:05:00.007715Z\"}"
2021-01-24T15:05:00Z 2021/01/24 15:05:00 stderr: 2021/01/24 15:05:00 received "{\"id\":\"bce01097-815d-4d73-90b9-89230ddca39b\",\"schema\":\"public\",\"table\":\"users\",\"action\":\"UPDATE\",\"data\":{\"email\":\"[email protected]\",\"id\":\"5a6a9672-722a-40c9-8f10-17cf527a6b41\",\"created_at\":\"2021-01-23 16:50:53.543372+00\",\"updated_at\":\"2021-01-23 16:50:53.543372+00\",\"name\":\"James Bond\"},\"commitTime\":\"2021-01-24T15:05:00.007715Z\"}"
2021-01-24T15:05:00Z 2021/01/24 15:05:00 POST / - 200 OK - ContentLength: 384
2021-01-24T15:05:00Z 2021/01/24 15:05:00 POST / - 200 OK - ContentLength: 385
2021-01-24T15:05:00Z 2021/01/24 15:05:00 stderr: 2021/01/24 15:05:00 received "{\"id\":\"45f0fc1f-a44f-4546-a7dd-2fec7079667d\",\"schema\":\"public\",\"table\":\"users\",\"action\":\"UPDATE\",\"data\":{\"name\":\"Elim Garak\",\"email\":\"[email protected]\",\"id\":\"b1f3cc48-6366-4bff-b0c5-3f5beed02f44\",\"created_at\":\"2021-01-23 16:50:53.543372+00\",\"updated_at\":\"2021-01-23 16:50:53.543372+00\"},\"commitTime\":\"2021-01-24T15:05:00.007715Z\"}"
2021-01-24T15:05:00Z 2021/01/24 15:05:00 POST / - 200 OK - ContentLength: 393
2021-01-24T15:05:00Z 2021/01/24 15:05:00 stderr: 2021/01/24 15:05:00 received "{\"id\":\"cd03a0b0-8c98-4809-9347-9f469c773de0\",\"schema\":\"public\",\"table\":\"users\",\"action\":\"UPDATE\",\"data\":{\"created_at\":\"2021-01-23 16:50:53.543372+00\",\"updated_at\":\"2021-01-23 16:50:53.543372+00\",\"name\":\"Sarah Walker\",\"email\":\"[email protected]\",\"id\":\"3879dc32-78cd-456d-8a64-fe7fab540a7f\"},\"commitTime\":\"2021-01-24T15:05:00.007715Z\"}"

About

Example repo that shows how to use wal-listener to trigger your OpenFaaS functions

Resources

Readme
Activity

Stars

7 stars

Watchers

2 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages

No packages published

Languages