Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEAUTURE] argilla: working with webhooks #5502

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
cce6878
add webhook required deps
frascuchon Sep 17, 2024
57a3fd2
feat: Add webhohooks module
frascuchon Sep 17, 2024
5ed0a6a
chore: Expose webhooks module
frascuchon Sep 17, 2024
ba672c7
add client.webhooks container
frascuchon Sep 17, 2024
ecf4467
add missing enabled property
frascuchon Sep 17, 2024
a09b4a6
add missing config to represent webhoooks as html
frascuchon Sep 17, 2024
0a0e74d
prevent parsing errors with .deleted events
frascuchon Sep 17, 2024
6c24db0
add basic example of usage for webhooks
frascuchon Sep 17, 2024
8a6b6cb
update example
frascuchon Sep 17, 2024
fb8e86c
install feature branch
frascuchon Sep 18, 2024
5a6ec5e
Merge branch 'feat/add-webhooks-feature-branch' into feat/argilla/wor…
frascuchon Sep 18, 2024
d2342fb
Merge branch 'feat/argilla/working-with-webhooks' of github.com:argil…
frascuchon Sep 18, 2024
8387e0b
Apply suggestions from code review
frascuchon Sep 18, 2024
2b50302
Merge branch 'feat/argilla/working-with-webhooks' of github.com:argil…
frascuchon Sep 18, 2024
f61ebe0
chore: Tag webhook endpoints
frascuchon Sep 18, 2024
4b9cca0
chore: Define some type checking
frascuchon Sep 18, 2024
3464d85
chore: Using IP address for default webhook server url
frascuchon Sep 18, 2024
8c95e3f
refactor: webhook server url setup
frascuchon Sep 18, 2024
a351a9d
refactor: Parsing SDK resources from webhook event
frascuchon Sep 19, 2024
f41ecbc
Merge branch 'feat/add-webhooks-feature-branch' into feat/argilla/wor…
frascuchon Sep 19, 2024
b9e329e
Merge branch 'feat/add-webhooks-feature-branch' into feat/argilla/wor…
frascuchon Sep 19, 2024
9b78850
refactor: Add record-related properties for response
frascuchon Sep 19, 2024
754fc33
wip
frascuchon Sep 19, 2024
a35bbf6
feat: Linking responses and records
frascuchon Sep 19, 2024
2962e6e
feat: Linking suggestions and records
frascuchon Sep 19, 2024
61309c6
refactor: record.from_model to link responses and suggestions with re…
frascuchon Sep 19, 2024
443b631
chore: Adapt test
frascuchon Sep 19, 2024
7be469b
Merge branch 'refactor/argilla/link-user-responses-and-suggestions-to…
frascuchon Sep 19, 2024
28896cd
prevent creating same webhook multiple times
frascuchon Sep 19, 2024
d1de9ca
Merge branch 'feat/add-webhooks-feature-branch' into feat/argilla/wor…
frascuchon Sep 20, 2024
1c6d573
fix: Prevent error for unlabelled ClassValue columns
frascuchon Sep 20, 2024
eecd0d6
fix: Prevent errors logging records from HF datasets with case sensit…
frascuchon Sep 20, 2024
2376abe
fix: Create dataset from hub when config files are not found
frascuchon Sep 20, 2024
51d0790
prevent run tests without HF token info
frascuchon Sep 20, 2024
055cfa5
fix: Lowercase names
frascuchon Sep 20, 2024
0ac4565
add tests
frascuchon Sep 20, 2024
3ae80e9
feat: Improve feature -> settings
frascuchon Sep 20, 2024
b430a47
adapt tests
frascuchon Sep 20, 2024
df74d46
Merge branch 'bugfixes/argilla/from_hub-method' into feat/argilla/wor…
frascuchon Sep 20, 2024
cfb8451
Merge branch 'feat/argilla/improve-from-hub-capabilities' into feat/a…
frascuchon Sep 20, 2024
6b2d245
feat: Add limit argument when fetching records
frascuchon Sep 23, 2024
5998e78
tests: add tests
frascuchon Sep 23, 2024
20d35c5
tests: add tests
frascuchon Sep 23, 2024
6737631
chore: Update changelog
frascuchon Sep 23, 2024
b673981
Merge branch 'feat/argilla/add-limit-argument-when-getting-records' o…
frascuchon Sep 23, 2024
d5a428c
Merge branch 'feat/add-webhooks-feature-branch' into feat/argilla/wor…
frascuchon Sep 23, 2024
2b31ccc
Merge branch 'feat/add-webhooks-feature-branch' into feat/argilla/wor…
frascuchon Sep 23, 2024
7b4a625
Merge branch 'feat/argilla/working-with-webhooks' of github.com:argil…
frascuchon Sep 23, 2024
e577b9d
Merge branch 'feat/argilla/add-limit-argument-when-getting-records' i…
frascuchon Sep 23, 2024
a248953
chore: Enable found webhooks
frascuchon Sep 23, 2024
14513c6
parse deleted events properly
frascuchon Sep 23, 2024
09b81d1
chore: renaming method
frascuchon Sep 23, 2024
2481d3d
refactor: separate webhook model
frascuchon Sep 24, 2024
7ce3e2e
chore: using event type defs
frascuchon Sep 24, 2024
d42c0e4
refactor: webhook event parsing
frascuchon Sep 24, 2024
4f4c9e1
chore: Renaming helper functions
frascuchon Sep 24, 2024
75b4b4f
Merge branch 'feat/add-webhooks-feature-branch' into feat/argilla/wor…
frascuchon Sep 24, 2024
a90224d
Merge branch 'feat/add-webhooks-feature-branch' into feat/argilla/wor…
frascuchon Oct 2, 2024
7afb126
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 2, 2024
f984f75
Update argilla/CHANGELOG.md
frascuchon Oct 2, 2024
5086830
Merge branch 'feat/add-webhooks-feature-branch' into feat/argilla/wor…
frascuchon Oct 2, 2024
4cabd65
refactor: Creating resource events and rename _server to _helpers
frascuchon Sep 25, 2024
538d641
Add python reference for webhooks
frascuchon Sep 25, 2024
e45f557
fix: Allow arbitrary attributes
frascuchon Sep 25, 2024
a4f7d88
fix: revert wrong merge
frascuchon Oct 2, 2024
db023d0
chore: Add helpers to start and stop the webhook server
frascuchon Oct 2, 2024
6b20b42
[DOCS] Argilla webhooks (#5533)
frascuchon Oct 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion argilla/docs/how_to_guides/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,22 @@ These guides provide step-by-step instructions for common scenarios, including d

<div class="grid cards" markdown>

- __Use webhooks to respond to server events__

---

Learn how to use Argilla webhooks to receive notifications about events in your Argilla Server.

[:octicons-arrow-right-24: How-to guide](webhooks.md)

- __Webhooks internals__

---

Learn how Argilla webhooks are implented under the hood and the structure of the different events.

[:octicons-arrow-right-24: How-to guide](webhooks_internals.md)

- __Use Markdown to format rich content__

---
Expand All @@ -98,4 +114,4 @@ These guides provide step-by-step instructions for common scenarios, including d

[:octicons-arrow-right-24: How-to guide](migrate_from_legacy_datasets.md)

</div>
</div>
160 changes: 160 additions & 0 deletions argilla/docs/how_to_guides/webhooks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
---
description: In this section, we will provide a step-by-step guide to create a webhook in Argilla.
---

# Use Argilla webhooks

This guide provides an overview of how to create and use webhooks in Argilla.

A **webhook** allows an application to submit real-time information to other applications whenever a specific event occurs. Unlike traditional APIs, you won’t need to poll for data very frequently in order to get it in real time. This makes webhooks much more efficient for both the provider and the consumer.

## Creating a webhook listener in Argilla

The python SDK provides a simple way to create a webhook in Argilla. It allows you to focus on the use case of the webhook and not on the implementation details. You only need to create your event handler function with the `webhook_listener` decorator.

```python
import argilla as rg

from datetime import datetime
from argilla import webhook_listener

@webhook_listener(events="dataset.created")
async def my_webhook_handler(dataset: rg.Dataset, type: str, timestamp: datetime):
print(dataset, type, timestamp)
```

In the example above, we have created a webhook that listens to the `dataset.created` event.
> You can find the list of events in the [Events](#events) section.

The python SDK will automatically create a webhook in Argilla and listen to the specified event. When the event is triggered,
the `my_webhook_handler` function will be called with the event data. The SDK will also parse the incoming webhook event into
a proper resource object (`rg.Dataset`, `rg.Record`, and `rg.Response`). The SDK will also take care of request authentication and error handling.

## Running the webhook server

Under the hood, the SDK uses the `FastAPI` framework to create the webhook server and the POST endpoint to receive the webhook events.

To run the webhook, you need to define the webhook server in your code and start it using the `uvicorn` command.

```python
# my_webhook.py file
from argilla import get_webhook_server

server = get_webhook_server()
```

```bash
uvicorn my_webhook:server
```

You can explore the Swagger UI to explore your defined webhooks by visiting `http://localhost:8000/docs`.


The `uvicorn` command will start the webhook server on the default port `8000`.

By default, the Python SDK will register the webhook using the server URL `http://127.0.0.1:8000/`. If you want to use a different server URL, you can set the `WEBHOOK_SERVER_URL` environment variable.

```bash
export WEBHOOK_SERVER_URL=http://my-webhook-server.com
```

All incoming webhook events will be sent to the specified server URL.

## Webhooks management

The Python SDK provides a simple way to manage webhooks in Argilla. You can create, list, update, and delete webhooks using the SDK.

### Create a webhook

To create a new webhook in Argilla, you can define it in the `Webhook` class and then call the `create` method.

```python
import argilla as rg

client = rg.Argilla(api_url="<api_url>", api_key="<api_key>")

webhook = rg.Webhook(
url="http://127.0.0.1:8000",
events=["dataset.created"],
description="My webhook"
)

webhook.create()

```

### List webhooks

You can list all the existing webhooks in Argilla by accessing the `webhooks` attribute on the Argilla class and iterating over them.

```python
import argilla as rg

client = rg.Argilla(api_url="<api_url>", api_key="<api_key>")

for webhook in client.webhooks:
print(webhook)

```

### Update a webhook

You can update a webhook using the `update` method.

```python
import argilla as rg

client = rg.Argilla(api_url="<api_url>", api_key="<api_key>")

webhook = rg.Webhook(
url="http://127.0.0.1:8000",
events=["dataset.created"],
description="My webhook"
).create()

webhook.events = ["dataset.updated"]
webhook.update()

```
> You should use IP address instead of localhost since the webhook validation expect a Top Level Domain (TLD) in the URL.

### Delete a webhook

You can delete a webhook using the `delete` method.

```python
import argilla as rg

client = rg.Argilla(api_url="<api_url>", api_key="<api_key>")

for webhook in client.webhooks:
webhook.delete()

```

## Deploying a webhooks server in a Hugging Face Space

You can deploy your webhook in a Hugging Face Space. You can visit this [link](https://huggingface.co/spaces/argilla/argilla-webhooks/tree/main) to explore an example of a webhook server deployed in a Hugging Face Space.


## Events

The following is a list of events that you can listen to in Argilla, grouped by resource type.

### Dataset events

- `dataset.created`: The Dataset resource was created.
- `dataset.updated`: The Dataset resource was updated.
- `dataset.deleted`: The Dataset resource was deleted.
- `dataset.published`: The Dataset resource was published.

### Record events
- `record.created`: The Record resource was created.
- `record.updated`: The Record resource was updated.
- `record.deleted`: The Record resource was deleted.
- `record.completed`: The Record resource was completed (status="completed").

### Response events
- `response.created`: The Response resource was created.
- `response.updated`: The Response resource was updated.
- `response.deleted`: The Response resource was deleted.
Loading
Loading