Skip to content

Commit

Permalink
Add support for namespaces
Browse files Browse the repository at this point in the history
* Add namespaces support for existing endpoints and implement new ListNamespaces and RemoveNamespace methods.

* update documentations.
  • Loading branch information
mehmettokgoz authored May 10, 2024
1 parent 5073751 commit bdd081c
Show file tree
Hide file tree
Showing 23 changed files with 878 additions and 474 deletions.
73 changes: 68 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,42 @@
# Upstash Vector Go Client

[![Go Reference](https://pkg.go.dev/badge/github.com/upstash/vector-go.svg)](https://pkg.go.dev/github.com/upstash/vector-go)

> [!NOTE]
> **This project is in GA Stage.**
>
> The Upstash Professional Support fully covers this project. It receives regular updates, and bug fixes. The Upstash team is committed to maintaining and improving its functionality.
> The Upstash Professional Support fully covers this project. It receives regular updates, and bug fixes.
> The Upstash team is committed to maintaining and improving its functionality.
[Upstash](https://upstash.com/) Vector is a serverless vector database designed for working with vector embeddings.

This is the Go client for [Upstash](https://upstash.com/) Vector.
This is the HTTP-based Go client for [Upstash](https://upstash.com/) Vector.

## Documentation

- [**Reference Documentation**](https://upstash.com/docs/vector/overall/getstarted)

## Installation

Use `go get` to install the Upstash Vector package:
```bash
go get github.com/upstash/vector-go
```

Import the Upstash Vector package in your project:

```go
import "github.com/upstash/vector-go"
```

## Usage

In order to use this client, head out to [Upstash Console](https://console.upstash.com) and create a vector database.

### Initializing the client

There are two pieces of configuration required to use the Upstash Vector index client: an REST token and REST URL.
Find your configuration values in the console dashboard at [https://console.upstash.com/](https://console.upstash.com/).
The REST token and REST URL configurations are required to initialize an Upstash Vector index client.
Find your configuration values in the console dashboard at [Upstash Console](https://console.upstash.com/).

```go
import (
Expand Down Expand Up @@ -80,14 +94,43 @@ func main() {

Upstash vector indexes support operations for working with vector data using operations such as upsert, query, fetch, and delete.

```go
import (
"github.com/upstash/vector-go"
)

func main() {
index := vector.NewIndex("<UPSTASH_VECTOR_REST_URL>", "<UPSTASH_VECTOR_REST_TOKEN>")
}
```

Upstash Vector allows you to partition a single index into multiple isolated namespaces.

You can specify a namespace for an index client with `Namespace(ns string)` function.
When you create a `Namespace` client, all index operations executed through this client become associated with the specified namespace.

By default, the `Index` client is associated with the default namespace.

```go
import (
"github.com/upstash/vector-go"
)

func main() {
index := vector.NewIndex("<UPSTASH_VECTOR_REST_URL>", "<UPSTASH_VECTOR_REST_TOKEN>")

// Returns a new Namespace client associated with the given namespace
ns := index.Namespace("<NAMESPACE>")
```
### Upserting Vectors
All vectors upserted to index must have the same dimensions.
Upsert can be used to insert new vectors into index or to update
existing vectors.
#### Upsert many
#### Upsert Many
```go
upserts := []vector.Upsert{
Expand Down Expand Up @@ -287,3 +330,23 @@ err := index.Reset()
```go
info, err := index.Info()
```
### List Namespaces
All the names of active namespaces can be listed.
```go
namespaces, err := index.ListNamespaces()
for _, ns : range namespaces {
fmt.Println(ns)
}
```
### Delete Namespaces
A namespace can be deleted entirely if it exists.
The default namespaces cannot be deleted.
```go
err := index.Namespace("ns").DeleteNamespace()
```
22 changes: 14 additions & 8 deletions delete.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,19 @@ package vector

const deletePath = "/delete"

// Delete deletes the vector with the given id and reports
// whether the vector is deleted. If a vector with the given
// id is not found, Delete returns false.
// Delete deletes the vector with the given id in the default namespace and reports whether the vector is deleted.
// If a vector with the given id is not found, Delete returns false.
func (ix *Index) Delete(id string) (ok bool, err error) {
data, err := ix.sendBytes(deletePath, []byte(id))
return ix.deleteInternal(id, defaultNamespace)
}

// DeleteMany deletes the vectors with the given ids in the default namespace and reports how many of them are deleted.
func (ix *Index) DeleteMany(ids []string) (count int, err error) {
return ix.deleteManyInternal(ids, defaultNamespace)
}

func (ix *Index) deleteInternal(id string, ns string) (ok bool, err error) {
data, err := ix.sendBytes(buildPath(deletePath, ns), []byte(id))
if err != nil {
return
}
Expand All @@ -16,10 +24,8 @@ func (ix *Index) Delete(id string) (ok bool, err error) {
return
}

// DeleteMany deletes the vectors with the given ids and reports
// how many of them are deleted.
func (ix *Index) DeleteMany(ids []string) (count int, err error) {
data, err := ix.sendJson(deletePath, ids)
func (ix *Index) deleteManyInternal(ids []string, ns string) (count int, err error) {
data, err := ix.sendJson(buildPath(deletePath, ns), ids)
if err != nil {
return
}
Expand Down
91 changes: 49 additions & 42 deletions delete_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,49 +15,56 @@ func randomString() string {
}

func TestDelete(t *testing.T) {
client, err := newTestClient()
require.NoError(t, err)

id := randomString()
err = client.Upsert(Upsert{
Id: id,
Vector: []float32{0, 1},
})
require.NoError(t, err)

t.Run("existing id", func(t *testing.T) {
ok, err := client.Delete(id)
require.NoError(t, err)
require.True(t, ok)
})

t.Run("non existing id", func(t *testing.T) {
ok, err := client.Delete(randomString())
require.NoError(t, err)
require.False(t, ok)
})
for _, ns := range namespaces {
t.Run("namespace_"+ns, func(t *testing.T) {
client, err := newTestClientWithNamespace(ns)
require.NoError(t, err)
id := randomString()
err = client.Upsert(Upsert{
Id: id,
Vector: []float32{0, 1},
})
require.NoError(t, err)

t.Run("existing id", func(t *testing.T) {
ok, err := client.Delete(id)
require.NoError(t, err)
require.True(t, ok)
})

t.Run("non existing id", func(t *testing.T) {
ok, err := client.Delete(randomString())
require.NoError(t, err)
require.False(t, ok)
})
})
}
}

func TestDeleteMany(t *testing.T) {
client, err := newTestClient()
require.NoError(t, err)

id0 := randomString()
id1 := randomString()
id2 := randomString()
err = client.UpsertMany([]Upsert{
{
Id: id0,
Vector: []float32{0, 1},
},
{
Id: id1,
Vector: []float32{5, 10},
},
})
require.NoError(t, err)

count, err := client.DeleteMany([]string{id0, id1, id2})
require.NoError(t, err)
require.Equal(t, 2, count)
for _, ns := range namespaces {
t.Run("namespace "+ns, func(t *testing.T) {
client, err := newTestClientWithNamespace(ns)
require.NoError(t, err)

id0 := randomString()
id1 := randomString()
id2 := randomString()
err = client.UpsertMany([]Upsert{
{
Id: id0,
Vector: []float32{0, 1},
},
{
Id: id1,
Vector: []float32{5, 10},
},
})
require.NoError(t, err)

count, err := client.DeleteMany([]string{id0, id1, id2})
require.NoError(t, err)
require.Equal(t, 2, count)
})
}
}
Loading

0 comments on commit bdd081c

Please sign in to comment.