Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add KV storage adapters #9913

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: add KV storage adapters #9913

wants to merge 1 commit into from

Conversation

r1tsuu
Copy link
Member

@r1tsuu r1tsuu commented Dec 11, 2024

Adds KV storage support to Payload Local API with an adapter pattern, includes 3 adapters. We'll use it for the Realtime API.
You can access it via payload.kv:

interface KVAdapter {
  /**
   * Clears all entries in the store.
   * @returns A promise that resolves once the store is cleared.
   */
  clear(): Promise<void>

  /**
   * Deletes a value from the store by its key.
   * @param key - The key to delete.
   * @returns A promise that resolves once the key is deleted.
   */
  delete(key: string): Promise<void>

  /**
   * Retrieves a value from the store by its key.
   * @param key - The key to look up.
   * @returns A promise that resolves to the value, or `null` if not found.
   */
  get(key: string): Promise<KVStoreValue | null>

  /**
   * Checks if a key exists in the store.
   * @param key - The key to check.
   * @returns A promise that resolves to `true` if the key exists, otherwise `false`.
   */
  has(key: string): Promise<boolean>

  /**
   * Retrieves all the keys in the store.
   * @returns A promise that resolves to an array of keys.
   */
  keys(): Promise<string[]>

  /**
   * Sets a value in the store with the given key.
   * @param key - The key to associate with the value.
   * @param value - The value to store.
   * @returns A promise that resolves once the value is stored.
   */
  set(key: string, value: KVStoreValue): Promise<void>
}

To configure the adapter you can use the kv property of the root config.

buildConfig({
  kv: adapter()
})

Database KV adapter (default)

No need to configure, as Payload uses it by default. It generates new collection payload-kv and uses the current database adapter to access it. The collection is hidden in the admin panel and access locked.
If you want to override the generated collection:

import { databaseKVAdapter } from 'payload'

buildConfig({
  kv: databaseKVAdapter({
    kvCollectionOverrides: {
      slug: 'custom-kv',
      ...(process.env.DEBUG === 'true' && {
        admin: { hidden: false },
        access: {},
      }),
    },
  }),
})

In Memory KV adapter

Simple and very fast storage using memory. Don't use it on Vercel / multiple instances or if you need data persistence.

import { inMemoryKVAdapter } from 'payload'

buildConfig({
  kv: inMemoryKVAdapter(),
})

Redis KV Adapter

Uses Redis. Probably the best option as it's faster than database, persistent and works with Vercel / multiple instances, but requires additional setup

pnpm add @payloadcms/kv-redis
import { redisKVAdapter } from '@payloadcms/kv-redis'

buildConfig({
  kv: redisKVAdapter({
    keyPrefix: "custom-prefix:", // defaults to 'payload-kv:'
    redisURL: "redis://127.0.0.1:6379" // defaults to process.env.REDIS_URL (Vercel generates this variable for you if you connect a project to Redis)
  }),
})

@r1tsuu r1tsuu requested a review from denolfe as a code owner December 11, 2024 21:43
@r1tsuu r1tsuu changed the title feat: KV storage adapters feat: kv storage adapters Dec 11, 2024
@r1tsuu r1tsuu changed the title feat: kv storage adapters feat: add KV storage adapters Dec 11, 2024
@r1tsuu r1tsuu force-pushed the feat/kv branch 2 times, most recently from 084d143 to 5f29665 Compare December 11, 2024 23:12
r1tsuu added a commit that referenced this pull request Dec 12, 2024
…ating existing ones (#9916)

### What?
`payload.db.updateOne` (and so `payload.db.upsert`) with drizzle
adapters used incoming `where` incorrectly and worked properly only
either if you passed `id` or some where query path required table joins
(like `where: { 'array.title'`) which is also the reason why `upsert`
_worked_ with user preferences specifically, because we need to join the
`preferences_rels` table to query by `user.relationTo` and `user.value`

Fixes #9915

This was found here - #9913,
the database KV adapter uses `upsert` with `where` by unique fields.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant