Last access time for database or collection

[vasudevabhat]

Is it possible to track last access time for database or collection in Milvus?
Idea is to delete unused database/collection if it breaches threshold time.

[yhmo]

You can try access log: https://milvus.io/docs/configure_access_logs.md#Configure-Access-Logs

You can configure it in the milvus.yaml:

proxy:
  accessLog:
    enable: true
    filename: "milvus-access.log"
    localPath: "/tmp/milvus"

With this configuration, the /tmp/milvus/milvus-access.log is generated inside the milvus container, you can map the path outside the container, or enter the container to read it.

You can know which interface is called by which client. Access log of search/query request tells which database and which collection is accessed.

For example, an access log of a search request:
[2024/11/07 07:11:24.048 +00:00] [ACCESS] <Unknown: tcp-172.22.0.1:46194> Search [status: Successful] [code: 0] [sdk: Python-2.4.7] [msg: ] [traceID: e3358449e02f19c04426929af4bc6ed7] [timeCost: 54.774908ms] [database: test_dv] [collection: dev_collection] [partitions: []] [expr: ]

[vasudevabhat]

Thanks!

[huangxueliang]

#With this configuration, the /tmp/milvus/milvus-access.log is generated inside the milvus container, you can map the path outside the container, or enter the container to read it.

how can i map the path outside the container?

[yhmo]

In the docker-compose.yaml, add one line under the standalone.volumes

- ${DOCKER_VOLUME_DIRECTORY:-.}/milvus-access.log:/tmp/milvus/milvus-access.log

standalone:
    container_name: milvus-standalone
    image: milvusdb/milvus:v2.4.15
    command: ["milvus", "run", "standalone"]
    security_opt:
    - seccomp:unconfined
    environment:
      ETCD_ENDPOINTS: etcd:2379
      MINIO_ADDRESS: minio:9000
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/milvus:/var/lib/milvus
      - ${DOCKER_VOLUME_DIRECTORY:-.}/milvus.yaml:/milvus/configs/milvus.yaml
      - ${DOCKER_VOLUME_DIRECTORY:-.}/milvus-access.log:/tmp/milvus/milvus-access.log