Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bugfix/indexing UI #2879

Draft
wants to merge 32 commits into
base: main
Choose a base branch
from
Draft

Bugfix/indexing UI #2879

wants to merge 32 commits into from
+73 −24

Commits on Oct 14, 2024

  1. fresh indexing feature branch

    @rkuo-danswer
    rkuo-danswer committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    1f8bafd View commit details
    Browse the repository at this point in the history

  2. cherry pick test

    @rkuo-danswer
    rkuo-danswer committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    2a62422 View commit details
    Browse the repository at this point in the history

  3. Revert "cherry pick test" 

    This reverts commit 2a62422.
    @rkuo-danswer
    rkuo-danswer committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    57823a2 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of https://github.com/danswer-ai/danswer into fea… 

    …ture/background_indexing_2
    @rkuo-danswer
    rkuo-danswer committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    77710cf View commit details
    Browse the repository at this point in the history

  5. set multitenant so that vespa fields match when indexing

    @rkuo-danswer
    rkuo-danswer committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    d8f19f2 View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2024

  1. cleanup pass

    @rkuo-danswer
    rkuo-danswer committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    96f68a6 View commit details
    Browse the repository at this point in the history

  2. mypy

    @rkuo-danswer
    rkuo-danswer committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    e8f132d View commit details
    Browse the repository at this point in the history

  3. pass through env var to control celery indexing concurrency

    @rkuo-danswer
    rkuo-danswer committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    58a42a2 View commit details
    Browse the repository at this point in the history

  4. comments on task kickoff and some logging improvements

    @rkuo-danswer
    rkuo-danswer committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    9a8a54a View commit details
    Browse the repository at this point in the history

  5. disentangle configuration for different workers and beats.

    @rkuo-danswer
    rkuo-danswer committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    409099b View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. use get_session_with_tenant

    @rkuo-danswer
    rkuo-danswer committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    3d2ccd8 View commit details
    Browse the repository at this point in the history

  2. comment out all of update.py

    @rkuo-danswer
    rkuo-danswer committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    5af90c8 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. Merge branch 'main' of https://github.com/danswer-ai/danswer into fea… 

    …ture/background_indexing_2

# Conflicts:
#	backend/danswer/background/celery/celery_app.py
#	backend/danswer/background/celery/tasks/shared/tasks.py
#	backend/danswer/background/celery/tasks/vespa/tasks.py
#	backend/danswer/background/update.py
#	backend/danswer/db/index_attempt.py
    @rkuo-danswer
    rkuo-danswer committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    d3d0d62 View commit details
    Browse the repository at this point in the history

  2. rename to RedisConnectorIndexingFenceData

    @rkuo-danswer
    rkuo-danswer committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    2b0f09a View commit details
    Browse the repository at this point in the history

  3. first check num_indexing_workers

    @rkuo-danswer
    rkuo-danswer committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    4296f02 View commit details
    Browse the repository at this point in the history

  4. refactor RedisConnectorIndexingFenceData

    @rkuo-danswer
    rkuo-danswer committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    9921251 View commit details
    Browse the repository at this point in the history

  5. comment out on_worker_process_init

    @rkuo-danswer
    rkuo-danswer committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    744f3c4 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' of https://github.com/danswer-ai/danswer into fea… 

    …ture/background_indexing_2

# Conflicts:
#	backend/danswer/db/index_attempt.py
    @rkuo-danswer
    rkuo-danswer committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    16e8d02 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'feature/background_indexing_2' of https://github.com/da… 

    …nswer-ai/danswer into feature/celery_refactor

# Conflicts:
#	backend/danswer/background/celery/celery_app.py
#	backend/danswer/background/celery/tasks/indexing/tasks.py
#	backend/danswer/background/celery/tasks/shared/tasks.py
#	backend/danswer/background/celery/tasks/vespa/tasks.py
#	backend/ee/danswer/background/celery/apps/primary.py
    @rkuo-danswer
    rkuo-danswer committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    6ddd5db View commit details
    Browse the repository at this point in the history

  8. missed a file

    @rkuo-danswer
    rkuo-danswer committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    4aec91f View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2024

  1. scope db sessions to short lengths

    @rkuo-danswer
    rkuo-danswer committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    649b357 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. Merge branch 'main' of https://github.com/danswer-ai/danswer into fea… 

    …ture/celery_refactor

# Conflicts:
#	backend/alembic/env.py
#	backend/danswer/background/celery/celery_app.py
#	backend/danswer/background/celery/configs/base.py
#	backend/danswer/background/celery/tasks/indexing/tasks.py
#	backend/danswer/background/celery/tasks/pruning/tasks.py
#	backend/danswer/background/celery/tasks/vespa/tasks.py
#	backend/danswer/danswerbot/slack/listener.py
#	backend/danswer/document_index/vespa/index.py
#	backend/danswer/server/documents/connector.py
#	backend/scripts/dev_run_background_jobs.py
#	backend/supervisord.conf
    @rkuo-danswer
    rkuo-danswer committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    f6acef8 View commit details
    Browse the repository at this point in the history

  2. update launch.json template

    @rkuo-danswer
    rkuo-danswer committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    71efa82 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. Merge branch 'main' of https://github.com/danswer-ai/danswer into fea… 

    …ture/celery_refactor
    @rkuo-danswer
    rkuo-danswer committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    25432d5 View commit details
    Browse the repository at this point in the history

  2. fix types

    @rkuo-danswer
    rkuo-danswer committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    26bd6c8 View commit details
    Browse the repository at this point in the history

  3. keep index button disabled until indexing is truly finished

    @rkuo-danswer
    rkuo-danswer committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    6060ac6 View commit details
    Browse the repository at this point in the history

  4. change priority order of tooltips

    @rkuo-danswer
    rkuo-danswer committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    98da28d View commit details
    Browse the repository at this point in the history

  5. should be using the logger from app_base

    @rkuo-danswer
    rkuo-danswer committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    eac6607 View commit details
    Browse the repository at this point in the history

  6. if we run out of retries, just mark the doc as modified so it gets sy… 

    …nced later
    @rkuo-danswer
    rkuo-danswer committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    e7761a4 View commit details
    Browse the repository at this point in the history

  7. tighten up the logging ... we know these are ID's

    @rkuo-danswer
    rkuo-danswer committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    355fa0e View commit details
    Browse the repository at this point in the history

  8. add logging

    @rkuo-danswer
    rkuo-danswer committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    03c4b7e View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. Merge branch 'main' of https://github.com/danswer-ai/danswer into bug… 

    …fix/indexing_ui

# Conflicts:
#	backend/danswer/background/celery/apps/primary.py
#	backend/danswer/background/celery/configs/indexing.py
#	backend/danswer/background/celery/configs/light.py
#	backend/danswer/background/celery/versioned_apps/heavy.py
#	backend/danswer/background/celery/versioned_apps/indexing.py
#	backend/danswer/background/celery/versioned_apps/light.py
#	backend/supervisord.conf
    @rkuo-danswer
    rkuo-danswer committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    94bd840 View commit details
    Browse the repository at this point in the history