Code

Add Tabby Serve Eval V2.#2698

Code

Open

moqimoqidea wants to merge 19 commits intoTabbyML:mainTabbyML/tabby:mainfrom moqimoqidea:dev-eval-v2moqimoqidea/tabby:dev-eval-v2Copy head branch name to clipboard

+739

Commits

Commits on Jul 14, 2024

• Eval-V2-Modal: init eval tabby serve v2.

  moqimoqidea

  committed
  582507e

  Copy full SHA for 582507e

Commits on Jul 16, 2024

• Update eval v2: Centralize env var setup in image config.

  moqimoqidea

  committed
  9e0529b

  Copy full SHA for 9e0529b

• rename code completion name.

  Show description for b707007
  

  moqimoqidea

  and

  wsxiaoys

  authored
  b707007

  Copy full SHA for b707007

• rename compte metrics

  Show description for ae35f1a
  

  moqimoqidea

  and

  wsxiaoys

  authored
  ae35f1a

  Copy full SHA for ae35f1a

• Eval-V2-Modal: rename compute py file names.

  moqimoqidea

  committed
  e61d9e3

  Copy full SHA for e61d9e3

• Eval-V2-Modal: add cpu/memory limit; use output_prediction_jsonl_file flag pass prediction file; add retry support for code completion; only save code completion success data to prediction file

  moqimoqidea

  committed
  b891e52

  Copy full SHA for b891e52

• Eval-V2-Modal: set compute_metrics.py output evaluation jsonl file; add avg_metrics.py calc avg metrics;

  moqimoqidea

  committed
  5e2e39d

  Copy full SHA for 5e2e39d

• Eval-V2-Modal: add 20240716-StarCoder-1B-evaluation.jsonl

  moqimoqidea

  committed
  0a3a835

  Copy full SHA for 0a3a835

• Eval-V2-Modal: code optimization

  moqimoqidea

  committed
  18531f7

  Copy full SHA for 18531f7

Commits on Jul 17, 2024

• Eval-V2-Modal: fix calling evaluation function parameter error

  moqimoqidea

  committed
  0302d78

  Copy full SHA for 0302d78

• rename need_manager_modal to start_tabby_server_on_modal

  Show description for c12632a
  

  moqimoqidea

  and

  wsxiaoys

  authored
  c12632a

  Copy full SHA for c12632a

• Eval-V2-Modal: rename need_manager_modal to start_tabby_server_on_modal

  moqimoqidea

  committed
  b8bdcde

  Copy full SHA for b8bdcde

• Eval-V2-Modal: set python file 755 permissions

  moqimoqidea

  committed
  737fd91

  Copy full SHA for 737fd91

• Eval-V2-Modal: add transform_legacy_data.py

  moqimoqidea

  committed
  b9bb4b7

  Copy full SHA for b9bb4b7

• Eval-V2-Modal: add two types requests to eval code completion.

  moqimoqidea

  committed
  350f9d9

  Copy full SHA for 350f9d9

• Eval-V2-Modal: resolve a combined data error in the cross_file_data_function

  moqimoqidea

  committed
  7cbc62f

  Copy full SHA for 7cbc62f

• Eval-V2-Modal: add pands-escape-enforce-problem.py

  moqimoqidea

  committed
  4733c7f

  Copy full SHA for 4733c7f

• Eval-V2-Modal: add CodeLlama-13B eval results.

  moqimoqidea

  committed
  c4a7809

  Copy full SHA for c4a7809

• Eval-V2-Modal: use segments.crossfile_context.text to compute code completion

  moqimoqidea

  committed
  a86d0fe

  Copy full SHA for a86d0fe