sparse: refactor approx dimension max score ratio #1029
Conversation
|
@sparknack 🔍 Important: PR Classification Needed! For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:
For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”. Thanks for your efforts and contribution to the community!. |
sparknack
force-pushed
the
sparse-approx
branch
3 times, most recently
from
f5f5eb7 to
bebde54
Compare
| auto refine_factor = cfg.refine_factor.value_or(10); | ||
| // if no data was dropped during search, no refinement is needed. | ||
| if (drop_ratio_search == 0) { | ||
| refine_factor = 1; |
There was a problem hiding this comment.
a dim_max_score_ratio < 1 also leads to info loss, we still want to do refine
There was a problem hiding this comment.
For now, we keep the choice of no refinement. Here's why:
A value of refine_factor higher than 1 tends to cause the result of dim_max_score_ratio to regress, and it doesn't bring obvious benefits to the growth of recall.
Test Results with MSMARCO BM25
| max score ratio | refine_factor | Recall rate of WAND(%) | Recall rate of MaxScore(%) | Query time of WAND(ms) | Query time of MaxScore(ms) |
|---|---|---|---|---|---|
| 1.1 | 1 | 0.996691 | 0.996576 | 4002 | 3861 |
| 1 | 1 | 0.996691 | 0.996576 | 3375 | 3456 |
| 0.9 | 1 | 0.988983 | 0.992708 | 2599 | 2908 |
| 0.8 | 1 | 0.942851 | 0.981490 | 1999 | 2417 |
| 0.7 | 1 | 0.797994 | 0.936948 | 1474 | 1912 |
| 0.6 | 1 | 0.568309 | 0.793997 | 1080 | 1409 |
| 0.5 | 1 | 0.369398 | 0.546805 | 827 | 879 |
| 0.9 | 2 | 0.993095 | 0.993897 | 4810 | 4691 |
| 0.8 | 2 | 0.970501 | 0.987607 | 4129 | 4178 |
| 0.7 | 2 | 0.880444 | 0.962579 | 3515 | 3609 |
| 0.6 | 2 | 0.709699 | 0.872235 | 3014 | 3018 |
| 0.5 | 2 | 0.516633 | 0.668095 | 2587 | 2296 |
| */ | ||
| KNOWHERE_CONFIG_DECLARE_FIELD(wand_bm25_max_score_ratio) | ||
| KNOWHERE_CONFIG_DECLARE_FIELD(dim_max_score_ratio) | ||
| .set_range(0.5, 1.3) | ||
| .set_default(1.05) | ||
| .description("ratio to upscale/downscale the max score of each dimension") |
There was a problem hiding this comment.
if set to below 1, make sure to check if you want to set refine_factor to a lower value
sparknack
force-pushed
the
sparse-approx
branch
from
bebde54 to
5c86ee0
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1029 +/- ##
=========================================
+ Coverage 0 73.12% +73.12%
=========================================
Files 0 82 +82
Lines 0 7495 +7495
=========================================
+ Hits 0 5481 +5481
- Misses 0 2014 +2014 |
|
issue: #1035 |
|
/kind improvement |
|
@sparknack: The label(s) In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/kind improvement |
|
/lgtm |
1. Move the dimension max score ratio from build params to search params, and rename it from `wand_bm25_max_score_ratio` to `dim_max_score_ratio`. 2. Remove template param `bm25` and add a new `SparseMetricType`. 3. Wrap some params of `Search()` to `InvertedIndexApproxSearchParams`. Signed-off-by: Shawn Wang <[email protected]>
sparknack
force-pushed
the
sparse-approx
branch
from
5c86ee0 to
e85e59c
Compare
Signed-off-by: Shawn Wang <[email protected]>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: foxspy, sparknack, zhengbuqian The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Move the dimension max score ratio from build params to search params, and rename it from
wand_bm25_max_score_ratiotodim_max_score_ratio.Remove template param
bm25and add a newSparseMetricType.Wrap some params of
Search()toInvertedIndexSearchParams.