sparse: disable refinement by default #1033
Conversation
|
@sparknack 🔍 Important: PR Classification Needed! For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:
For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”. Thanks for your efforts and contribution to the community!. |
sparknack
changed the title
sparknack
force-pushed
the
sparse-refine
branch
from
a5d3373 to
74baa01
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1033 +/- ##
=========================================
+ Coverage 0 73.15% +73.15%
=========================================
Files 0 82 +82
Lines 0 7479 +7479
=========================================
+ Hits 0 5471 +5471
- Misses 0 2008 +2008 |
|
lgtm, but should it affect existing users? |
Yes. For the current code in main branch, if refinement is used, it will lead to a significant degradation in performance, thus affecting the usability of all range of the drop_ratio_search. Although turning off refinement by default will cause a certain decline in the recall rate with the same drop_ratio_search, it can still ensure that it is basically usable. Users still can use small drop_ratio_search like 0.1. |
| KNOWHERE_CONFIG_DECLARE_FIELD(refine_factor) | ||
| .description("refine factor") | ||
| .set_default(10) | ||
| .set_default(1) | ||
| .set_range(1, 10, true, true) | ||
| .for_search() | ||
| .for_range_search(); |
There was a problem hiding this comment.
remove for_range_search
| KNOWHERE_CONFIG_DECLARE_FIELD(refine_factor) | ||
| .description("refine factor") | ||
| .set_default(10) | ||
| .set_default(1) | ||
| .set_range(1, 10, true, true) |
There was a problem hiding this comment.
no need to set a upper limit
|
issue: #1035 |
sparknack
force-pushed
the
sparse-refine
branch
from
74baa01 to
2b9ee37
Compare
Because of the removal of forward index, the current refine searching is slower than before. To prevent performance degradation when using a non-zero value of drop_ratio_search, disable refinement by default. Signed-off-by: Shawn Wang <[email protected]>
sparknack
force-pushed
the
sparse-refine
branch
from
2b9ee37 to
db09d51
Compare
|
/lgtm |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: sparknack, zhengbuqian The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
sparknack
changed the title
|
/kind improvement |
Because of the removal of forward index, the current refine searching is
slower than before. To prevent performance degradation when using a
non-zero value of drop_ratio_search, disable refinement by default.