[BUG] 2.15 Hybrid Search With query_text and radial vector search producing array out of bounds #973
Comments
|
@iyoung thank you for informing us about this scenario. I have tried following scenario, it works fine on my side:
I tried multiple search words and different values of |
|
Same for my query after 2.13 > 2.15 upgrade. Especially this:
Issue happens randomly and it is possible to reproduce only for several minutes/hours. I cannot reproduce it for totally the same query after (probably, index data changes affect this). 2 nodes, 1 primary shard, 1 replica shard, ~600k documents (~13GB), Query is mostly the same as topic starter query but with three subqueries (text search + 2 knn). And also no Any ideas? |
Thank you, I am in contact with the Opensearch managed service team within AWS about this issue and looking to replicate this with a smaller index. The index we're using has around 2m documents in it. Once I have a more concrete way to reproduce this I will update. Thank you for replying. |
What is the bug?
Running a hybrid query which contains a min score for the vector side below 0.5 and providing a query_text form lexical search for certain searches (possibly related to number of matches) results in the following response:
{ "error": { "root_cause": [ { "type": "index_out_of_bounds_exception", "reason": "index_out_of_bounds_exception: null" } ], "type": "search_phase_execution_exception", "reason": "all shards failed", "phase": "query", "grouped": true, "failed_shards": [ { "shard": 0, "index": "xxxxxx", "node": "xxxxxx", "reason": { "type": "index_out_of_bounds_exception", "reason": "index_out_of_bounds_exception: null" } } ] }, "status": 500 }How can one reproduce the bug?
This is the structure of query I am using which always throws the exception.
{ "size": 50, "track_total_hits": true, "query": { "hybrid": { "queries": [ { "knn": { "vector": { "vector": [ -0.018432617, 0.05419922, ... ], "min_score": 0.4 } } }, { "query_string": { "fields": [ "essential_words^2", "caption^3", "description^3", "main_words", "extra_words" ], "query": "\"cold weather\"", "default_operator": "AND" } } ] } }, "post_filter": { "bool": { "must": [ { "exists": { "field": "vector" } } ] } }, "search_pipeline": { "description": "Inline post processor for hybrid search", "phase_results_processors": [ { "normalization-processor": { "normalization": { "technique": "min_max" }, "combination": { "technique": "arithmetic_mean", "parameters": { "weights": [ 0.5, 0.5 ] } } } } ] } }What is the expected behaviour?
Search results returned
What is your host/environment?
AWS managed Opensearch 2.15
Do you have any additional context?
Increasing the radial search threshold by increasing the min score on the vector search to 0.5 to 1 stops this, also by changing the query term or fields included and therefore the matched items also avoids this.
There are numerous exact phrase search terms which cause this issue for us, such as "group hiking" and "cold weather"
In the error logs I get: -
The text was updated successfully, but these errors were encountered: