Update SQL/PPL multiple value field limitation (#8646)

* Update SQL/PPL multiple value field limitation

Signed-off-by: Peng Huo <[email protected]>

* add plugins.query.field_type_tolerance setting

Signed-off-by: Peng Huo <[email protected]>

* address comments

Signed-off-by: Peng Huo <[email protected]>

* Apply suggestions from code review

Signed-off-by: kolchfa-aws <[email protected]>

* Update _search-plugins/sql/limitation.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

---------

Signed-off-by: Peng Huo <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>

Author: 3 people

_search-plugins/sql/limitation.md

@@ -94,7 +94,7 @@ Such queries are successfully executed by the `V2` engine unless they have `V1`-
 * `json` formatted output is supported in `V1` engine only. 
 * The `V2` engine does not track query execution time, so slow queries are not reported.
 * The `V2` query engine not only runs queries in the OpenSearch engine but also supports post-processing for complex queries. Accordingly, the `explain` output is no longer OpenSearch domain-specific language (DSL) but also includes query plan information from the `V2` query engine.
-Suggested change
 * The `V2` query engine does not support aggregation queries such as `histogram`, `date_histogram`, `percentiles`, `topHits`, `stats`, `extended_stats`, `terms`, or `range`.
 * JOINs and sub-queries are not supported. To stay up to date on the development for JOINs and sub-queries, track [GitHub issue #1441](https://github.com/opensearch-project/sql/issues/1441) and [GitHub issue #892](https://github.com/opensearch-project/sql/issues/892).
-* PartiQL syntax for `nested` queries are not supported.  Additionally, arrays of objects and primitive types return the first index of the array, while in `V1` they return the entire array as a JSON object. 
+* OpenSearch does not natively support the array data type but does allow multi-value fields implicitly. The SQL/PPL plugin adheres strictly to the data type semantics defined in index mappings. When parsing OpenSearch responses, it expects data to match the declared type and does not interpret all data in an array. If the [`plugins.query.field_type_tolerance`](https://github.com/opensearch-project/sql/blob/main/docs/user/admin/settings.rst#plugins-query-field-type-tolerance) setting is enabled, the SQL/PPL plugin handles array datasets by returning scalar data types, allowing basic queries (for example, `SELECT * FROM tbl WHERE condition`). However, using multi-value fields in expressions or functions will result in exceptions. If this setting is disabled or not set, only the first element of an array is returned, preserving the default behavior.
+* PartiQL syntax for `nested` queries is not supported.

_search-plugins/sql/settings.md

@@ -79,6 +79,7 @@ Setting | Default | Description
 `plugins.query.memory_limit` | 85% | Configures the heap memory usage limit for the circuit breaker of the query engine.
 `plugins.query.size_limit` | 200 | Sets the default size of index that the query engine fetches from OpenSearch.
 `plugins.query.datasources.enabled` | true | Change to `false` to disable support for data sources in the plugin.
+`plugins.query.field_type_tolerance` | true | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` will be reduced to `1`. If `true`, then the array is preserved. Default is `true`.
 
 ## Spark connector settings