Fulltextsearch bulk indexing fails with HTTP 400 due to incompatible Elasticsearch client headers

[harryv1]

During further analysis I checked the Nextcloud application log (nextcloud.log). This revealed a concrete error during fulltextsearch operations that does not show up in the Elasticsearch server logs.

Relevant log excerpt (shortened):
OCA\FullTextSearch_Elasticsearch\Vendor8\Elastic\Elasticsearch\Exception\ClientResponseException
400 Bad Request
Invalid media-type value on headers [Accept, Content-Type].
Accept version must be either version 8 or 7, but found 9.
Accept=application/vnd.elasticsearch+json; compatible-with=9

This error appears during bulk indexing and index or pipeline related operations triggered by
php occ fulltextsearch:index and similar commands.

Analysis
The log indicates that the Elasticsearch PHP client used by fulltextsearch_elasticsearch sends HTTP headers with compatible-with=9. Elasticsearch 8.19.x explicitly rejects these headers and responds with HTTP 400.

This also explains why the behavior is inconsistent and hard to reproduce:
Commands like fulltextsearch:check or fulltextsearch:test still work, because they do not appear to exercise the same code path as productive bulk indexing. These commands likely use simpler request paths without bulk payloads or attachment processing and therefore do not trigger the same header validation.
In contrast, productive indexing via fulltextsearch:index uses bulk requests and the attachment processing path, which results in Elasticsearch rejecting the request before ingest pipelines or attachment processing are applied.

As a result:
the ingest-attachment pipeline exists but is never reached
documents may be created or updated with metadata only
PDF content remains empty
bulk indexing fails reproducibly with ClientResponseException
Elasticsearch server logs remain largely clean

Conclusion (as a hypothesis)
It is likely that the core issue is a compatibility mismatch between the Elasticsearch PHP client bundled with fulltextsearch_elasticsearch (sending compatible-with=9 headers) and Elasticsearch 8.x, which only accepts compatible-with=7 or 8.
The observed behavior strongly suggests that different internal code paths are used for simple test/check operations versus productive bulk indexing with attachments, and that the incompatibility manifests only in the latter path.
Possible next steps would be to align the client headers with ES 8.x, or to wait for a release where ES9 compatibility is paired with an actual ES9 backend.

[szaimen]

Hi, this needs to be fixed in the elasticsearch app. Please report this to https://github.com/nextcloud/fulltextsearch_elasticsearch/issues if not already done

[harryv1]

OK. I'll do that and hope that the issue gets attention.

[harryv1]

Additional finding after further analysis:
In fulltextsearch_elasticsearch 32.0.2 the composer.json requires
"elasticsearch/elasticsearch": "^9.2"
This forces the bundled PHP client to send HTTP headers with
compatible-with=9
When used against Elasticsearch 8.x (e.g. AIO 8.19.x), these requests are rejected with
400 media_type_header_exception
"Accept version must be either 8 or 7, but found 9"
This explains why fulltextsearch:check and test may work, while productive bulk indexing via fulltextsearch:index fails reproducibly with ClientResponseException and "unknown error".

[harryv1]

I reported the issue in nextcloud/fulltextsearch_elasticsearch. Unfortunately, there has been no response. To solve the problem, AIO elasticsearch could also be upgraded to version 9. Could this be a short-term option for troubleshooting? I need the search function in my AIO environment and very much regret that the incompatibility of the app cannot be resolved.

[szaimen]

This needs still to be fixed in the elasticsearch app. Tracking this in nextcloud/fulltextsearch_elasticsearch#456