MongoDB Source: Expose Change Stream fullDocument option (e.g. UPDATE_LOOKUP) in source-mongodb-v2

[virajs]

The MongoDB v2 source connector uses MongoDB Change Streams for CDC, but currently does not expose any configuration option to control the fullDocument behavior (e.g. DEFAULT vs UPDATE_LOOKUP).

By default, update events only include _id and updateDescription, and fullDocument is null. For many CDC use cases (data lakes, audit logs, downstream transformations, idempotent upserts), consumers require the full post-update document.

MongoDB Change Streams natively support this via:
fullDocument: "updateLookup"

but unable to find a way to enable this via:

• Source configuration
• Connection string
• UI / JSON config
• Environment variables

Any suggestions or guidance to help get this working would be greatly appreciated.

[ismaildasci]

For MongoDB Change Streams with Airbyte, the UPDATE_LOOKUP option isnt directly exposed in the connector config yet. But you have a few workarounds:

1. Fork and modify the connector - In the source code, find where the change stream is created and add:

ChangeStreamIterable<Document> changeStream = collection.watch()
    .fullDocument(FullDocument.UPDATE_LOOKUP);

2. Use a custom MongoDB view or trigger - Create a change stream trigger that writes full documents to a separate collection, then sync that collection normally (not via CDC).

3. Post-process in your destination - If you have the _id from the update event, you can do a lookup join in your data lake to get the full document.

4. Request this feature - Open a GitHub issue on airbytehq/airbyte requesting fullDocument configuration be exposed in the UI. Its a reasonable ask since many CDC use cases need it.

The cleanest solution is option 1 if youre comfortable building the connector, or option 4 to get official support.