FAIL_ON_UNKNOWN_PROPERTIES ... is an enum a property?

[bmschwa]

Hey @dvankley,

Continuing the conversation we were having over in a pull request here instead, because I think its different enough to warrant a new thread.

For background, I'm now running v 1.1.2 (via docker) and I'm getting this error.

Caused by: com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type net.djvk.fireflyPlaidConnector2.api.plaid.models.Products from String "statements": not one of the values accepted for Enum class: [investments, assets, auth, income, transfer, employment, income_verification, transactions, recurring_transactions, standing_orders, credit_details, identity, signal, liabilities, deposit_switch, identity_verification, balance, payment_initiation]

The root of the error is that in the openapi auto generated models used in the project, are based on an old version of the openapi spec plaid makes available. It would be nice to easily update the models based on the latest open api spec automagically (what I think PR #96 will ultimately facilitate). Ignoring differences and handling only those fields, objects, enums, etc explicitly defined would work for me just as well. I think that was the intention of commit 3165f7f you mentioned in the other thread.

The FAIL_ON_UNKNOWN_PROPERTIES parameter has the following behavior:

Used to control whether encountering of unknown properties (one for which there is no setter; and there is no fallback "any setter" method defined using @JsonAnySetter annotation) should result in a JsonMappingException (when enabled), or just quietly ignored (when disabled)

The error thrown in my case, InvalidFormatException, is derived from JsonMappingException (through MismatchedInputException, since v 2.9.

Rather than an unknown jackson 'property', I think we need to specify behavior when encountering an unknown enum... either READ_UNKNOWN_ENUM_VALUES_AS_NULL or READ_UNKNOWN_ENUM_VALUES_USING_DEFAULT_VALUE

(Cudos to this on Stack Overflow)

[dvankley]

Thanks for digging into this. Looks like the right approach.

Can you post your whole stack trace? I'm wondering what the downstream effect of this field being null would be (hopefully nothing).

[bmschwa]

No problem. Happy to have made some small contribution to this project... its really valuable for me.

Here's the stack trace for when the bug is encountered:

java.lang.reflect.UndeclaredThrowableException: Failed to invoke event listener method
HandlerMethod details:
Bean [net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2Application$$SpringCGLIB$$0]
Method [public void net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2Application.appReady()]
Resolved arguments:

at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:382) ~[spring-context-6.1.8.jar:6.1.8]
at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:237) ~[spring-context-6.1.8.jar:6.1.8]
at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:168) ~[spring-context-6.1.8.jar:6.1.8]
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:185) ~[spring-context-6.1.8.jar:6.1.8]
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:178) ~[spring-context-6.1.8.jar:6.1.8]
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:156) ~[spring-context-6.1.8.jar:6.1.8]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:451) ~[spring-context-6.1.8.jar:6.1.8]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:384) ~[spring-context-6.1.8.jar:6.1.8]
at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:109) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:80) ~[spring-boot-3.3.0.jar:3.3.0]
at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[na:na]
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:80) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:349) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1363) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1352) ~[spring-boot-3.3.0.jar:3.3.0]
at net.djvk.fireflyPlaidConnector2.FireflyPlaidConnector2ApplicationKt.main(FireflyPlaidConnector2Application.kt:31) ~[main/:na]

Caused by: io.ktor.serialization.JsonConvertException: Cannot deserialize value of type net.djvk.fireflyPlaidConnector2.api.plaid.models.Products from String "statements": not one of the values accepted for Enum class: [investments, assets, auth, income, transfer, employment, income_verification, transactions, recurring_transactions, standing_orders, credit_details, identity, signal, liabilities, deposit_switch, identity_verification, balance, payment_initiation]
at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 41, column: 7] (through reference chain: java.util.ArrayList[5])
at _COROUTINE.BOUNDARY.(CoroutineDebugging.kt:42) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:na]
at io.ktor.client.plugins.HttpCallValidator$Companion$install$2.invokeSuspend(HttpCallValidator.kt:142) ~[ktor-client-core-jvm-2.3.11.jar:2.3.11]
at io.ktor.client.call.HttpClientCall.bodyNullable(HttpClientCall.kt:89) ~[ktor-client-core-jvm-2.3.11.jar:2.3.11]
at io.ktor.client.call.HttpClientCall.body(HttpClientCall.kt:115) ~[ktor-client-core-jvm-2.3.11.jar:2.3.11]
at net.djvk.fireflyPlaidConnector2.api.plaid.infrastructure.TypedBodyProvider.body(HttpResponse.kt:35) ~[main/:na]
at net.djvk.fireflyPlaidConnector2.sync.BatchSyncRunner$run$1.invokeSuspend(BatchSyncRunner.kt:98) ~[main/:na]
Caused by: io.ktor.serialization.JsonConvertException: Cannot deserialize value of type net.djvk.fireflyPlaidConnector2.api.plaid.models.Products from String "statements": not one of the values accepted for Enum class: [investments, assets, auth, income, transfer, employment, income_verification, transactions, recurring_transactions, standing_orders, credit_details, identity, signal, liabilities, deposit_switch, identity_verification, balance, payment_initiation]
at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 41, column: 7] (through reference chain: java.util.ArrayList[5])
... 6 common frames omitted
Caused by: io.ktor.serialization.JsonConvertException: Cannot deserialize value of type net.djvk.fireflyPlaidConnector2.api.plaid.models.Products from String "statements": not one of the values accepted for Enum class: [investments, assets, auth, income, transfer, employment, income_verification, transactions, recurring_transactions, standing_orders, credit_details, identity, signal, liabilities, deposit_switch, identity_verification, balance, payment_initiation]
at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 41, column: 7] (through reference chain: java.util.ArrayList[5])
at io.ktor.serialization.jackson.JacksonConverter.deserialize(JacksonConverter.kt:67) ~[ktor-serialization-jackson-jvm-2.1.1.jar:2.1.1]
at io.ktor.serialization.jackson.JacksonConverter$deserialize$1.invokeSuspend(JacksonConverter.kt) ~[ktor-serialization-jackson-jvm-2.1.1.jar:2.1.1]
at _COROUTINE.BOUNDARY.(CoroutineDebugging.kt:42) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:na]
at io.ktor.client.HttpClient$4.invokeSuspend(HttpClient.kt:177) ~[ktor-client-core-jvm-2.3.11.jar:2.3.11]
... 5 common frames omitted
Caused by: com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type net.djvk.fireflyPlaidConnector2.api.plaid.models.Products from String "statements": not one of the values accepted for Enum class: [investments, assets, auth, income, transfer, employment, income_verification, transactions, recurring_transactions, standing_orders, credit_details, identity, signal, liabilities, deposit_switch, identity_verification, balance, payment_initiation]
at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 41, column: 7] (through reference chain: java.util.ArrayList[5])
at com.fasterxml.jackson.databind.exc.InvalidFormatException.from(InvalidFormatException.java:67) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.DeserializationContext.weirdStringException(DeserializationContext.java:1958) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.DeserializationContext.handleWeirdStringValue(DeserializationContext.java:1245) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.EnumDeserializer._deserializeAltString(EnumDeserializer.java:447) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.EnumDeserializer._fromString(EnumDeserializer.java:304) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.EnumDeserializer.deserialize(EnumDeserializer.java:273) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:361) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:246) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.MapDeserializer._deserializeUsingCreator(MapDeserializer.java:675) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:432) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:32) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.MapDeserializer._deserializeUsingCreator(MapDeserializer.java:675) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:432) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:32) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4905) ~[jackson-databind-2.17.1.jar:2.17.1]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3877) ~[jackson-databind-2.17.1.jar:2.17.1]
at io.ktor.serialization.jackson.JacksonConverter$deserialize$2.invokeSuspend(JacksonConverter.kt:64) ~[ktor-serialization-jackson-jvm-2.1.1.jar:2.1.1]
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.9.22.jar:1.9.22-release-704]
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:na]
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:111) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:na]
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:99) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:811) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:715) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:702) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:na]
Caused by: io.ktor.serialization.JsonConvertException: Cannot deserialize value of type net.djvk.fireflyPlaidConnector2.api.plaid.models.Products from String "statements": not one of the values accepted for Enum class: [investments, assets, auth, income, transfer, employment, income_verification, transactions, recurring_transactions, standing_orders, credit_details, identity, signal, liabilities, deposit_switch, identity_verification, balance, payment_initiation]

[dvankley]

Ah I see, Products is used through:

firefly-plaid-connector-2/src/main/kotlin/net/djvk/fireflyPlaidConnector2/api/plaid/models/TransactionsGetResponse.kt

Line 50 in 7cdfcab

val item: Item,

firefly-plaid-connector-2/src/main/kotlin/net/djvk/fireflyPlaidConnector2/api/plaid/models/Item.kt

Lines 73 to 79 in 7cdfcab

	/* A list of authorized products for the Item. */
	@field:JsonProperty("products")
	val products: kotlin.collections.List<Products>? = null,
	/* Beta: A list of products that have gone through consent collection for the Item. Only present for those enabled in the beta. */
	@field:JsonProperty("consented_products")
	val consentedProducts: kotlin.collections.List<Products>? = null

and the connector doesn't use anything in that response aside from

firefly-plaid-connector-2/src/main/kotlin/net/djvk/fireflyPlaidConnector2/api/plaid/models/TransactionsGetResponse.kt

Line 43 in 7cdfcab

val transactions: kotlin.collections.List<Transaction>,

, so I think this should be safe.

[dvankley]

Fixed in #105