emit default values for non populated fields in protobuf #3022
Conversation
|
|
|
Thanks for this! I will review after Thanksgiving. One quick note is it would be good if the configuration option was either nested in a protobuf_options object or if the field was prefixed with protobuf to make it more clear what it was for (also for alignment with avro). I personally prefer the former option |
There was a problem hiding this comment.
Looks great thank you for adding tests! I have one request for the configuration to namespace the option under a protobuf field so that it's clear this only works for protobuf and we have a place for more protobuf related configs as we inevitably add more options here 😄
| Field(service.NewURLField("url").Description("The base URL of the schema registry service.")) | ||
| Field(service.NewURLField("url"). | ||
| Description("The base URL of the schema registry service.")). | ||
| Field(service.NewBoolField("emit_default_values"). |
There was a problem hiding this comment.
Can we instead create a nested config object and add a boolean field for emit_defaults? I'd like to namespace these as over time I expect to have more protobuf knobs available.
That will look something like this:
service.NewObjectField("protobuf", service.NewBoolField("emit_defaults"))
When you use schema_registry_decode processor, the protobuf default values (0 for int32, "" for string, false for bool, 0.0 for float, etc...) are not populated in the row.
With this new parameter you can have all those default values populated, no need to check for nulls after that processor