Java Record @JsonAnySetter value is null after deserialization

[oujesky]

Describe the bug
When deserializing a Java Record with @JsonAnySetter annotated field the field is left as null and the unmapped values are ignored. Given the nature of Java records, there is no other way to get the unmapped fields values (like in case of annotating a setter method).

Version information
2.13.2.2

To Reproduce

import com.fasterxml.jackson.annotation.JsonAnySetter;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.junit.jupiter.api.Test;

import java.util.Map;

import static org.junit.jupiter.api.Assertions.assertEquals;

public class JsonAnySetterRecordTest {

    record TestRecord(
        @JsonProperty String field,
        @JsonAnySetter Map<String, Object> anySetter
    ) {}

    @Test
    void testJsonAnySetterOnRecord() throws JsonProcessingException {
        var json = """
            {
                "field": "value",
                "unmapped1": "value1",
                "unmapped2": "value2"
            }
            """;

        var objectMapper = new ObjectMapper();

        var deserialized = objectMapper.readValue(json, TestRecord.class);

        assertEquals("value", deserialized.field());
        assertEquals(Map.of("unmapped1", "value1", "unmapped2", "value2"), deserialized.anySetter());
    }

}

Running this test the result will be:

Expected :{unmapped1=value1, unmapped2=value2}
Actual   :null

Expected behavior
The @JsonAnySetter annotated field should contain a Map instance with all unmapped fields and their values.

Additional context
The problem happens in com.fasterxml.jackson.databind.deser.SettableAnyProperty class in method set(...). The value of the @JsonAnySetter annotated field is null and therefore setting the property value is skipped.

The suggested solution would be for Java Record to provide a new empty Map instance for the annotated field to gather the unmapped properties and this would then be provided to Record's constructor when the deserialization is concluded. Given the immutable nature of Java Records, this should ideally be some kind of immutable Map implementation (i.e. Map.copyOf(...) ?)

There might be a workaround (which however won't be probably feasible in all cases) by supplying an additional @JsonCreator constructor or factory method, where the @JsonAnySetter field is initialized to an empty map and then to ensure the immutability, the getter for this map needs to be overridden and the value returned wrapped as immutable.

record TestRecord(
        String field,
        @JsonAnySetter Map<String, Object> anySetter
    ) {

        @JsonCreator
        TestRecord(@JsonProperty("field") String field) {
            this(field, new HashMap<>());
        }

        public Map<String, Object> anySetter() {
            return Collections.unmodifiableMap(anySetter);
        }
    }

[cowtowncoder]

Currently @JsonAnySetter is not supported for Creator arguments (#562), unfortunately, which is probably why this does not work.

[fprochazka]

Thans for the hint @oujesky, I've managed to make it work:

AnySetterOnCreatorJacksonModule

@AutoService(Module.class)
public class AnySetterOnCreatorJacksonModule extends SimpleModule
{

    private static final long serialVersionUID = 1L;

    public AnySetterOnCreatorJacksonModule()
    {
        super(AnySetterOnCreatorJacksonModule.class.getSimpleName());

        setDeserializerModifier(new BeanDeserializerModifier()
        {
            @Override
            public BeanDeserializerBuilder updateBuilder(final DeserializationConfig config, final BeanDescription beanDesc, final BeanDeserializerBuilder builder)
            {
                if (!beanDesc.getBeanClass().isRecord()) {
                    return builder;
                }

                @Nullable AnnotatedMember anySetterAccessor = beanDesc.findAnySetterAccessor();
                if (anySetterAccessor == null) {
                    return builder;
                }

                JavaType parameterType = anySetterAccessor.getType();
                if (!parameterType.isMapLikeType() && !Map.class.isAssignableFrom(parameterType.getRawClass())) {
                    return builder;
                }

                builder.setValueInstantiator(new FixedAnySetterCreatorValueInstantiator(builder.getValueInstantiator(), anySetterAccessor));

                return builder;
            }

        });
    }

    private static final class FixedAnySetterCreatorValueInstantiator extends ValueInstantiator.Delegating
    {

        private final AnnotatedMember anySetterAccessor;

        FixedAnySetterCreatorValueInstantiator(final ValueInstantiator delegate, final AnnotatedMember anySetterAccessor)
        {
            super(delegate);
            this.anySetterAccessor = anySetterAccessor;
        }

        @Override
        public Object createFromObjectWith(final DeserializationContext context, final SettableBeanProperty[] props, final PropertyValueBuffer buffer) throws IOException
        {
            if (buffer.isComplete()) {
                return super.createFromObjectWith(context, props, buffer);
            }

            SettableBeanProperty anySetterProperty = getAnySetterCreatorProperty(props);

            buffer.assignParameter(anySetterProperty, new LinkedHashMap<>());

            return super.createFromObjectWith(context, props, buffer);
        }

        private SettableBeanProperty getAnySetterCreatorProperty(final SettableBeanProperty[] props)
        {
            for (SettableBeanProperty prop : props) {
                if (Objects.equals(anySetterAccessor.getName(), prop.getName())) {
                    return prop;
                }
            }

            throw new IllegalStateException(String.format("Cannot find %s in %s", anySetterAccessor, List.of(props)));
        }

    }

}

Now I can write just

record Data(
    @JsonProperty("Status") String status,
    @JsonAnyGetter @JsonAnySetter Map<String, Object> otherFields
)
{

}

and it does what I wanted 🚀 I have yet to fix the immutability problem, but that shouldn't be much hard to hack :)

[cowtowncoder]

I think this is basically duplicate of #562.

[mwisnicki]

Thans for the hint @oujesky, I've managed to make it work:

AnySetterOnCreatorJacksonModule

and it does what I wanted 🚀 I have yet to fix the immutability problem, but that shouldn't be much hard to hack :)

Unfortunately it returns entries in reverse order :(

[kleino]

So far, I've used the workaround with @JsonCreator provided by oujesky. Unfortunately starting with Jackon 2.15.1 this doesn´t work anymore and the @JsonAnySetter annotated field is null again :(

[yihtserns]

@kleino JsonAnySetter documents two ways:

1. Using field (of type Map or POJO)
2. Using non-static two-argument method (first argument name of property, second value to set)

You've been using #​1, which is no longer usable because #3737 ignores Record fields for deserialization (so Jackson cannot "see" the @JsonAnySetter on the now-gone field).

You can still use #​2:

record TestRecord(String field, Map<String, Object> anySetter) {

    @JsonCreator
    TestRecord(@JsonProperty("field") String field) {
        this(field, new HashMap<>());
    }

    public Map<String, Object> anySetter() {
        return Collections.unmodifiableMap(anySetter);
    }

    @JsonAnySetter 
    private void updateProperty(String name, Object value) {
        anySetter.put(name, value);
    }
}

[cowtowncoder]

Hmmh. Ideally we'd support @JsonAnySetter the way intended. But thank you @yihtserns for showing a work-around until then.

[lordvlad]

Is there any intent to tackle this issue? It's blocking us from using records in many places, unfortunately.

[cowtowncoder]

@lordvlad If someone was working on this, they'd likely add a note. So I doubt anyone is working on it. I do not have bandwidth myself, but perhaps @JooHyukKim might have?

[JooHyukKim]

FYI, 2.15 or later throws UnrecognizedPropertyException instead of returning null (see #4346 for reproduction).

EDIT: Crossed out in favor of #562

[cowtowncoder]

Note: #562 is the main issue wrt inability to use @JsonAnySetter on Creator properties.

[tayloj]

Just came across this issue, because I ran into it as well, but I wanted to share my workaround, which is similar to the one provided but uses a compact constructor to avoid the need for a "full constuctor" annotated with JsonCreator that replicates the entire arglist. This doesn't remove the fact that we're adding a mutable map to the record instance, which might be undesirable in some cases.

  record Person(String name, Map<String, Object> attributes) {

    /**
     * Creates a new instance. If {@code attributes} is null, a new <em>mutable</em> map is created
     * and used instead.
     *
     * @param name the name
     * @param attributes the attributes map, or null
     */
    Person {
      attributes = attributes == null
          ? new HashMap<>()
          : attributes;
    }

    @JsonAnySetter
    public void addAttribute(final String key, final Object value) {
      attributes.put(key, value);
    }
  }

[yihtserns]

@tayloj

• Bad news: your workaround will no longer work in the upcoming 2.18.0.
• Good news: workarounds are no longer necessary as Allow @JsonAnySetter to flow through Creators #562 has been implemented.

@cowtowncoder I think this can be closed?

[cowtowncoder]

Yes, will mark as fixed as of 2.18.0; close.

[cowtowncoder]

Added a new test (similar to one for #562 but still); passes, closing as implemented