[FLINK-37175][table] Support JSON built-in function for JSON_OBJECT

[gustavodemorais]

What is the purpose of the change

It is currently not possible to declare a SQL string that contains existing JSON as valid JSON for JSON_OBJECT. Something like JSON_OBJECT(KEY 'K' VALUE '{"value": 42}') returns {"K", "{"value": 42}"}, where the value is a string a not a json object.

This PR adds support for the JSON() function. It's the initial support for it, until this function returns the JSON datatype (what we still don't have in flink).

Example:

jsonObject(JsonOnNull.NULL, "nested", json('{"value": 42}'))
JSON_OBJECT(KEY 'K' VALUE JSON('{"value": 42}'))
// {"nested":{"value":42}}

Brief change log

• Add JSON BuiltInFunction definition
• Update JsonObjectCallGen to support JSON function logic
• Update expressions
• Add tests for json function

Verifying this change

This change added tests for multiple uses cases of the function. Also added tests to make sure the fucnction is only called within JSON_OBJECT.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): yes
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? yes
• If yes, how is the feature documented? docs, JavaDocs

[flinkbot]

CI report:

• Unknown: CANCELED
• 1dea414 UNKNOWN

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[twalthr]

Suggested change

	* jsonObject(JsonOnNull.NULL, "K1", json('{"K2":"V"}'))
	* jsonObject(JsonOnNull.NULL, "K1", json("{'K2':'V'}"))

[twalthr]

or escaped? not sure how strict the user-defined JSON needs to be? maybe we should check and document this.

[gustavodemorais]

The string doesn't have to be escaped, I've updated the javadoc for the java expression and added one example using string literals that show that there's no need to escape it

     * // {"K":{"K2":{"K3":42}}}
     * jsonObject(
     *         JsonOnNull.NULL,
     *         "K",
     *         json("""
     *                {
     *                  "K2": {
     *                    "K3": 42
     *                  }
     *                }
     *              """))

We escape in the java code since java also uses " to express strings. If using single quotes in python or string literals in java, there's no need to escape. If they're escaped, they are also processed properly as well and result in the same json object.

[gustavodemorais]

Made the changes as we discussed @twalthr. One additional thing I've changed is that we not only parse the json, but convert the json back to string before storing it. I think that makes sense, so we optimize the storage space by getting rid of unnecessary whitespaces/line breaks and so on before returning the value.

[gustavodemorais]

@flinkbot run azure