wit/bindgen,cm: marshal & unmarshal for record types #225

lxfontes · 2024-10-31T20:45:07Z

Adds json struct tags to Records, based on wit names.

Also adds MarshalJSON & UnmarshalJSON to:

cm.List
cm.Tuple ( all of them )
cm.Option
cm.Variant

Example, encoding a complex record type:

  record response {
      headers: list<tuple<string, list<string>>>,
      http-code: u16,
      body: response-body
  }
  variant response-body {
    ok(list<list-element>),
    err(function-error),
    platform-err(platform-error)
  }
  record list-element {
      optional-int: option<u64>,
      optional-bool: option<bool>,
  }
  record function-error {
      error: string
  }
  record platform-error {
    code: string,
    message: string
  }

and filling it up:

	hdrVals := cm.ToList([]string{"value1", "value2"})
	hdrTuple := cm.Tuple[string, cm.List[string]]{
		F0: "header-name",
		F1: hdrVals,
	}
	headers := cm.ToList([]cm.Tuple[string, cm.List[string]]{hdrTuple})
	v := somefunctioninterface.Response{
		Headers:  headers,
		HTTPCode: 200,
		Body:     somefunctioninterface.ResponseBodyErr(somefunctioninterface.FunctionError{Error: "failed"}),
	}

Without this Pull Request:

{"Headers":{},"Body":{"RequiredParam":"required","OptionalParam":{}}}

With this Pull Request:

{"headers":[["header-name",["value1","value2"]]],"http-code":200,"body":{"err":{"error":"failed"}}}

Tuple handling

Tuples are encoded as json arrays with explicit nulls.

Tuple[string,int] -> [ "some string", 42 ]
Tuple3[string, *string, int] -> [ "some string", null, 42 ]

Variant handling

Variants are encoded as json dictionaries, so they can carry the variant data.

 record somerecord {
   myvariant: somevar,
 }
  variant somevar {
    ok,
    err(string),
  }

// JSON
{ "myvariant": { "ok": true }}
{ "myvariant": { "error": "error value" }}

Option handling

Options rely on explicit null for the "None" case.

{ "myoptional": null } // cm.None
{ "myoptional": "this" } // cm.Option[string]

If this direction seems reasonable I will add:

Marshal/Unmarshal to cm.Result
Encoding/Decoding Tests

Signed-off-by: Lucas Fontes <[email protected]>

ydnar · 2024-11-01T05:10:34Z

Amazing. This looks great so far. Thanks!

ydnar · 2024-11-01T05:24:21Z

wit/bindgen/generator.go

@@ -679,13 +679,13 @@ func (g *generator) recordRep(file *gen.File, dir wit.Direction, r *wit.Record,
 	exported := len(goName) == 0 || token.IsExported(goName)
 	var b strings.Builder
 	b.WriteString("struct {\n")
-	stringio.Write(&b, "_ ", file.Import(g.opts.cmPackage), ".HostLayout")
+	stringio.Write(&b, "_ ", file.Import(g.opts.cmPackage), ".HostLayout", "`json:\"-\"`")


Don't forget the space before the backtick, or just have a single string with HostLayout and the tag.

ydnar · 2024-11-01T05:33:18Z

wit/bindgen/generator.go

@@ -881,6 +899,39 @@ func (g *generator) variantRep(file *gen.File, dir wit.Direction, t *wit.TypeDef
 	stringio.Write(&b, "return ", stringsName, "[v.Tag()]\n")
 	b.WriteString("}\n\n")

+	file.Import("encoding/json")


Make sure to capture the name of the import. It might not be json.

ydnar · 2024-11-01T05:40:41Z

wit/bindgen/generator.go

+	file.Import("encoding/json")
+	b.WriteString(formatDocComments("MarshalJSON implements [json.Marshaler].", true))
+	stringio.Write(&b, "func (v ", goName, ") MarshalJSON() ([]byte, error) {\n")
+	stringio.Write(&b, "ret := make(map[string]any)\n")


I think there might be a better way to do this.

Possibly by putting MarshalJSON and UnmarshalJSON in package cm, and generate only the minimal mapping from case name to case type.

ydnar · 2024-11-02T14:15:03Z

cm/list.go

@@ -9,6 +12,20 @@ type List[T any] struct {
 	list[T]
 }

+func (l List[T]) MarshalJSON() ([]byte, error) {


These methods should be on the inner list[T] struct.

ydnar · 2024-11-02T14:15:38Z

cm/option.go

@@ -8,6 +12,27 @@ type Option[T any] struct {
 	option[T]
 }

+// MarshalJSON implements the json.Marshaler interface for [Option].
+func (o Option[T]) MarshalJSON() ([]byte, error) {


These methods should be on the inner struct.

ydnar · 2024-11-06T19:09:01Z

cm/option.go

+
+// UnmarshalJSON unmarshals the Option from JSON.
+func (o *Option[T]) UnmarshalJSON(buf []byte) error {
+	if len(buf) == 0 {


This should also check if buf == null.

👍 the null story here is interesting. will push it up as part of the breaking this PR into many

lxfontes · 2024-11-07T20:46:43Z

going to close this PR and move over to #239

wit/bindgen,cm: marshal & unmarshal for record types

baf3764

Signed-off-by: Lucas Fontes <[email protected]>

ydnar reviewed Nov 1, 2024

View reviewed changes

ydnar reviewed Nov 2, 2024

View reviewed changes

ydnar reviewed Nov 6, 2024

View reviewed changes

lxfontes mentioned this pull request Nov 7, 2024

wit/bindgen: support JSON encoding of WIT types #239

Open

11 tasks

lxfontes closed this Nov 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wit/bindgen,cm: marshal & unmarshal for record types #225

wit/bindgen,cm: marshal & unmarshal for record types #225

lxfontes commented Oct 31, 2024

ydnar commented Nov 1, 2024

ydnar Nov 1, 2024

ydnar Nov 1, 2024

ydnar Nov 1, 2024

ydnar Nov 2, 2024

ydnar Nov 2, 2024

ydnar Nov 6, 2024

lxfontes Nov 7, 2024

lxfontes commented Nov 7, 2024

wit/bindgen,cm: marshal & unmarshal for record types #225

wit/bindgen,cm: marshal & unmarshal for record types #225

Conversation

lxfontes commented Oct 31, 2024

Tuple handling

Variant handling

Option handling

ydnar commented Nov 1, 2024

ydnar Nov 1, 2024

Choose a reason for hiding this comment

ydnar Nov 1, 2024

Choose a reason for hiding this comment

ydnar Nov 1, 2024

Choose a reason for hiding this comment

ydnar Nov 2, 2024

Choose a reason for hiding this comment

ydnar Nov 2, 2024

Choose a reason for hiding this comment

ydnar Nov 6, 2024

Choose a reason for hiding this comment

lxfontes Nov 7, 2024

Choose a reason for hiding this comment

lxfontes commented Nov 7, 2024