Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error when passing empty list as argument to Lua function #4

Closed
JonnyHaystack opened this issue Aug 25, 2024 · 2 comments · Fixed by #5
Closed

Error when passing empty list as argument to Lua function #4

JonnyHaystack opened this issue Aug 25, 2024 · 2 comments · Fixed by #5
Assignees
@cam72cam
Labels
help wanted Extra attention is needed

Comments

@JonnyHaystack
Copy link

JonnyHaystack commented Aug 25, 2024 •
edited
Loading

First of all, thanks to the OpenTofu team for this cool and useful functionality!

I have had some success with using this provider, but there are some awkward failure cases as well. One of them was that it can handle an array being passed in, but not an empty array. This seems to be blocked by some validation in the Go code of the provider, rather than in the execution of my Lua code.

Also unrelated to this specific issue but of note in my snippet below: arrays (tables) seem to be 0-indexed not 1-indexed as they usually are in Lua?
I had to use #arr == 0 and return arr[0] to get the correct behaviour. Initially I tried #arr == 1 and return arr[1] but that behaved incorrectly.

OpenTofu version: v1.8.1
terraform-provider-lua version: v0.0.2

Sample code:

locals {
  normalize_array = <<-EOT
  -- If input array contains single value, return that value, otherwise return the original array.
  function normalize(arr)
    if type(arr) == "table" and #arr == 0 then
      return arr[0]
    end
    return arr
  end

  return normalize
  EOT
}

resource "vyos_config" "firewall_zones" {
  for_each = var.firewall.zones

  path = "firewall zone ${each.key}"
  value = jsonencode({
    foo1 = provider::lua::exec(local.normalize_array, ["a", "b"])
    foo2 = provider::lua::exec(local.normalize_array, ["c"])
    foo3 = provider::lua::exec(local.normalize_array, [])
  })
}

The error:

╷
│ Error: Error in function call
│ 
│   on modules/vyos/config/firewall.tf line 24, in resource "vyos_config" "firewall_zones":
│   24:     foo3 = provider::lua::exec(local.normalize_array, [])
│     ├────────────────
│     │ while calling provider::lua::exec(code, args...)
│     │ local.normalize_array is "function normalize(arr)\n  if type(arr) == \"table\" and #arr == 0 then\n    return arr[0]\n  end\n  return arr\nend\n\nreturn normalize"
│ 
│ Call to function "provider::lua::exec" failed: rpc error: code = Canceled desc = context canceled.
╵

Stack trace from the terraform-provider-lua_v0.0.2 plugin:

panic: unknown type implementation

goroutine 25 [running]:
github.com/zclconf/go-cty/cty.Type.MarshalJSON({{0x0?, 0x0?}})
        github.com/zclconf/[email protected]/cty/json.go:87 +0x647
github.com/zclconf/go-cty/cty/msgpack.(*dynamicVal).MarshalMsgpack(0xc0002d8200)
        github.com/zclconf/[email protected]/cty/msgpack/dynamic.go:18 +0x31
github.com/vmihailenco/msgpack/v5.marshalValue(0xc0002c05a0, {0xa51e60?, 0xc0002d8200?, 0x7f1475f60108?})
        github.com/vmihailenco/msgpack/[email protected]/encode_value.go:165 +0x11c
github.com/vmihailenco/msgpack/v5.(*Encoder).EncodeValue(0xc0002c05a0, {0xa51e60?, 0xc0002d8200?, 0x38?})
        github.com/vmihailenco/msgpack/[email protected]/encode.go:240 +0x7a
github.com/vmihailenco/msgpack/v5.(*Encoder).Encode(0xa323e0?, {0xa51e60?, 0xc0002d8200?})
        github.com/vmihailenco/msgpack/[email protected]/encode.go:226 +0x229
github.com/zclconf/go-cty/cty/msgpack.marshalDynamic(...)
        github.com/zclconf/[email protected]/cty/msgpack/marshal.go:208
github.com/zclconf/go-cty/cty/msgpack.marshal({{{0x0?, 0x0?}}, {0x0?, 0x0?}}, {{0xc1fd50?, 0x1103800?}}, {0x0, 0x0, 0x0}, 0xc0002c05a0)
        github.com/zclconf/[email protected]/cty/msgpack/marshal.go:53 +0x2d3
github.com/zclconf/go-cty/cty/msgpack.Marshal({{{0x0?, 0x0?}}, {0x0?, 0x0?}}, {{0xc1fd50?, 0x1103800?}})
        github.com/zclconf/[email protected]/cty/msgpack/marshal.go:37 +0x32b
main.CtyToProto({{{0x0?, 0x0?}}, {0x0?, 0x0?}})
        github.com/opentofu/terraform-provider-lua/main.go:329 +0x39
main.LuaToProto(0xc0002dc000?)
        github.com/opentofu/terraform-provider-lua/main.go:313 +0x2b
main.main.func1.2({0xc0002cf710, 0x2, 0x2})
        github.com/opentofu/terraform-provider-lua/main.go:283 +0x418
main.(*FunctionProvider).CallFunction(0xc00007d320, {0xc0002bd270?, 0xa68900?}, 0xc0002d4b70)
        github.com/opentofu/terraform-provider-lua/main.go:96 +0x17a
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).CallFunction(0xc0001a4140, {0xc1fb58?, 0xc0002d4570?}, 0xc0002bd220)
        github.com/hashicorp/[email protected]/tfprotov6/tf6server/server.go:987 +0x623
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_CallFunction_Handler({0xb001c0, 0xc0001a4140}, {0xc1fb58, 0xc0002d4570}, 0xc0002ab000, 0x0)
        github.com/hashicorp/[email protected]/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:608 +0x1a6
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000103000, {0xc1fb58, 0xc0002d44e0}, {0xc23ec0, 0xc0000d9860}, 0xc0002c9d40, 0xc0002412c0, 0x1095090, 0x0)
        google.golang.org/[email protected]/server.go:1386 +0xdf8
google.golang.org/grpc.(*Server).handleStream(0xc000103000, {0xc23ec0, 0xc0000d9860}, 0xc0002c9d40)
        google.golang.org/[email protected]/server.go:1797 +0xe87
google.golang.org/grpc.(*Server).serveStreams.func2.1()
        google.golang.org/[email protected]/server.go:1027 +0x8b
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 16
        google.golang.org/[email protected]/server.go:1038 +0x125

Error: The terraform-provider-lua_v0.0.2 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

P.S. Would love to see a Python provider in future as well one day, as I'm much more familiar with it than Lua or Go 😄
@cam72cam cam72cam self-assigned this Aug 27, 2024
@apparentlymart
Copy link

I'm not super familiar with this code but something that sticks out to me is that LuaToCty has a case which returns cty.NilVal:

terraform-provider-lua/main.go

Lines 399 to 400 in ce1d19a

case lua.TypeNil:
return cty.NilVal, nil

cty.NilVal is not actually a valid value -- it represents the absence of a value in the Go type system, rather than in cty's own type system -- and so other functions like the MessagePack encoder do not expect to receive it.

The arguments in the first entry in the stack trace seem like they might be a raw representation of cty.NilVal: {0x0?, 0x0?}. And I think that makes sense for foo3 because it's trying to take the first element of an empty table, which would return nil in Lua.

I think a more appropriate translation of Lua's nil would be cty.NullVal(cty.DynamicPseudoType), which is a null value of an unknown type and is what null returns in HCL.

Amusingly, some time ago I wrote an entirely unrelated library for converting between Lua and cty, but using GopherLua instead of Shopify's runtime: github.com/zclconf/gopherlua-cty.

Looking at that code again today, it seems to have its own variation of this bug: if you ask it to convert Lua nil then it will return an error due to not being able to figure out which type to use to represent it. I designed mine primarily for the situation where the application knows which cty type it's expecting which therefore hints on what to convert nil to, but if you use it in a similar mode as this provider works -- just converting to the closest possible cty type to the dynamically-selected Lua type -- it doesn't know what to return. So I guess I should fix that at some point too. 😀

@apparentlymart
Copy link

On the separate question about the list being zero-indexed, that does seem to be what the implementation is doing:

terraform-provider-lua/main.go

Lines 377 to 388 in ce1d19a

if t.IsListType() || t.IsSetType() || t.IsTupleType() {
l.NewTable()
for k, v := range arg.AsValueSlice() {
l.PushInteger(k)
err := CtyToLua(v, l)
if err != nil {
return err
}
l.SetTable(-3)
}
return nil
}

Iterating over the result of AsValueSlice will give k starting at zero, which it then uses as the table key. Since Lua tables are associative arrays with some special conventions to pretend they are lists, it's valid to create a table with a key 0, but unconventional in Lua.

My gopherlua-cty ends up creating a similar result in a different way: it wraps the cty values as Lua userdata with a metatable that translates various Lua operators to the corresponding cty functionality, and so Lua's index operator is overloaded to perform cty.Value.Index. I had done that intentionally so that using HCL and Lua together in the same program would use indices consistently, but I definitely dithered a bunch on that decision since it's a typical tricky tradeoff when bridging between two languages: is it better to remain true to the idioms of the inner language, or better to ensure consistency with the surrounding program written in another language?

It would be a breaking change to make this 1-indexed now, anyway.

@cam72cam cam72cam added the help wanted Extra attention is needed label Jan 7, 2025
@cam72cam cam72cam assigned cam72cam and unassigned cam72cam Jan 7, 2025
@cam72cam cam72cam mentioned this issue Jan 13, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cam72cam cam72cam

Labels
help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix null/list encoding + example opentofu/terraform-provider-lua
3 participants
@apparentlymart @cam72cam @JonnyHaystack