Add function inspections

[cam72cam]

With the focus on functions in recent hcl releases, I thought it time to introduce the ability to inspect which functions are required to evaluate expressions. This mirrors the Variable traversal present throughout the codebase.

This allows for better error messages and optimizations to be built around supplying custom functions by consumers of HCL.

These changes are backwards compatible and is not a breaking change. I explicitly introduced hcl.ExpressionWithFunctions to allow consumers to opt-into supporting function inspection. I would recommend that this be moved into hcl.Expression if a major version with API changes is ever considered.

[cam72cam]

Hi @apparentlymart, would you be able to take a look at this? It looks like you have done most of the reviews on function related features.

[yolo-minseongseo]

sry. I made a mistake. IDK how to dismiss my approval🤣

[cam72cam]

@jbardin @liamcervante, could I get some eyes on this?

[jbardin]

Thanks for the contribution @cam72cam! I should be able to take a look at this soon.

[cam72cam]

@jbardin any updates on having time to review this?

[apparentlymart]

Some belated commentary on this after I've got some relevant context loaded into my brain for other reasons:

• hcl.StaticCall is effectively an analog of hcl.Traversal for function calls. It's currently used only for hcl.ExprCall in situations where the entire expression is a function call, such as in the typeexpr extension's type constraint syntax, but it seems like it should also work to describe function calls nested inside larger expressions similar to the relationship between hcl.Expression.Variables and hcl.AbsTraversalForExpr.

  I think that would be a more "natural" representation than using hcl.TraverseRoot, which usually means to access one of the keys from hcl.EvalContext.Variables.

• Because hcl.StaticCall includes the expressions representing the function call arguments, an alternative design would be to introduce a new function that describes both direct variable references and function calls, and to exclude variable references discovered in function call arguments from the set of direct variable references.

  That could be a useful building block for Terraform to finally support typeexpr.ConvertFunc without upsetting the reference analysis, because Terraform's reference analyzer could have a special case to skip calling Variables on the expression in the second argument, so that primitive type keywords like string won't be misunderstood as references to managed resources of type string.

  To make things easier for callers who don't have that requirement to give special treatment to certain functions, maybe this function would return an instance of a struct type that has separate fields for the naked variable references vs. the function calls but also has a method AllVariables() iter.Seq[hcl.Traversal] that iterates over all of the naked references and all of the function call arguments, with the result then ignoring the distinction between naked vs. nested variable references.

  There's a tricky question here of what should happen when a function call argument contains another function call... that seems to suggest that the AllVariables function would need to keep recursively unpacking function call arguments until it reaches a result that doesn't include any more function calls, and Terraform's own variant of that logic that knows to treat convert as special would also need to do a similar recursive walk. But maybe that's okay as long as the AllVariables function is there to make things easier for most callers; Terraform often has unusual needs that most other HCL callers don't¹.

I dunno if maintainers are even still interested in this functionality at this point, but I share this just in case it's helpful!

Footnotes

1. Although Terraform is the main caller that has this special need to pre-detect references to dynamically build an hcl.EvalContext, the general idea of being able to treat certain function arguments as something other than normal expression evaluation is exposed for other callers in ext/customdecode, so other callers certainly could implement features like this and so potentially benefit from the ability to do reference analysis in a custom way that treats certain functions differently. ↩