NodeTypes in Microsoft.PowerFx.ParseResult

[cchannon]

I want to explore how I can add Lookup Function support to my custom app running the C# interpreter. In order to do this, there is really no getting around the need to dynamically declare variables (or, more specifically, a List of Records). In general, I figure the only way to do this without losing my mind is something similar to:

1. Parse the command before running it
2. Check parse obj.Root.Head.Name.Value to see if it is Lookup
   -->2a. If so, get tablename from engine.Eval(Root.Args.Children[0])
   --> 2b. get QE by converting from Root.Args.Children[1] (doing this conversion will be rough, but not impossible)
   --> 2c. Create List of Records from result
3. If not, get Args from Root.Args.Children and loop through each, returning to 1

But that is, of course, an oversimplification because the different NodeTypes all have different properties and I wouldn't actually need to loop over every Arg if I knew for sure which args definitely could not contain a Lookup (Microsoft.PowerFx.Syntax.NodeKind.BoolLit looks like something I might be able to ignore, for example)

So, my question is this: is there a test case somewhere that illustrates all the different NodeKind values that I could use to make sure I'm not missing anything? I am sure I could write up something for most of them, but I'd rather not guess at it if there's already something out there.

[lesaltzm]

I'd suggest not relying on NodeKind, but rather, using the visitor pattern, which should ensure you exhaustively cover all the AST nodes.

Either

Power-Fx/src/libraries/Microsoft.PowerFx.Core/Syntax/Visitors/TexlVisitor.cs

Line 9 in dd20b35

public abstract class TexlVisitor

or

Power-Fx/src/libraries/Microsoft.PowerFx.Core/Syntax/Visitors/TexlFunctionalVisitor.cs

Line 11 in dd20b35

public abstract class TexlFunctionalVisitor<TResult, TContext>

should work depending on your use case. I'm not entirely sure what you mean by "get QE" above, but your approach seems possible.

On that note, we don't currently provide a facility for registering custom/reflection functions that define and make use of scope, and that may be something we should look into.

[cchannon]

Ok, super interesting! Thanks, I'll dig in!

[MikeStall]

@cchannon - I'm not sure what you're asking.
The LookUp() function is implemented in the interpreter:

Impl here:

Power-Fx/src/libraries/Microsoft.PowerFx.Interpreter/Functions/LibraryTable.cs

Line 16 in a45e82a

public static async ValueTask<FormulaValue> LookUp(EvalVisitor runner, EvalVisitorContext context, IRContext irContext, FormulaValue[] args)

Tests here:
https://github.com/microsoft/Power-Fx/blob/main/src/tests/Microsoft.PowerFx.Core.Tests/ExpressionTestCases/Lookup.txt

[cchannon]

Yes, absolutely. What I meant to say was, "Lookup support across entire tables of my database"...

Let me explain... The reason I am so enthusiastic about adopting pfx for my users is that they are already accustomed to it through canvas apps as a window to Dataverse. So they are already accustomed to using Lookup to search entire (often very large) db tables. And this is a pattern they will want to keep.

I am not so crazy as to think I should stringify and put an entire DB table into memory just so I can string match on it (pretty crazy, but not quite that crazy 😁), so instead I figure the only logical recourse is to intercept any Lookup functions that reference a "data source" and write my own DB query for them. This is the source of my question.

[MikeStall]

Ok, it sounds the real issue is "Delegation" (https://learn.microsoft.com/en-us/power-apps/maker/canvas-apps/delegation-overview) where the expression runs server side.

This is another thing we're working on but just isn't quite available yet ...

There's an early preview in the Connector library which compiles to an OData query.
https://github.com/microsoft/Power-Fx/blob/main/src/libraries/Microsoft.PowerFx.Connectors/ODataVisitor.cs

This uses a visitor as McCall said.

The gist here is we're keying it off a derived TableValue. So rather than intercept Lookup, Lookup uses the derived table value.
There's a start of this even with InMemory.
https://github.com/microsoft/Power-Fx/blob/main/src/libraries/Microsoft.PowerFx.Core/Public/Values/CollectionTableValue.cs

Ie, Doing Index(table, 10000) on just calls a virtual

Power-Fx/src/libraries/Microsoft.PowerFx.Core/Public/Values/CollectionTableValue.cs

Line 98 in bd458dc

protected override bool TryGetIndex(int index1, out DValue<RecordValue> record)

which a derived Table can override.

But all that said .. you can see the direction we're heading in, but we're probably not quite where you need it for SQL.