Production Running

Can run the complete notebook that demos xaod_usage

• Do not require clients to install the func-adl.ast package.
• Fix bug in how QMetaData updates items in its dict (overriden items were ignored!)
• Add a warning if there is a lambda that isn't scanned for type forwarding
• Fix bug where function names were being captured during lambda resolution
• Move to a single package rather than two (no more .ast). This is certianly a breaking change!

Bug Fixes

• incorrect __init__ file - made meta data clearing for the unreleased version of func_adl_servicex fail.
• QMetaData was not dealing with updated items properly.

Remove the ast package - everything as one now

• There is no ast package now - everything comes down at once. This will cause knock-on effects in everyone's configuration, sadly.

Move metadata into client part of library

Another cross-include into the ast part of the library fixed.

Fix bug introduced

Moved the wrong file

Do not require func-adl.ast in client installs

I'd accidentally added a dependency!

Type Hint Support, MetaData, Syntactic Sugar

Lots of big features. This should be backwards compatible, but there are major upgrades. Hence the change in version number.

New Features:

• It is possible to properly type-hint so that a good editor will be able to understand typing in func_adl queries. This makes heavy use of python's Generic type as we are often working with streams of objects (like events or jets, etc.).
• Python list comprehensions and generator expressions inside lambda's are now converted to func_adl automatically. Ifs and multiple generators are supported.
• Variables referenced inside lambda expressions are now properly captured. So you can parameterize a cut some external variables. Method and attribute and function calls on captured variables are not supported.

New Extensibility Points

• Parameterized method calls are supported. You can write something like [j.getAttribute[cpp_float]('EMF') for j in jets] and with the proper support it will generate a C++ template function. The func_adl library provides none of the support, but you can use the decorator func_adl_parameterized_call
• You can now use the @func_adl_callback decorator to decorate a class or a method in your type hints file to invoke custom python code as the query is built. This allows extensions, default argument processing, etc. The C++ xAOD backend heavily uses this to implement things like calibration, etc.
• Query Metadata can be attached to an expression. It is stripped out before being sent down to ServiceX. This can be used to cache information that needs to be tracked in a query. The C++ xAOD type system uses this to track calibration configuration.
• MetaData can be sent down to the backend at arbitrary points in the query. This can be used, for example, to declare types to the C++ backend.

Technical Debt:

• Numerous clean ups to the type hinting in the library
• Test coverage improvements, which uncovered some python version incompatibilities (which were fixed).
• Using black as the formatted for the repo
• README documentation improvements
• Supporting 3.7 is getting to be work. Python 3.10 is supported.
• Added full support for the type bool.

Add Python 3.10 Support

Python 3.10 is out. This has now been tested with func_adl.

Add type information to Where method

• Object stream's Where gets proper type hinting

Typed Datasets

• ObjectStream is now a generic type that will track the type of the object it surrounds
• EventDataset is also typed, as a result, and should be declared as such by anyone sub-classing this

This should not break any existing code downstream.

Finally, after years, we are tracking here a typed monad in python!