Runtime dispatch when reducing to scalar

[Kolaru]

When reducing to scalar it seems like a runtime dispatch on the Eval struct remains.

It is weird, since it appears in VScode @profview but @code_warntype shows that all types can be inferred.

It has however a performance effect. For the example below on my maching this results in 73ms and 1M allocation vs 55ms and no allocations when I manually remove the Eval step from the generated code.

Example

using BenchmarkTools, Tullio

struct MiniSquareMatrix{T}
    data::Matrix{T}
    rows::Vector{Int}
    cols::Vector{Int}
end

function baddot(a::Vector, Q::MiniSquareMatrix, b::Vector)
    @tullio res := a[Q.rows[i]] * Q.data[i, j] * b[Q.cols[j]]
    return res
end

Q = MiniSquareMatrix(rand(6, 6), collect(1:6), collect(1:6))
a = rand(6)
b = rand(6)

f(a, Q, b) = for i in 1:1000_000
    baddot(a, Q, b)
end

@code_warntype f(a, Q, b)  # Everything is inferred here
@profview f(a, Q, b)  # See below for result
@btime f(a, Q, b)  # 73.253 ms (1000000 allocations: 15.26 MiB)

# This uses the code generated by Tullio._tullio, with the `Eval` manually removed at the end
function gooddot(a::Vector, Q::MiniSquareMatrix, b::Vector)
    # Code not reported for brevity
end

g(a, Q, b) = for i in 1:1000_000
    gooddot(a, Q, b)
end

@btime g(a, Q, b)  # 55.074 ms (0 allocations: 0 bytes)

@profview result (Eval is flagged with runtime dispatch)