[Python] Illegal implicit conversions from C++ functors to function pointers accidentally work from Python

[bgrube]

Check duplicate issues.

• Checked for duplicates

Description

Creating a TF1 from a C++ functor with the number of parameters set to zero and drawing this function into a TCanvas, causes a Python TypeError when attempting to save the canvas as a PDF file.

Reproducer

Running the script tf1bug.py

#!/usr/bin/env python3

import ROOT

# function without parameters
def f1(args, pars):
  return 4.0 * args[0]

# functor without parameters
F2_CPP = """
class Functor {
public:

	Functor()
	{ }

	double
	operator () (
		double* args,
		double*
	) {
		return 4.0 * args[0];
	}
};
"""

if __name__ == "__main__":
  ROOT.gROOT.SetBatch(True)
  fcn1 = ROOT.TF1("f1", f1, 0, 1, 0)  # using a Python function works fine
  ROOT.gInterpreter.Declare(F2_CPP)
  functor = ROOT.Functor()
  fcn2 = ROOT.TF1("f2", functor, 0, 1, 1)  # setting the number of parameters to 1 instead of 0 makes the bug go away
  fcn3 = ROOT.TF1("f3", functor, 0, 1, 0)  # <- setting the number of parameters to 0 causes the bug
  for fcn in (fcn1, fcn2, fcn3):
    canv = ROOT.TCanvas()
    fcn.Draw()
    canv.SaveAs(f"{fcn.GetName()}.pdf")

in ROOT 6.38.00 yields

Info in <TCanvas::Print>: pdf file f1.pdf has been created
Info in <TCanvas::Print>: pdf file f2.pdf has been created
Traceback (most recent call last):
  File "./tf1bug.py", line 35, in <module>
    canv.SaveAs(f"{fcn.GetName()}.pdf")
TypeError: could not convert argument to buffer or nullptr

With ROOT 6.32.20, I get

Info in <TCanvas::Print>: pdf file f1.pdf has been created
Info in <TCanvas::Print>: pdf file f2.pdf has been created
Traceback (most recent call last):
  File "/u/home/bgrube/./tf1bug.py", line 35, in <module>
    canv.SaveAs(f"{fcn.GetName()}.pdf")
TypeError: void TPad::SaveAs(const char* filename = "", Option_t* option = "") =>
    TypeError: double Functor::operator()(double* args, double*) =>
    TypeError: could not convert argument 2 (could not convert argument to buffer or nullptr)

ROOT version

The bug appears at least in versions 6.38.00, 6.36.06, and 6.32.20.

Installation method

pre-built binary

Operating system

Linux and MacOS

Additional context

No response

[guitargeek]

Thanks for the error report! The problem is that ROOT/cppyy resolves to an inappropriate constructor of the TF1, leading to further problems downstream in the code.

Here is a standalone cppyy reproducer of the underlying issue:

import cppyy

cppyy.cppdef("""
class Functor {
public:
    double operator () (double* args, double*) {
        return 4.0 * args[0];
    }
};

void foo(double (*fcn)(double*, double*)) {
    std::cout << "foo(double (*fcn)(double*, double*))" << std::endl;
}

template<class T>
void foo(T fcn) {
    std::cout << "foo(T fcn)" << std::endl;
}

void bar(double (*fcn)(double*, double*)) {
    std::cout << "bar(double (*fcn)(double*, double*))" << std::endl;
}
""")

# Both wrongly resolve to the function pointer overload
functor = cppyy.gbl.Functor()
cppyy.gbl.foo(functor) # should resolve to foo(T fcn)
cppyy.gbl.bar(functor) # not allowed, because there is no bar(T fcn overload)!

# See what should happen in C++
cppyy.cppexec("Functor functor{};")
cppyy.cppexec("foo(functor);") # foo(T fcn)
cppyy.cppexec("bar(functor);") # error

The output is:

foo(double (*fcn)(double*, double*))
bar(double (*fcn)(double*, double*))
foo(T fcn)
input_line_48:2:2: error: no matching function for call to 'bar'
 bar(functor);
 ^~~
input_line_38:18:6: note: candidate function not viable: no known conversion from 'Functor' to 'double (*)(double *, double *)' for 1st argument
void bar(double (*fcn)(double*, double*)) {
     ^   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

So not only does cppyy fail to resolve the right overloads (which would be the templated ones), but it resolves overloads that should not be called because they are impossible in C++. A functor can't be converted to a function pointer. The same thing happens in your bug report with the TF1 constructor.

The problem also exists in the standalone cppyy by @wlav, and the compiler-research fork by @aaronj0 and @Vipul-Cariappa.

I think the cleanest solution is to make sure that the implicit conversion from C++ functor to function pointer is not possible in cppyy, respecting the C++ language and avoiding that the wrong overload is selected.

[guitargeek]

I have renamed the issue in an attempt to better reflect what is actually the underlying problem here.

[guitargeek]

Okay even with my PR, your code example still picks the wrong constructor, but this time a different one! More fixes are required.

The TF1 constructor seems to be the final boss of cppyy overload resolution, with 12 overloads in total, involving various function object types, and some of them being templated... Not trivial

[Vipul-Cariappa]

I am working on a gigantic patch that changes the way cppyy does overload resolution (The new cppyy based on CppInterOp). And looks like it is picking the correct overload. This is from @guitargeek's standalone reproducer

❯ python ./tmp/bug10.py 2> /dev/null
foo(T fcn)
bar(double (*fcn)(double*, double*))
foo(T fcn)
"cppyy.cppexec("bar(functor);")" raised SyntaxError

But it is still picking up the bar, so it is only partially fixed.

[guitargeek]

Nice! Well the problem that it picks up bar would be fixed by #20696, which is orthogonal to your overload resolution developments I guess? My PR only touches the function pointer converter.