Parameter Typing

[till-m]

fixes #93 #191 #308 #376 by adding support for different parameter types: float (the default), int, and categorical. This should cover the majority of the usecases.

(there is still some stuff left to do before this can be merged but I would like to know whether merging this is something we want to consider in the first place before doing that work).

Adds experimental support for parameter typing.

This adds a pretty serious layer of complexity (one could even call it spaghettification in some cases) to the code base.

Checklist:

• Make sure the original functionality works as before
• Add basic support for parameters
• Add more tests
• Check parameters and constraints
• Check parameters and Domain Reduction
• ...probably more?
• Figure out how to retain the serializability

I'm primarily looking for feedback here: Is this worth adding? Is this worth the additional technical debt? Is there a better way to handle the problem of parameter types?

[till-m]

Demo: notebook.

[fmfn]

Upon a quick first glance this looks pretty damn interesting. Did you follow any other implementation, a paper or something? Or is this your approach? Either way, looks exciting, I'll try to play around with it this coming weekend.

[till-m]

Based on this paper.

[bwheelz36]

the implementation in the notebook appears really elegant. I think a lot of people would be happy with this feature. It does look like a pretty large amount of work though! But I would say yes this would be a desired feature.

[PassengerC07]

Hi,

I think for hyperparameter tuning, this feature is a must-have, especially if bound transformer is used. This is really nice work! This is going to improve the algorithm's efficiency for tuning the parameters which are in a lot of cases not float.

[till-m]

I will close this for now, since I think the added technical debt is not really worth it. If I can think of a better way of handling the parameter problem, I will make a new PR.