x_num, x_cat in apply_model

[bbb801]

Dear Sir/Madam,
In your 'rtdl.ipynb', it says,
IF you use MLP, ResNet or any other simple feed-forward model (NOT transformer-based model) and there are categorical features...Then the model should be used as model(x_num, x_cat) instead of of model(x).

Then in 'apply_model', it seems x_cat cannot be used in rtdl.MLP, rtdl.ResNet. I don't quite understand. Could any explain when cat and num are used in all of these models, how should I input the data into the model and apply_model?

def apply_model(x_num, x_cat=None):
if isinstance(model, rtdl.FTTransformer):
return model(x_num, x_cat)
elif isinstance(model, (rtdl.MLP, rtdl.ResNet)):
assert x_cat is None
return model(x_num)
else:
raise NotImplementedError(
f'Looks like you are using a custom model: {type(model)}.'
' Then you have to implement this branch first.'
)

[Yura52]

Hi! In the same long comment (which starts with === ABOUT CATEGORICAL FEATURES ===), there is the Model class that you need to uncomment (and you may have to adjust it for your needs). Then you have to use it to create the model variable (instead of MLP/ResNet, as shown at the end of the same comment). Then you have to implement the else branch in apply_model which currently simply throws an error. In simple cases, it can look like this:

def apply_model(x_num, x_cat=None):
    if isinstance(model, rtdl.FTTransformer):
        return model(x_num, x_cat)
    elif isinstance(model, (rtdl.MLP, rtdl.ResNet)):
        assert x_cat is None
        return model(x_num)
    else:
        # raise NotImplementedError(
        #     f'Looks like you are using a custom model: {type(model)}.'
        #     ' Then you have to implement this branch first.'
        # )
        assert isinstance(model, Model)
        return model(x_num, x_cat)

it seems x_cat cannot be used in rtdl.MLP, rtdl.ResNet

yes, since raw (that is, without any wrappers) rtdl.MLP, rtdl.ResNet are capable of processing only numerical features. You can check it in the implementation of their forward methods:

• MLP: link
• ResNet: link

Does this help?