Split forward pass and loss calculation during training

[mwindsp]

In the current implementation, the forward pass and loss calculation during training is done in one step. Both input and ground truth data are fed into the model.

In certain scenarios, it is however useful to split this into two distinct steps. E.g. I would like to calculate the loss twice using different ground truth data, which currently requires to perform the actual forward pass twice resulting in unnecessary computations.

Therefore, it would be beneficial, if the framework supports to split both steps.

[Programmer-RD-AI]

Hi,
Is there anything supporting this that shows this will increase performance or provide any sort of benefit?
And could you elaborate on how this would work..?
Thank you