Concern regarding parameter initialization in PoincareLinear module

[nbuton]

Dear Authors,

Thank you for your work on this project. I have a concern regarding the initialization of parameters in the PoincareLinear module, specifically with regard to the two parameters defined: one for the weight and the other for the norm of the weight.

It appears that the two parameters are defined independently, which could lead to inconsistencies when the optimizer changes the weight and weight norm separately. I believe that these two parameters should be connected in order to ensure that they are changed together. More precisely, only weight_v should be a parameter and weigh_g should be computed in the forward pass.

In order to validate my hypothesis, I recommend temporarily adding an assert statement in the forward pass as shown below:

assert torch.sum(torch.abs(self.weight_v.norm(dim=0)-self.weight_g))<0.1

This will not verify the assertion very quickly.

I could be wrong, but I would appreciate your clarification on this matter.

Thank you in advance.

[abfariah]

I very much agree with @nbuton, as I also was astonished by this parameter initialization. It would be great to hear from the authors.