embedding group

[xiexbing]

Can you explain how the embedding group can contribute to the better performance of embedding layer?

[tiankongdeguiji]

During training, TorchEasyRec's EmbeddingGroup leverages TorchRec's EmbeddingBagCollection and EmbeddingCollection to fusion a large number of feature embedding lookup operations into as few operations as possible.

During inference, TorchEasyRec's EmbeddingGroup further optimizes the embedding computation on the user side, reducing the required calculations from batch_size times to just once.

[xiexbing]

for the training, how the fusion helps? more specifically, at low level, where the benefit comes from? e.g., does the fusion helps on the better utilization on the communication in the forward pass by better batching the communication data? does the fusion helps on reduce the number of cuda kernel launching on pooling in forward? please help provide details or point to the detailed documentations/papers.

[tiankongdeguiji]

the fusion helps on reduce the number of embedding lookup cuda kernel launching and better utilization on the communication.

[xiexbing]

thanks! I'm trying to write a single example on using embeddinggroup. Can you point me to some example code that I can refer? in the taobao multi-tower example, seem the tzrec model is built from features and embeddinggroup is not used.

[tiankongdeguiji]

thanks! I'm trying to write a single example on using embeddinggroup. Can you point me to some example code that I can refer? in the taobao multi-tower example, seem the tzrec model is built from features and embeddinggroup is not used.

For an example of EmbeddingGroup, please refer to the embedding_test.py file here. In the taobao Multi-Tower model example, the embedding_group is utilized within the base RankModel, which you can find here.