Throughput is not improved with the increase of batch size

[Irr-free]

I'm using llama3 with a single Nividia V100 GPU (32GiB memory). When I increase the batch size from 1 to 8, the inference throughput does not increase, but it decreases. However, when I set the batch size to 16, throughput increases.
The max_seq_len in my config is 512, and I have not modified any other codes of the model.
Is this related to the cuda core or tensor core config of NVIDIA GPU?