Feeding small chunks of data to decoder cause latency?

[iownthegame]

In the source code I see you push 4096 bytes to decoder each time. I'm now using libde265.js to do streaming and instead of 4096bytes I use the data that the server side send me every time to feed the decoder. The length of the data I received is the size of a frame, thus it may be variant, sometimes large while sometimes very small. I think most of the time the data size is smaller than 4096 bytes. Then I call decoder.push_data() and decoder.decode() per frame, which is very frequent.

Is this why it causes some latency and I feel lag when I do other things in javascript such as tracking my mouse position? I don't know if I need to wait the received data size to be larger than 4096 bytes and then do decode() once, since it could also causes some 'waiting time' to reach the chunk size.