Repalcetrack is invalid

[sxmzou]

Vesion :release v4.1.6
Test scenario: I am modifying the backend based on examples/play from disk negotiation, using PION in conjunction with ffmpeg to capture two real cameras. The frontend uses the WebRTC API in the browser
I want to implement camera switching through replacetrack, but the phenomenon I have observed now is that when the first track is just started, it can be successfully switched through replacetrack. However, when the first track is started for a certain period of time, replacing track to the second track will cause the image to freeze. I found through Wireshark packet capture that the seq and timestamp after replacetrack are not continuous. Could this be the problem

[Sean-Der]

Hi @sxmzou

How are you generating the video? Does the encoder generate a keyframe on replaceTrack?

For testing can you try a low keyframe interval?

Are you using WriteSample or WriteRTP?

[sxmzou]

@Sean-Der Thank you for replying to me. I used ffmpeg to read H264 data and sent it out through the writiesample function. When I sent it out, I found that the SSRC was consistent with the previous track, but the seq and timestamp were not continuous

[sxmzou]

And every time I perform a replacetrack, I restart ffmpeg and ask it to force a refresh of an i-frame

[Sean-Der]

Ok this does seem to be broken.

We maintain Sequence numbers/Timestamp in Tracks. I think in Bind we should include this.

I will work on fixing now

[Sean-Der]

@sxmzou Would it be possible to use one Track and write your different H264 streams to it? I don’t think this is going to be a quick fix unfortunately

[sxmzou]

Yes, that's what I'm doing now. Thank you for your suggestion😁