That's an amazing find! Nice work :)

After work I will respond. Just need to fix those issues first :(

@arjunshajitech This is so cool and useful! I've read the changes you made to address this issue, which makes me wonder is there anyway we can acheve this without breaking existing interceptor API (I'd like to see the fix being merged as soon as possible)? Perhaps we can stop resending same RTX packet by detecting the original sequence number in nack generator_interceptor? And for media decoding to work, perhaps we can create a new interceptor that transforms all rtx packets back to original rtp packets?

@3DRX I'm glad you think so!

we can create a new interceptor that transforms all rtx packets back to original rtp packets?

I think creating a new interceptor would only address the NACK issue.
However, this issue also affects TWCC and receiver reports, as they shouldn't report packet loss if the packet was successfully received via RTX.

The fix also resolves the ( TWCC + Simulcast ) issue, but there are still some packets being reported as not received. That part still needs to be addressed.

What if we move the new interceptor to the front, let it be the first interceptor and pass down extra attributes to mark that it's an rtp media packet that's obtained via retransmission, will this fix all the problems?

Btw, I think TWCC shouldn't be affected since it only cares about the transport wise sequence number in header extension, and rtx packet should be treated as distinct ones from it's original media packet.

I'm not familiar with simulcast, so please correct me if I'm wrong.
Again, thank you so much for the work on this!

What if we move the new interceptor to the front, let it be the first interceptor and pass down extra attributes to mark that it's an rtp media packet that's obtained via retransmission, will this fix all the problems?

The root cause was that the RTX packet wasn't properly logged in the corresponding RTP received log.
From what I understand of the current behavior, the receiverLog is a map where the key is the SSRC.

So, are you suggesting that we map the corresponding SSRC in the StreamInfo of the RTX StreamInfo, and if the packet is flagged as an RTX via attributes, we process it, extract the original sequence number (OSN), and then log it in the receiverLog?
Is that what you have in mind, or is there a different approach you're thinking of?

Btw, I think TWCC shouldn't be affected since it only cares about the transport wise sequence number in header extension, and rtx packet should be treated as distinct ones from it's original media packet.

So, do we really need a sender interceptor to handle TWCC?
As you mentioned — and as I also currently understand — TWCC only depends on the transport-wide sequence number in the header extension.

Given that, can't we just pass every packet to a standalone function after reading it, where the TWCC logic is applied (e.g., extracting the transport-wide sequence number as done in the current sender interceptor), and then write the feedback RTCP using a random SSRC?

What if we move the new interceptor to the front, let it be the first interceptor and pass down extra attributes to mark that it's an rtp media packet that's obtained via retransmission, will this fix all the problems?

Additionally, since RTX packets only require TWCC feedback, there's no need for them to go through the NACK or other related interceptors, correct me if I'm wrong.

@arjunshajitech Thanks for your reply!

So, are you suggesting that we map the corresponding SSRC in the StreamInfo of the RTX StreamInfo, and if the packet is flagged as an RTX via attributes, we process it, extract the original sequence number (OSN), and then log it in the receiverLog?

Yeah that’s exactly what I’m thinking of.

Additionally, since RTX packets only require TWCC feedback, there's no need for them to go through the NACK or other related interceptors

I think by design, all packets should go through all interceptors, it’s the interceptor’s choice to whether ignore it, and whether to forward it to next interceptor.

Given that, can't we just pass every packet to a standalone function after reading it, where the TWCC logic is applied (e.g., extracting the transport-wide sequence number as done in the current sender interceptor), and then write the feedback RTCP using a random SSRC?

That’s exactly what the TWCC sender interceptor does.

@Sean-Der @JoeTurki @jech what do you think?