Enable user timestamp/pacing in Tx Audio pipeline

[DianaEs2]

In order to achieve synchronization between 2110-20 and 2110-30 streams, so-called lip sync, we need to pass to MTL the custom timestamps. There are two flags for that in the library.
/**

• Flag bit in flags of struct st30_tx_ops.
• User control the frame pacing by pass a timestamp in st30_tx_frame_meta,
• lib will wait until timestamp is reached for each frame.
  /
  #define ST30_TX_FLAG_USER_PACING (MTL_BIT32(3))
  /*
• Flag bit in flags of struct st30_tx_ops.
• If enabled, lib will assign the rtp timestamp to the value in
• st30_tx_frame_meta(ST10_TIMESTAMP_FMT_MEDIA_CLK is used)
  */
  #define ST30_TX_FLAG_USER_TIMESTAMP (MTL_BIT32(4))

The flags are available in frame-mode st30_xxx but not in pipeline mode st30p_xxx. Please expose them in the pipeline mode. If possible please extend RxTxApp sample to test this feature.

[DianaEs2]

I checked the latest code of MTL, and ST30P_TX_FLAG_USER_PACING is enabled now. If my understanding is correct, the user timestamp should be submitted via st30p_tx_put_frame() in st30_frame::timestamp with st30_frame::tfmt = ST10_TIMESTAMP_FMT_TAI. When st30_frame::timestamp is in the past, the MTL library sends the audio frame immediately, which is likely correct. The other option would be to drop such a frame. But transmitting them fast is also acceptable.

When st30_frame::timestamp is in the future, by more than a second, the MTL also sends audio at once, without waiting for the scheduled time. It does not look correct to me.

MTL: Error: tx_audio_session_tasklet_transmit(0), invalid tsc cur 1918934866624967 target 1919575444749187
MTL: Error: tx_audio_session_tasklet_transmit(0), invalid tsc cur 1918934866632004 target 1919575445749025
MTL: Error: tx_audio_session_tasklet_transmit(0), invalid tsc cur 1918934866638744 target 1919575446749250
MTL: Error: tx_audio_session_tasklet_transmit(0), invalid tsc cur 1918934866645583 target 1919575447749117
MTL: Error: tx_audio_session_tasklet_transmit(0), invalid tsc cur 1918934866656752 target 1919575448749127
MTL: Error: tx_audio_session_tasklet_transmit(0), invalid tsc cur 1918934866663681 target 1919575449749346
....

I will send you a patch shortly to repo this issue in RxTxApp.

[DianaEs2]

Here is audio_pacing.diff for RxTxApp to reproduce the user pacing problem.

audio_pacing.diff.patch

[DianaEs2]

If I remove two lines in the diff above. I get a better result, but still, the stream is non-compliant.
// int64_t tmsp_shift = 10. * NS_PER_S; // 10 sec
// tai_ns += tmsp_shift; // to show pacing problem

MTL log is here mtl_audio.log

The audio pcap audio_380_7.zip

The EBU list shows this warning. If I understand it correctly, audio packets are sent too early relative to PTP.

[DianaEs2]

There is an interesting setting in struct st20p_tx_ops
/**

• Optional. The rtp timestamp delta(us) to the start time of frame.
• Zero means the rtp timestamp at the start of the frame.
  */
  int32_t rtp_timestamp_delta_us;

If you update the struct st30p_tx_ops with a similar setting, then audio latency changes, and the stream will be compliant.
Here is the patch I tested.

commit-f9f90c8.patch