RADE V2 prototyping

[drowe67]

Bandwidth/PAPR

Exploring ideas to improve 99% power bandwidth (spectral mask) from RADE V1. Just prototyping with "mixed rate" training and inference, i.e. no pilots or CP, genie phase.

• Worked out how to put a BPF in training loop (conv1d with training disabled)
• Take away that phase only (PAPR 0dB) works quite well
• clip-BPF x 3 produces reasonable 99% power BW, 0dB PAPR, good loss
• Doc ML EQ training and inference in README.md when we get to final V2 version. Just collect notes here in comments until then

Training:

python3 train.py --cuda-visible-devices 0 --sequence-length 400 --batch-size 512 --epochs 200 --lr 0.003 --lr-decay-factor 0.0001 ~/Downloads/tts_speech_16k_speexdsp.f32 250117_test --bottleneck 3 --h_file h_nc20_train_mpp.f32 --range_EbNo --plot_loss --auxdata --txbpf
Epoch 200 Loss 0.116

Testing:

./inference.sh 250117_test/checkpoints/checkpoint_epoch_200.pth wav/brian_g8sez.wav - --bottleneck 3 --auxdata --write_tx tx_bpf.f32 --write_latent z.f32 --txbpf
          Eb/No   C/No     SNR3k  Rb'    Eq     PAPR
Target..: 100.00  133.01   98.24  2000
Measured: 102.89          101.12       1243.47  0.00
loss: 0.121 BER: 0.000

octave:154> radae_plots; do_plots('z.f32','tx_bpf.f32')
bandwidth (Hz): 1255.813953 power/total_power: 0.990037

Red lines mark 99% power bandwidth:

ML EQ

Classical DSP:

python3 ml_eq.py --eq dsp --notrain --EbNodB 4 --phase_offset

MSE loss function:

python3 ml_eq.py --EbNodB 4 --phase_offset --lr 0.001 --epochs 100

Phase loss function:

python3 ml_eq.py --EbNodB 4 --phase_offset --lr 0.001 --epochs 100 --loss_phase

[drowe67]

Frame 2 EQ examples

1. Ideal (perfect EQ)

   python3 ml_eq.py --frame 2 --notrain --eq bypass --EbNodB 4
   <snip>
   EbNodB:  4.00 n_bits: 240000 n_errors: 3027 BER: 0.013
   

2. Classical DSP lin:

   python3 ml_eq.py --eq dsp --notrain --EbNodB 4 --phase_offset --frame 2
   <snip>
   EbNodB:  4.00 n_bits: 240000 n_errors: 3921 BER: 0.016
   

3. ML EQ (using MSE loss function):

   python3 ml_eq.py --frame 2 --lr 0.1 --epochs 100 --EbNodB 4 --phase_offset --n_syms 1000000 --batch_size 128
   <snip>
   EbNodB:  4.00 n_bits: 24000000 n_errors: 437933 BER: 0.018
   

[drowe67]

ML waveform training

1. Generate 10 hour complex h file:

   Fs=8000; Rs=50; Nc=20; multipath_samples('mpp', Fs, Rs, Nc, 10*60*60, 'h_nc20_train_mpp.c64',"",1);
   

2. Training:

   python3 train.py --cuda-visible-devices 0 --sequence-length 400 --batch-size 512 --epochs 200 --lr 0.003 --lr-decay-factor 0.0001 ~/Downloads/tts_speech_16k_speexdsp.f32 250204_test --bottleneck 3 --h_file h_nc20_train_mpp.c64 --h_complex --range_EbNo --plot_loss --auxdata
   

[drowe67]

250225: training with pilot injection (every 6th symbol a pilot), complex fading channel, +/2 Hz freq offsets, +/-1ms timing offsets:

python3 train.py --cuda-visible-devices 0 --sequence-length 400 --batch-size 512 --epochs 200 --lr 0.003 --lr-decay-factor 0.0001 ~/Downloads/tts_speech_16k_speexdsp.f32 250225_test --bottleneck 3 --h_file h_nc30_mpp_test.c64 --h_complex --range_EbNo --range_EbNo_start -3 --plot_loss --auxdata --pilots2 --latent-dim 120 --freq_rand --timing_rand

Inference (in time domain with CP). This can handle --time_offset -7 which shows training for random timing induced phase offsets works.

./inference.sh 250225_test/checkpoints/checkpoint_epoch_200.pth wav/brian_g8sez.wav /dev/null --bottleneck 3 --auxdata --rate_Fs --write_rx rx.f32 --write_latent z.f32 --latent-dim 120 --pilots2 --cp 0.004

[drowe67]

250213: complex fading channel, +/2 Hz freq offsets, +/-1ms timing offsets:

python3 train.py --cuda-visible-devices 0 --sequence-length 400 --batch-size 512 --epochs 200 --lr 0.003 --lr-decay-factor 0.0001 ~/Downloads/tts_speech_16k_speexdsp.f32 250213_test --bottleneck 3 --h_file h_nc30_mpp_test.c64 --h_complex --range_EbNo --range_EbNo_start -3 --plot_loss --auxdata --latent-dim 120 --timing_rand --freq_rand

250213a: Bottleneck 2 for comparison, had to tweak --range_EbNo_start 0 to get sensible results:

python3 train.py --cuda-visible-devices 0 --sequence-length 400 --batch-size 512 --epochs 200 --lr 0.003 --lr-decay-factor 0.0001 ~/Downloads/tts_speech_16k_speexdsp.f32 25^C13a_test --bottleneck 2 --h_file h_nc30_mpp_test.c64 --h_complex --range_EbNo --range_EbNo_start 0 --plot_loss --auxdata --latent-dim 120 --timing_rand --freq_rand

[drowe67]

Testing --correct_time_shift. This simulates a genie coarse timing estimator, so we can use proto ML EQ models with time domain --rate_Fs multipath channels, and compare to RADE V1 DSP EQ. Scatter diagram and BER is ideal when --correct_time_offset used below, fixing freq dep phase shift from --time_offset -16

./inference.sh model05/checkpoints/checkpoint_epoch_100.pth wav/brian_g8sez.wav /dev/null --rate_Fs --cp 0.004 --time_offset -16 --write_rx rx.f32 --write_latent z_hat.f32 --ber_test --correct_time_offset -16
octave:3> radae_plots; do_plots('z_hat.f32','rx.f32')

[drowe67]

First successful use of ML EQ with MPP:

./inference.sh 250227b_test/checkpoints/checkpoint_epoch_200.pth wav/all.wav /dev/null --bottleneck 3 --write_rx rx.f32 --write_latent z.f32 --rate_Fs --latent-dim 40 --cp 0.004 --time_offset -16 --correct_time_offset -32 --EbNodB 4.25 --g_file g_mpp.f32
          Eb/No   C/No     SNR3k  Rb'    Eq     PAPR
Target..:   4.25   34.25   -0.52  1250
Measured:   3.28   34.25   -0.52                 0.00
loss: 0.287

Compare with RADE V1 with DSP based EQ:

./inference.sh model19_check3/checkpoints/checkpoint_epoch_100.pth wav/all.wav /dev/null --bottleneck 3 --write_rx rx.f32 --write_latent z_hat.f32 --rate_Fs --latent-dim 80 --cp 0.004 --time_offset -16 --g_file g_mpp.f32 --auxdata --pilots --pilot_eq --eq_ls --EbNodB 3.5
         Eb/No   C/No     SNR3k  Rb'    Eq     PAPR
Target..:   3.50   36.51    1.74  3000
Measured:   1.74   36.51    1.74                 0.00
loss: 0.284 BER: 0.056

So in this test 250217b is about 2.2dB better than RADE V1. Both have CP. Note V1 running with hard clipper (not tanh) but this wasn't found to make much difference.

[drowe67]

Generating complex MPP training material for Nc=10, Note Fs=Rs:

Rs=50; Nc=10; multipath_samples('mpp', Rs, Rs, Nc, 250*60*60, 'h_nc10_mpp_train.c64',"",1);