0x7F 0x7F block header is not at offset 0 after frequency change on Raspberry Pi Zero 2W in Sweep Mode

[targetod]

What type of issue is this?

permanent - occurring repeatedly

What issue are you facing?

After changing the frequency in my C++ application (using libhackrf), the expected block header 0x7F 0x7F is not the first two bytes of the data buffer. Instead, it consistently appears at an offset of 15872 bytes. This behavior is reproducible on a Raspberry Pi Zero 2W but does not seem to occur on a Raspberry Pi 4.

What are the steps to reproduce this?

Steps to Reproduce

1. Initialize HackRF device and libhackrf.
2. Set a specific frequency and sample rate.
3. Initialize sweep mode, start sweep mode
4. Run 10 seconds
5. Stop, change frequency, start sweep mode
6. Check sweep mode

C++ code

#include <hackrf.h>
#include <iostream>
#include <thread>
#include <chrono>

#define EXPECTED_BLOCKS 262144

static bool sweep_running = true;
static int iteration = 0;

static int rx_callback(hackrf_transfer* transfer) {
    iteration++;
    if (iteration % 10000 == 0) {
        std::cout << "Iteration: " << iteration << std::endl;
    }

    if (transfer->valid_length != EXPECTED_BLOCKS) {
        std::cerr << "[" << iteration << "] Received less than " << EXPECTED_BLOCKS << " bytes; valid_length=" << transfer->valid_length << std::endl;
    }
    int8_t* buf = (int8_t*) transfer->buffer;
    uint8_t* ubuf = (uint8_t*) buf;
    for (size_t i = 0; i < EXPECTED_BLOCKS - 1; ++i) {
        if (ubuf[i] == 0x7F && ubuf[i + 1] == 0x7F) {
            if (i != 0) {
                std::cout << "[" << iteration << "] 7F 7F found at offset: " << i << std::endl;
            }
            break;
        }
    }
    return sweep_running ? 0 : -1;
}

bool run_sweep(hackrf_device* device, uint16_t* frequencies, size_t freq_len) {
    sweep_running = true;
    iteration = 0;

    auto res = hackrf_init_sweep(device, frequencies, freq_len, BYTES_PER_BLOCK, 20000000, 0, LINEAR);
    if (res != HACKRF_SUCCESS) {
        std::cerr << "Failed to initialize sweep: " << hackrf_error_name((hackrf_error)res) << std::endl;
        return false;
    }

    int ret = hackrf_start_rx_sweep(device, rx_callback, nullptr);
    if (ret != HACKRF_SUCCESS) {
        std::cerr << "Failed to start sweep: " << hackrf_error_name((hackrf_error)ret) << std::endl;
        return false;
    }

    std::this_thread::sleep_for(std::chrono::seconds(10)); // Use 10s for demo; adjust as needed
    sweep_running = false;

    hackrf_stop_rx(device);
    std::this_thread::sleep_for(std::chrono::milliseconds(500));
    return true;
}

int main() {
    hackrf_device* device = nullptr;

    if (hackrf_init() != HACKRF_SUCCESS) {
        std::cerr << "hackrf_init failed\n";
        return -1;
    }

    if (hackrf_open(&device) != HACKRF_SUCCESS || device == nullptr) {
        std::cerr << "Failed to open HackRF device\n";
        hackrf_exit();
        return -1;
    }

    hackrf_set_sample_rate(device, 20000000);
    hackrf_set_lna_gain(device, 16);
    hackrf_set_vga_gain(device, 20);
    hackrf_set_amp_enable(device, 0);

    // First sweep
    uint16_t frequencies1[2] = {5000, 5300}; // MHz
    std::cout << "Starting sweep 1..." << std::endl;
    if (!run_sweep(device, frequencies1, 2)) {
        std::cerr << "Sweep 1 failed" << std::endl;
        hackrf_close(device);
        hackrf_exit();
        return -1;
    }
    std::cout << "Sweep 1 completed successfully" << std::endl;

    // Full reset and re-initialization
    hackrf_close(device);
    hackrf_exit();


    if (hackrf_init() != HACKRF_SUCCESS) {
        std::cerr << "hackrf_init failed after reset\n";
        return -1;
    }
    if (hackrf_open(&device) != HACKRF_SUCCESS || device == nullptr) {
        std::cerr << "Failed to re-open HackRF device\n";
        hackrf_exit();
        return -1;
    }
    hackrf_set_sample_rate(device, 20000000);
    hackrf_set_lna_gain(device, 16);
    hackrf_set_vga_gain(device, 20);
    hackrf_set_amp_enable(device, 0);
    std::cout << "HackRF device re-initialized successfully" << std::endl;

    // Second sweep with different frequencies
    uint16_t frequencies2[2] = {6000, 6300}; // MHz
    std::cout << "Starting sweep 2..." << std::endl;
    if (!run_sweep(device, frequencies2, 2)) {
        std::cerr << "Sweep 2 failed" << std::endl;
        hackrf_close(device);
        hackrf_exit();
        return -1;
    }
    std::cout << "Sweep 2 completed successfully" << std::endl;

    hackrf_close(device);
    hackrf_exit();
    return 0;
}

Can you provide any logs? (output, errors, etc.)

Operating system:

pi@rasp:~ $ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

pi@rasp:~ $ lsusb -t
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=dwc2/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=usbfs, 480M

pi@rasp:~ $ ldconfig -p | grep libusb
  libusb-1.0.so.0 (libc6,AArch64) => /lib/aarch64-linux-gnu/libusb-1.0.so.0

pi@rasp:~ $ dpkg -s libusb-1.0-0 | grep Version
Version: 2:1.0.26-1

hackrf_info output:

hackrf_info version: 2024.02.1
libhackrf version: 2024.02.1 (0.9)
Found HackRF
Index: 0
Serial number: 0000000000000000a31c64dc2160920f
Board ID Number: 2 (HackRF One)
Firmware Version: 2024.02.1 (API:1.08)
Part ID Number: 0xa000cb3c 0x004f475a
Hardware Revision: older than r6
Hardware supported by installed firmware:
    HackRF One

I have collected two libusb debug log files for analysis:

libusb_debug.log: Collected on the Raspberry Pi Zero 2W, where the issue is reproducible
libusb_debug_nonrepro_1.log: Collected on the Raspberry Pi 4

libusb_debug.log
libusb_debug_nonrepro_1.log

[AlfaScript]

Hi guys, @miek,

I've been investigating the totally same issue and I'm observing similar behavior while using sweep mode. Specifically, the pattern "0x7F 0x7F" becomes shifted in the received data stream - by a constant but variable offset (e.g., 15872 bytes, or even 5 in one case). Once the shift occurs, it persists until the device is reset

What I've tested:

1. Increased wait/sleep time after frequency change - didn't help
2. Added totally close and re-init during frequency change - didn't help
3. Added wait/sleep time between re-initialization - didn't help
4. I verified that the issue exists even when callback is almost empty

Thanks in advance
Hope my info somehow helps you

[martinling]

What does hackrf_debug -S report if you run it right after a failed sweep?

[targetod]

Hi @martinling , @miek ,
I see the next output

Debug output before

pi@rasp:~ $ ./small_test_one; ./hackrf-tools/hackrf_debug -S
Starting sweep 1...
Sweep 1 completed successfully
M0 state:
Requested mode: 0 (IDLE) [complete]
Active mode: 0 (IDLE)
M0 count: 400137632 bytes
M4 count: 400097280 bytes
Number of shortfalls: 0
Longest shortfall: 0 bytes
Shortfall limit: 0 bytes
Mode change threshold: 400146432 bytes
Next mode: 0 (IDLE)
Error: 0 (NONE)

And the offset happened after the first start

pi@rasp:~ $ ./small_test_one; ./hackrf-tools/hackrf_debug -S
Starting sweep 1...
[1] 7F 7F found at offset: 15872
[2] 7F 7F found at offset: 15872
[3] 7F 7F found at offset: 15872
...
[504] 7F 7F found at offset: 15872
[505] 7F 7F found at offset: 15872
[506] 7F 7F found at offset: 15872
[507] 7F 7F found at offset: 15872
[508] 7F 7F found at offset: 15872
Sweep 1 completed successfully
M0 state:
Requested mode: 0 (IDLE) [complete]
Active mode: 0 (IDLE)
M0 count: 400156704 bytes
M4 count: 400146432 bytes
Number of shortfalls: 0
Longest shortfall: 0 bytes
Shortfall limit: 0 bytes
Mode change threshold: 400162816 bytes
Next mode: 0 (IDLE)
Error: 0 (NONE)

@AlfaScript, FYI

[miek]

I bought a Pi Zero 2W, set it up this morning, and was able to reproduce the issue - thank you for the work you've done narrowing down a reliable test case!

pi@pi2w:~ $ ./small_test_one 
Starting sweep 1...
Sweep 1 completed successfully
HackRF device re-initialized successfully
Starting sweep 2...
[1] 7F 7F found at offset: 15872
[2] 7F 7F found at offset: 15872
[3] 7F 7F found at offset: 15872
[4] 7F 7F found at offset: 15872
[...]

I also took a capture with Cynthion while it happened and the relevant part is below:

What's interesting is that there doesn't seem to be anything going wrong on the firmware side - after starting the second sweep, the 0x7F 0x7F marker is still at the start of each transfer as it should be.

I suspect that, since sweep 1 ended with a partial transfer, the host is rolling the first packet(s) of sweep 2 into the previous transfer (though the maths doesn't quite add up - that transfer missed 3 packets but the offset is 1 packet?).

Now, libhackrf cancels all transfers in progress when calling hackrf_close, so it really shouldn't matter whether that previous transfer was incomplete. So, I'm wondering if there is an issue further up in the stack? Or a disagreement about the appropriate behaviour in this situation, since the USB2 spec doesn't seem to specify what should happen when a client cancels an IRP.

[AlfaScript]

Hi @miek ,

Thank you for your involvement in this issue

As I understood, the problem is on the host side. Do you have any suggested workarounds?
Also, did you record the capture using Cynthion during the reproduction of offsets? (just to be sure, we're on the same page)

@targetod , do you have a workaround? (if it's not private)

Thanks, everyone :)

[targetod]

Hi @AlfaScript, unfortunately, I don't have a workaround yet.

[miek]

I had another idea about the cause and I've done a bit more experimenting.

The only offset I've seen in my testing is 15872, which suggests that a single 512 byte packet has gone missing and put everything out of sync. We've seen this sort of thing a few times and it often comes down to the data toggle bits being out of sync between the host and device.

So I tried with the following workaround patch in libhackrf:

diff --git a/host/libhackrf/src/hackrf.c b/host/libhackrf/src/hackrf.c
index 728f8961..10a21754 100644
--- a/host/libhackrf/src/hackrf.c
+++ b/host/libhackrf/src/hackrf.c
@@ -256,6 +256,8 @@ static int cancel_transfers(hackrf_device* device)
                }
                pthread_mutex_unlock(&device->transfer_lock);
 
+               libusb_clear_halt(device->usb_device, RX_ENDPOINT_ADDRESS);
+
                return HACKRF_SUCCESS;
        } else {
                return HACKRF_ERROR_OTHER;

This forces both sides to reset the data toggle bit on the RX endpoint after cancelling all of the transfers.

I've run the test case again in a loop for a while now and I haven't seen it fail again with this patch in place. Please give it a try and let me know your results.

[AlfaScript]

Hi @miek,

I've tried to run your fix. But unfortunately, it's not working for my application
Then I've tried this fix + @targetod 's test application and it's not working too :(

Maybe @targetod will have another result. But I still see offsets:

pi@rpi:~ $ for i in {1..10} ; do echo "Iteration="$i ; echo "" ; ./test_offsets ; echo "" ; done
Iteration=1

Starting sweep 1...
Sweep 1 completed successfully
HackRF device re-initialized successfully
Starting sweep 2...
[1] 7F 7F found at offset: 15872
[2] 7F 7F found at offset: 15872
[3] 7F 7F found at offset: 15872
[4] 7F 7F found at offset: 15872
[5] 7F 7F found at offset: 15872
...

Maybe I didn't build it properly? What I did:

$ cd ..
$ git submodule init
$ git submodule update
$ cd hackrf_usb
$ mkdir build
$ cd build
$ cmake ..
$ make
$ hackrf_spiflash -w hackrf_usb.bin

Thanks

[miek]

The workaround I proposed is a change to libhackrf on the host, it's not a change to the firmware. You need to build and install the host tools from source on the Pi. These would be the rough steps to follow:

1. Apply the patch by editing host/libhackrf/src/hackrf.c and adding that libusb_clear_halt line. You can check it's correct by running git diff and comparing the output to what I pasted above.

2. If you've already installed distro-provided copies of libhackrf and hackrf-tools, remove those:

sudo apt remove libhackrf-dev libhackrf0 hackrf

3. Install the dependencies for building from source:

sudo apt install build-essential cmake libusb-1.0-0-dev libfftw3-dev

4. Build and install the host tools:

$ cd host
$ mkdir build
$ cd build
$ cmake ..
$ make
$ sudo make install
$ sudo ldconfig

5. Rebuild the test application

[AlfaScript]

BINGOOOO!
It's really working!

THANKS A LOT FOR YOU, @miek <3

@targetod , just FYI

[targetod]

Hi @miek ,
This seems to be a good solution for me too.
I really appreciate your quick response and the resources you spent on purchasing the RPI Zero 2W.