-
Notifications
You must be signed in to change notification settings - Fork 425
Properly determine libusb read size for large reports (Fixes #274) #728
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
068fd62
to
52c6cd9
Compare
PR updated to remove different signedness comparison compiler warning when compiling with |
Thank you for finally taking care of this longstanding bug. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this, I'll perform a thorrow review a bit later.
@Be-ing can you confirm it fixes the issue with long reports too?
libusb/hid.c
Outdated
Requires an opened device with *claimed interface*. | ||
|
||
The return value is the size on success and -1 on failure. */ | ||
static size_t get_max_input_size(libusb_device_handle *handle, int interface_num, uint16_t expected_report_descriptor_size) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function seems to be the functional the same as InputReportByteLength
from https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/hidpi/ns-hidpi-_hidp_caps on Windows.
We have a lot of test data for the Windows unit test, which stores the result of this function:
pp_data->caps_info[0]->ReportByteLength = 16 |
together with the real ReportDescriptor https://github.com/libusb/hidapi/blob/master/windows/test/data/045E_02FF_0005_0001_real.rpt_desc
What do you think about creating a unit test for the new get_max_input_size
function using the _real.rpt_desc
files as input and compare the result with the value stored in the .pp_data
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice! That does look like a great dataset. I'll see if I can figure a clean way to make a unit test, and see how it fares with that data.
Currently the get_max_input_size
is a static
function, but I need a way to use it externally in the unit test. It seems that the windows library has some extensions prefixed with winapi
. Should I rename the get max function to hid_libusbapi_get_max_input_report_size
and mark it HID_API_EXPORT_CALL
?
I will also need to change how it gets the report descriptor, but that won't be a problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Making it with HID_API_EXPORT_CALL
would make it a public (at least on binary level) function, which is undesirable, if it is required for tests only. Maybe export it only when building unit-tests etc. or try to avoid it by using static linking or smth else (or have it in an internal header file?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have a look at the Windows unit test ( https://github.com/libusb/hidapi/tree/master/windows/test ) for the report descriptor reconstructor. There we had exactly the same challenges, hid_winapi_descriptor_reconstruct_pp_data was the internal function to test there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JoergAtGithub I did actually look at hid_winapi_descriptor_reconstruct_pp_data
, and it is marked as HID_API_EXPORT_CALL
in the header file:
hidapi/windows/hidapi_winapi.h
Line 68 in 0ab6c14
int HID_API_EXPORT_CALL hid_winapi_descriptor_reconstruct_pp_data(void *hidp_preparsed_data, unsigned char *buf, size_t buf_size); |
Although, now that I am looking, it is not marked in the C file:
int hid_winapi_descriptor_reconstruct_pp_data(void *preparsed_data, unsigned char *buf, size_t buf_size) |
WRT @Youw's suggestion. I could simplify the function a bit so that it doesn't depend on anything in hid.c
and then move it to hid_max_input_report.h
. If that is acceptable, it is probably the easiest solution.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hid_winapi_descriptor_reconstruct_pp_data, and it is marked as HID_API_EXPORT_CALL in the header file
Right - that is a public API function. That is also the reason why there is a winapi
prefix in the name.
it is not marked in the C file
Not required, if it is marked in the header.
I could simplify the function a bit so that it doesn't depend on anything in hid.c and then move it to hid_max_input_report.h. If that is acceptable, it is probably the easiest solution.
Yes, sounds like simples solution for now. Go for it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a test, but it isn't passing. It could be that I am misunderstanding the Windows HID structures. I have only a vague understanding of them.
For example, in the 046A_0011_0006_0001
test data, it gives a max input size of 9 bytes:
hidapi/windows/test/data/046A_0011_0006_0001.pp_data
Lines 16 to 19 in 0ab6c14
pp_data->caps_info[0]->FirstCap = 0 | |
pp_data->caps_info[0]->LastCap = 2 | |
pp_data->caps_info[0]->NumberOfCaps = 2 | |
pp_data->caps_info[0]->ReportByteLength = 9 |
But two input caps (cap[0]
and cap[1]
) are:
hidapi/windows/test/data/046A_0011_0006_0001.pp_data
Lines 37 to 38 in 0ab6c14
pp_data->cap[0]->BitSize = 1 | |
pp_data->cap[0]->ReportCount = 8 |
hidapi/windows/test/data/046A_0011_0006_0001.pp_data
Lines 83 to 84 in 0ab6c14
pp_data->cap[1]->BitSize = 8 | |
pp_data->cap[1]->ReportCount = 6 |
According to the documentation I read, the report size is just BitSize * ReportCount
(plus a byte for the report number I assume).
This gives report sizes of (1*8)/8 + 1 = 2
and (8*6)/8 + 1 = 7
. So I don't know where the ReportByteLength = 9
value is coming from.
Do either of you know what I am missing here?
BTW, there is a typo here.
|
I put
The InputReport contains:
|
There is something missing, as this is autogenerated by pp_data_dump, I guess there was something in the manufacturer_string , that pp_data_dump couldn't handle. Could you please open a dedicated issue for this, as this is unrelated to this PR. |
|
52c6cd9
to
2708264
Compare
2708264
to
3710895
Compare
if(NOT EXISTS "${TEST_PP_DATA}") | ||
message(FATAL_ERROR "Missing '${TEST_PP_DATA}' file for '${TEST_CASE}' test case") | ||
endif() | ||
set(TEST_EXPECTED_DESCRIPTOR "${CMAKE_CURRENT_LIST_DIR}/../../windows/test/data/${TEST_CASE}_expected.rpt_desc") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You need to parse the _real.rpt_desc
, not the _expected.rpt_desc
. The real is what is dumped from the device, and the expected is what the Windows ReportDescriptor-Reconstructor generates out of the .pp_data.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. I assumed that the real
and expected
should contain equivalent reports.
The _real.rpt_desc
files do not follow a consistent format, so should I parse them manually (should be pretty doable with a bit of regex) into new files and add them to the repo?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe you can put them into https://eleccelerator.com/usbdescreqparser/ to unify the format. That makes them also human readable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea.
if (report_count < 0 || report_size < 0) { | ||
/* We are missing size or count. That isn't good. */ | ||
return 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (report_count < 0 || report_size < 0) { | |
/* We are missing size or count. That isn't good. */ | |
return 0; | |
if (report_count < 0 || report_size < 0) { | |
/* We are missing size or count. That isn't good. */ | |
return -1; |
This would mean a corrupt ReportDescriptor
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't consistent with whether I was using size_t
or ssize_t
. In my last commit, I settled on size_t
, but perhaps I should change it back to ssize_t
so that we can more clearly differentiate between errors and a zero value (if there are no feature reports, it will return 0, which is not an error).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense!
Thanks @JoergAtGithub for walking me through that. I was misreading the descriptor. I have added tests for libusb using the same data as the windows tests. And I extended the functionality of the libusb method to be able to calculate the maximum output and feature report sizes as well. This has no functional use currently, but it lets us run three times as many tests since the pp_data files have all three max sizes available. |
Independent of this PR, I think it would generally make sense to store these 3 values in the device structure. On Windows we would have to use the values |
@@ -70,8 +70,8 @@ if(HIDAPI_ENABLE_ASAN) | |||
endif() | |||
endif() | |||
|
|||
if(WIN32) | |||
# so far only Windows has tests | |||
if(WIN32 OR HIDAPI_WITH_LIBUSB) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also need to update builds.yml
add_test(NAME "LibUsbHidMaxInputReportSizeTest_${TEST_CASE}" | ||
COMMAND max_input_report_size_test "${TEST_PP_DATA}" "${TEST_EXPECTED_DESCRIPTOR}" | ||
WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}" | ||
#WORKING_DIRECTORY "$<TARGET_FILE_DIR:hidapi_winapi>" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#WORKING_DIRECTORY "$<TARGET_FILE_DIR:hidapi_winapi>" |
set(CMAKE_VERSION_SUPPORTS_ENVIRONMENT_MODIFICATION "3.22") | ||
|
||
foreach(TEST_CASE ${HID_DESCRIPTOR_RECONSTRUCT_TEST_CASES}) | ||
set(TEST_PP_DATA "${CMAKE_CURRENT_LIST_DIR}/../../windows/test/data/${TEST_CASE}.pp_data") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe move the test data to <root>/test_data
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't have a strong opinion on test implementation (I trust @JoergAtGithub on this one).
libusb implementation seem fine.
Lets make sure it runs with CI on Github Actions and we're good to go here.
Lets continue here: #731 |
This PR does not seem to work. Test device is the same as the one used in Issue #274. The FW is a mod of Jan Axelson's FX2HID example and codes are included in the following #274 comment. I can reproduce the issue reported in #274 with hidapi git libusb backend, hidraw backend is okay.
With this PR, no change in hidraw backend behavior, which is good.
But the libusb backend fix is not working.
|
HID Report Descriptor.
hidtest is okay.
|
If I have time, I'll try to address @JoergAtGithub and @Youw's comments on Monday. @mcuee I tried feeding the report descriptor that you sent into I did notice a typo in your test. When you ran |
Let me try again.
|
Same problem under FreeBSD 14.1 Release. This is with a physical machine, Chuwi mini PC, Intel J4125 CPU, 8GB/256GB configuration. Without this PR, hidapi git has the problem mentioned in #274. But then somehow this PR does not work.
|
If I use the original fx2hid example (loop back of two bytes output report and input report), it seems to me this PR works fine. But that just means this PR has no regression.
|
Just want to make sure the FW works fine, here is the test under Windows (native Windows HID backend). Original fx2hid FW.
My mod fx2hid FW (128 bytes report).
|
Just wondering if you can carry out the test using your test device with long reports as well. Thanks. |
There are also github action build failure.
|
Currently the libusb version of hidapi simply reads up to
wMaxPacketSize
bytes as the report. This is problematic when reports are longer thanwMaxPacketSize
. The current behavior will split that report up.The proper solution is to review the report descriptor to find the longest input report and use that as the length of the
libusb_fill_interrupt_transfer
buffer. (Note: there is no need to manually get multiple USB packets and concatenate them together to fit the report length. USB already handles that for us.)This will still work for HID devices when some input reports are shorter than others. The HID device will just send a short packet terminator and libusb will give us the shorter buffer.
The substance of these changes is in the
get_max_input_size
method. It uses the same basic report descriptor parsing asget_usage
. I considered changing the code so that there could be shared parsing code, but I decided that was overkill for this.Fixes #274