You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Just as a sanity check I checked and pandas/vaex/cuDF/modin all return the type described in Column.dtype for the get_buffers() values. A. TODO I realize for dataframe-interchange-tests is generalise test_dtype and use it in test_get_buffers.
Note that this is actually not correct, depending on how you interpret it. Yes, the buffers' dtype returns a similar type of DType tuple, but it should not necessarily return the same dtype tuple as its Column.dtype does, as the buffer can have a different dtype than the column.
It seems that we all interpreted this wrongly and all implementations got this wrong (or the text about "the data buffer's associated dtype" is wrong), see apache/arrow#37598, pandas-dev/pandas#54781, pola-rs/polars#10787 (and the same for StaticFrame mentioned above, from a quick look).
jorisvandenbossche
changed the title
Clarify the exact meaning of the buffer's dtype (Dtype tuple in get_buffers())
[protocol] Clarify the exact meaning of the buffer's dtype (Dtype tuple in get_buffers())
Oct 12, 2023
Note that this is actually not correct, depending on how you interpret it. Yes, the buffers' dtype returns a similar type of DType tuple, but it should not necessarily return the same dtype tuple as its
Column.dtype
does, as the buffer can have a different dtype than the column.It seems that we all interpreted this wrongly and all implementations got this wrong (or the text about "the data buffer's associated dtype" is wrong), see apache/arrow#37598, pandas-dev/pandas#54781, pola-rs/polars#10787 (and the same for StaticFrame mentioned above, from a quick look).
Originally posted by @jorisvandenbossche in #87 (comment)
The text was updated successfully, but these errors were encountered: