-
-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
error reading bigquery table having column of type datetime #20700
Comments
These don't appear to be Arrow Datetime values, but rather some custom/extension type called |
I can confirm this issue is still present in polars 1.20.0. When trying to handle BigQuery's Steps to reproduce:
The specific error encountered is: ComputeError: cannot create series from Extension(ExtensionType { name: "google:sqlType:datetime", inner: Timestamp(Microsecond, None), metadata: None }) |
Indeed - but then, we shouldn't have to; ideally BigQuery should be able to produce standard Arrow Datetime values rather than a custom "google:sqlType:datetime" ExtensionType (I'm unsure why they do this 🤔). |
Removing the metadata from the fields seems to work as a workaround:
Any idea why it worked in polars<1.18.0? Here's some sample data in case someone wants to investigate: https://storage.googleapis.com/cosmic-mariner-294413-ew6/table.parquet
|
Checks
Reproducible example
Log output
Issue description
Starting in version 1.18.0, the code gives the error shown above. I think this issue is related to the changes made in #20248 .
Expected behavior
In version 1.17.0, the code does not give an error and instead returns a dataframe having a column of type "datetime[us]".
Installed versions
The text was updated successfully, but these errors were encountered: