Skip to content

Pandas parser error upon access to pre 2012 rap historical data #437

@DiamondFlight

Description

@DiamondFlight

I encounter an error when attempting to grib any date before may 5th 2012 on the rap historical
Using any date past May 5th 2012 does not have this error.
I use Linux with conda, python 3.13.2

I was able to confirm that TMP:2 m does exist on the index file
index file:
https://www.ncei.noaa.gov/data/rapid-refresh/access/historical/analysis/201104/20110427/ruc2anl_252_20110427_1800_000.inv

Code below

H = Herbie(
         datetime.datetime(2011, 4, 27, 18),
         model = 'rap_historical',
         product = 'analysis',
         fxx = 0,
 )
H.download("TMP:2 m")
print(H.xarray("TMP:2 m"))

Output below

✅ Found ┊ model=rap_historical ┊ product=analysis ┊ 2011-Apr-27 18:00 UTC F00 ┊ GRIB2 @ ruc_anl_252 ┊ IDX @ ruc_anl_252
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/mnt/c/Users/-/Downloads/PythonDataPlotter/.venv/__main__.py", line 143, in <module>
    Data = PR.PD()
           ^^^^^^^
  File "/mnt/c/Users/-/Downloads/PythonDataPlotter/.venv/processdataherbie.py", line 161, in __init__
    H.download("TMP:2 m")
          ^^^^^^^^^^^^^^^^^^^^^
  File "/home/diamond/miniconda3/lib/python3.12/site-packages/herbie/core.py", line 984, in download
    outFile = self.get_localFilePath(search)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/diamond/miniconda3/lib/python3.12/site-packages/herbie/core.py", line 566, in get_localFilePath
    idx_df = self.inventory(search)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/diamond/miniconda3/lib/python3.12/site-packages/herbie/core.py", line 806, in inventory
    df = self.index_as_dataframe
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/diamond/miniconda3/lib/python3.12/functools.py", line 995, in __get__
    val = self.func(instance)
          ^^^^^^^^^^^^^^^^^^^
  File "/home/diamond/miniconda3/lib/python3.12/site-packages/herbie/core.py", line 652, in index_as_dataframe
    df = pd.read_csv(
         ^^^^^^^^^^^^
  File "/home/diamond/miniconda3/lib/python3.12/site-packages/pandas/io/parsers/readers.py", line 1026, in read_csv
    return _read(filepath_or_buffer, kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/diamond/miniconda3/lib/python3.12/site-packages/pandas/io/parsers/readers.py", line 626, in _read
    return parser.read(nrows)
           ^^^^^^^^^^^^^^^^^^
  File "/home/diamond/miniconda3/lib/python3.12/site-packages/pandas/io/parsers/readers.py", line 1923, in read
    ) = self._engine.read(  # type: ignore[attr-defined]
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/diamond/miniconda3/lib/python3.12/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 234, in read
    chunks = self._reader.read_low_memory(nrows)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "parsers.pyx", line 838, in pandas._libs.parsers.TextReader.read_low_memory
  File "parsers.pyx", line 905, in pandas._libs.parsers.TextReader._read_rows
  File "parsers.pyx", line 874, in pandas._libs.parsers.TextReader._tokenize_rows
  File "parsers.pyx", line 891, in pandas._libs.parsers.TextReader._check_tokenize_status
  File "parsers.pyx", line 2061, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: EOF inside string starting at row 158

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions