Skip to content

Switch MD to use the ENTSOE parser #8278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

VIKTORVAV99
Copy link
Member

Issue

The current parser only get gas, hydro and renewable and assigns all renewable to biomass. This is a problem since most of the renewable is actually solar. This causes the quality validator to flag almost all of the events during the day.

Description

This PR switches it to use the ENTSO-E parser which has the following production modes:

  • Biomass
  • Coal
  • Gas
  • Hydro
  • Oil
  • Solar
  • Wind

So this is a clear improvement in the quality of the data we get.
However the catch is that this data is hourly while the current Moldova (MD) data is 15 minute. So this means we'll have to run a database migration after we have merged this and refetched all the data to remove the data from the old Moldova source.

Double check

  • I have tested my parser changes locally with poetry run test_parser "zone_key"
  • I have run pnpx prettier@2 --write . and poetry run format in the top level directory to format my changes.

@github-actions github-actions bot added parser python Pull requests that update Python code zone config Pull request or issue for zone configurations labels Jul 10, 2025
@VIKTORVAV99 VIKTORVAV99 removed python Pull requests that update Python code parser labels Jul 10, 2025
@BastienBigue
Copy link
Contributor

LGTM ! Did you check that we had the same historical depth with ENSTOE than we have with the MD source ?
We should also update the changelog to reflect this change to our users https://docs.google.com/spreadsheets/d/15ELdAhEQDQxuwlefZE6X7blrrRNwTUuDm-B1GMIxSvg/edit?gid=279914678#gid=279914678

@VIKTORVAV99
Copy link
Member Author

LGTM ! Did you check that we had the same historical depth with ENSTOE than we have with the MD source ?

We should also update the changelog to reflect this change to our users https://docs.google.com/spreadsheets/d/15ELdAhEQDQxuwlefZE6X7blrrRNwTUuDm-B1GMIxSvg/edit?gid=279914678#gid=279914678

I checked back to 2020, and we have data at least until then. Ill do a more exact one when I write the database migration so we don't lose any data.

@VIKTORVAV99
Copy link
Member Author

2020-01-15 seems to be the earliest ENTSO-E has any data for Moldova. Our current source go further back but I don't see the same issue with the earlier data as we do now, likely because they didn't have any wind or solar installed at the time. So this shouldn't cause any issues.

Copy link
Contributor

@seljaks seljaks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great find

@VIKTORVAV99
Copy link
Member Author

I'll merge this when I have the migration ready. But to be on the safe side I might not do anything until Monday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
zone config Pull request or issue for zone configurations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants