Skip to content

feat(openem): hackily add exchanges #8289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Jul 18, 2025
Merged

Conversation

seljaks
Copy link
Contributor

@seljaks seljaks commented Jul 17, 2025

  • I have tested my parser changes locally with poetry run test_parser "zone_key"

This is a hacky and temporary solution because the source is migrating to a new API. That API does not provide exchange or export/import data at the moment.

We parse data from openelectricity that powers their dashboard. The data includes total exports and imports for each of the connected zones in eastern Australia.

Most of the zones only have one exchange, so it's easy calculate the netflow to it's only neighbor. The only exchange that requires some math is AU-NSW->AU-VIC which can be calculated as described below

    Calculate AU-NSW->AU-VIC netflow from exports and imports of AU-NSW and AU-QLD.

    NSW has two exchanges one to QLD, one to VIC. QLD only has one exchange to NSW.
    We want to know
        AU-NSW->AU-VIC = NSW_exports_to_VIC - NSW_imports_from_VIC

    NSW has two exchanges one to QLD, one to VIC:
        NSW_exports = NSW_exports_to_QLD + NSW_exports_to_VIC
        NSW_imports = NSW_imports_from_QLD + NSW_imports_from_VIC

    and because QLD only has one exchange to NSW:
        NSW_exports_to_QLD = QLD_imports_from_NSW = QLD_imports
        NSW_imports_from_QLD = QLD_exports_to_NSW = QLD_exports

    thus
        NSW_exports_to_VIC = NSW_exports - QLD_imports
        NSW_imports_from_VIC = NSW_imports - QLD_exports

    and
        AU-NSW->AU-VIC = NSW_exports - QLD_imports - NSW_imports + QLD_exports
        = NSW_exports - NSW_imports + QLD_exports - QLD_imports

Side note: also cleaned up some of the parser code because it made no sense

@seljaks seljaks requested a review from VIKTORVAV99 as a code owner July 17, 2025 16:30
@github-actions github-actions bot added parser python Pull requests that update Python code labels Jul 17, 2025
@seljaks seljaks requested a review from BastienBigue July 17, 2025 16:30
Copy link
Member

@VIKTORVAV99 VIKTORVAV99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but I'd really like to add a snapshot test at the very least to ensure this is working correctly and that we don't get regressions in the future.

I can assist if needed 🙂

@seljaks
Copy link
Contributor Author

seljaks commented Jul 18, 2025

tests added!

@github-actions github-actions bot added the tests label Jul 18, 2025
Comment on lines 362 to 365
return [
{
"datetime": dt.to_pydatetime(),
"netFlow": value * direction,
"source": SOURCE,
"sortedZoneKeys": key,
}
for dt, value in series.iteritems()
Exchange(datetime=dt, netFlow=netflow, zoneKey=exchange_key, source=SOURCE)
for dt, netflow in datetimes_and_netflows
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't this need to use the ExchangeList model if we are to use exchanges internally as the Exchange class itself don't serialize to JSON automatically?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be happy to make the changes if you wish?

Copy link
Member

@VIKTORVAV99 VIKTORVAV99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

Nice work!

@seljaks seljaks enabled auto-merge (squash) July 18, 2025 08:42
@seljaks seljaks merged commit cbc9629 into master Jul 18, 2025
22 checks passed
@seljaks seljaks deleted the ss/fix-openem-to-add-exchanges branch July 18, 2025 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parser python Pull requests that update Python code tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants