order_data: inputs for electrical or heat "consumption" are converted to efficiencies or filtered out

[BertoGBG]

Checklist

• I am using the current master branch or the latest release. Please indicate: YES

ISSUE:

When adding new technologies (e.g. from DEA) which are consuming electricity or heat (i.e. processes for e- fuel production), the electricity and heat consumption is read from the initial excel files but filtred out in the final df, or re-calculated as electrical or heat efficiency

Following the main, if the indexes "electricity consumption" and "heat consumption" are present in the original excel file they are correctly saved in the df "tech_data" but are not included (or strangely modified) in the df "data".

this happens within function "order_data" because all the indexes containing "Electrical" or "Heat" are converted to efficiencies:

technology-data/scripts/compile_cost_assumptions.py

Lines 2045 to 2054 in 98fa921

	# check if electric and heat efficiencies are given
	if any(["Electric" in ind for ind in efficiency.index]) and any(
	["Heat" in ind for ind in efficiency.index]
	):
	efficiency_heat = efficiency[efficiency.index.str.contains("Heat")].copy()
	efficiency_heat["parameter"] = "efficiency-heat"
	clean_df[tech_name] = pd.concat([clean_df[tech_name], efficiency_heat])
	efficiency = efficiency[efficiency.index.str.contains("Electric")].copy()
	efficiency["parameter"] = "efficiency"
	clean_df[tech_name] = pd.concat([clean_df[tech_name], efficiency])

This requires extra code specific for each technology to bypass this issue, which is not sustainable in the long term.
e.g.:

technology-data/scripts/compile_cost_assumptions.py

Lines 2056 to 2093 in 98fa921

	elif tech_name == "biomass-to-methanol":
	efficiency_heat = efficiency[
	efficiency.index.str.contains("District heat")
	].copy()
	efficiency_heat["parameter"] = "efficiency-heat"
	efficiency_heat.loc[:, years] *= 100 # in %
	clean_df[tech_name] = pd.concat([clean_df[tech_name], efficiency_heat])
	efficiency_elec = efficiency[
	efficiency.index.str.contains("Electric")
	].copy()
	efficiency_elec["parameter"] = "efficiency-electricity"
	clean_df[tech_name] = pd.concat([clean_df[tech_name], efficiency_elec])
	efficiency_meoh = efficiency[
	efficiency.index.str.contains("Methanol")
	].copy()
	efficiency_meoh["parameter"] = "efficiency"
	efficiency_meoh.loc[:, years] *= 100 # in %
	clean_df[tech_name] = pd.concat([clean_df[tech_name], efficiency_meoh])
	elif tech_name == "biochar pyrolysis":
	efficiency_biochar = efficiency[
	efficiency.index.str.contains("efficiency biochar")
	].copy()
	efficiency_biochar["parameter"] = "efficiency-biochar"
	clean_df[tech_name] = pd.concat([clean_df[tech_name], efficiency_biochar])
	efficiency_biochar_mass = efficiency[
	efficiency.index.str.contains("yield biochar")
	].copy()
	efficiency_biochar_mass["parameter"] = "yield-biochar"
	clean_df[tech_name] = pd.concat(
	[clean_df[tech_name], efficiency_biochar_mass]
	)
	efficiency_heat = efficiency[
	efficiency.index.str.contains("efficiency heat")
	].copy()
	efficiency_heat["parameter"] = "efficiency-heat"
	clean_df[tech_name] = pd.concat([clean_df[tech_name], efficiency_heat])

Error Message

there is no error in the code, but the final .csv files have the wrong data

Suggestion

modify this part of the code :

technology-data/scripts/compile_cost_assumptions.py

Lines 2045 to 2054 in 98fa921

	# check if electric and heat efficiencies are given
	if any(["Electric" in ind for ind in efficiency.index]) and any(
	["Heat" in ind for ind in efficiency.index]
	):
	efficiency_heat = efficiency[efficiency.index.str.contains("Heat")].copy()
	efficiency_heat["parameter"] = "efficiency-heat"
	clean_df[tech_name] = pd.concat([clean_df[tech_name], efficiency_heat])
	efficiency = efficiency[efficiency.index.str.contains("Electric")].copy()
	efficiency["parameter"] = "efficiency"
	clean_df[tech_name] = pd.concat([clean_df[tech_name], efficiency])

to don't label electrical and heat consumption as efficiencies.

[euronion]

Thanks for the suggestion, how would you label and report the input fields instead?

[BertoGBG]

I think we should not assume that all "electricity" and all 'heat' fields are efficiencies (generation). Although it is practical because the most of technologies are generating. it makes it very complicated to bypass it for comsuming technologies.

Maybe we cna create a dict with output labels: ['Output', 'output', 'production', etc...] and another with input labels ['Input', 'intput', 'consumption', etc...], to be associated with the strings 'electricity' and 'heat' ?

it is an idea, but it needs testing.

I just think that the issue will grow overtime with more process technologies added to the cost file.

[euronion]

I agree with the notion that this issue will increase over time. 👍

We're currently thinking about restructuring the repository, this could be something that we can also take into account while we are at it.

Would it be better to have for every process "inputs" and "outputs" and report them in the csv outputs, and in addition also calculate the efficiencies based on the inputs and outputs? E.g. for water/hydrogen electrolysis

input (electricity), 70 MWh
output (hydrogen), 33.33 MWh_LHV
output (oxygen), 8 t
efficiency (electric), 0.476