Validation data questions (validation.mif)

[Nils-Bohr]

Hi all,

Reposting this here following Jan's advice

We have been comparing MAgPIE outputs to different validation data sources in a couple of projects. A number of questions thereby emerged on the validation data reported in the validation.mif file, especially with respect to the crop production estimates based on FAO. We tried to replicate the calculations with FAO data and are running into some blockers.

1.Oilpalms: We seem to be unable to replicate the validation data results for oilpalm production reported in the validation.mif under FAO. We calculate the comparison data from raw FAO estimates using the "254|Oil, palm fruit" entry as described in the FAOitems_online.csv and applying the MagPIE dry mass conversion factor stated in the dry_wet_conversion.csv (1.0096).The value we arrive at tends to be much higher than the comparison data in the validation.mif. Our question would be what the methodology for calculating the values for oilpalms based on FAO data is in the validation.mif and what wet-mass/dry-mass conversion factor is used? Is there any step that we are missing that might result in this large divergence?

2.Cotton seed: The FAOitems_online.csv file maps the cottn_pro variable to two different FAO items: 328|Seed cotton and 329|Cottonseed. However, if we sum FAO production data for these two categories, convert it to dry mass and compare to Cotton seed production in the validation.mif file, our calculated sum is much larger. On the FAO website it seems like "Seed Cotton" is classified as a primary crop, while "Cottonseed" is classified as a processed crop ( https://www.fao.org/faostat/en/#data/QCL). Therefore, we are thinking that we should only use "Seed cotton" to compare to what's in the validation.mif file. Is that right?

Thanks a lot!

Best,
Nils

[bodirsky]

Dear Nils,
yes, we diverge in the case of oilpalm and oilseeds from FAOSTAT production stats.
The problem imerges from difficulties of marrying FAOSTAT prodstat and FBS without creating extra commodities in MAgPIE.

Oil palm fruits are processed into kernels + oil, and the kernels are again processed to kernel oil and cakes. To simplify (and given there is virtually no trade of the primary product before processing), i just used the sum of the final products (also because the nutrent density of the oil fruit bunches before processng are not easy to estiamte).

Seed cotton is processed to lint+cotton seed, and cotton seed is processed to oil+cake. Seed cotton does not appear in food balance sheets (because cant be eaten), ony cottonseed. So we use cottonseed.

[bodirsky]

calculations can be found in calcFAOmassbalance_pre

[Nils-Bohr]

Hi Benjamin,

Thanks a lot for the clarifications! This is super helpful. Really sorry, can you point me to where I can find the calcFAOmassbalance_pre file?

[caviddhen]

Hi Nils! It's here: https://github.com/pik-piam/mrcommons/blob/master/R/calcFAOmassbalance_pre.R

which feeds into: https://github.com/pik-piam/mrcommons/blob/master/R/calcFAOmassbalance.R

which feeds into: https://github.com/pik-piam/mrvalidation/blob/master/R/calcValidProduction.R

which goes into the validation.mif

[Nils-Bohr]

Hi @caviddhen @bodirsky
Thanks so much for your help! I am sorry but I have two follow up questions on the calculation of the comparison data for the production of dairy and ruminant meat. For both variables we also find quite large divergences between the data in validation.mif and the one we manually calculate.

For both we use the mapping specified in FAOitems_online.csv, summing the "ProductionItem" categories mapped to the respective GAMS names (livst_rum, livst_milk) and applying the dry-mass conversion factors (2.4570 for ruminant meat and 8.1967 for dairy). We find that our estimates for ruminant meat are systematically smaller than the data in validation.mif (between 10-50%), while our estimates for dairy are systematically larger (between 10-60%).

Our working hypothesis for dairy is that potentially not all "ProductionItems" mapped to livst_milk in the FAO_items_online.csv are actually used in compiling the comparison data for validation.mif . Some of the "ProductionItems" assigned to livst_milk in are classified as "Livestock processed" (secondary products) within FAO (e.g. "886|Butter, cow milk", "897|Milk, whole dried", "900|Whey, dry", "901|Cheese, whole cow milk" etc.). Our thinking would be that maybe those are not included in the compilation of the comparison data for validation.mif (?)

I have tried to look at the calcFAOmassbalance_pre.R script it seems like all item codes defined in the "removethem" vector are excluded from the compilation of the comparison data (e.g. "2741|Cheese","2948|Milk - Excluding Butter", "2946|Animal fats", etc.). Is this correct? Even when taking this into account, it seems like the divergence in ruminant and dairy variable remains

Our question would be if we are missing anything that might be able to explain the divergence?

Again, thanks a lot!