allow encoding of additional data into BUFR when using CSV-to-BUFR and BUFR-to-BUFR

[maaikelimper]

@efucile requested the encoding of additional data into BUFR when using CSV-to-BUFR and BUFR-to-BUFR using station metadata

[maaikelimper]

@efucile please specify your requirements

[efucile]

csv or BUFR are usually coming from external systems that are sometimes difficult to modify or access for staff of the met service. We need to provide a way to change some of the elements in the final BUFR at wis2box level. The idea is to provide a simple way to fix errors in lat, Lon, heights, ... in the wis2box.

[maaikelimper]

Users need to have an OPTIONAL way to change BUFR using a separate post-processing-filter defining the BUFR-overwrite-settings, based on eccodes-key

Each dataset should have be able to have their own post-processing-filter

csv2bufr-template does something similar already, we need to review the use of the AWS-template to avoid confusion with metadata defined in multiple places

[maaikelimper]

Who can provide examples for initial testing/development to enable implementing this feature? Answer: @alhelguera

[maaikelimper]

Enrico is asking for:
When a BUFR (or CSV) enter wis2box-incoming, the transformation csv2bufr or bufr2bufr should check if after the transformation is done if certain fields should be changed.
proposed logic:
lookup wigos-id in configuration and check if configuration requested has replacement-condition for this station, if yes apply replacement-condition.

This needs to be implemented in the wis2box-api, where we have eccodes
This requires a place to store the replacement-condition, use a new collection in wis2box-api

work required:

• update wis2box-management to add new collection, and CLI to write to this collection to enable automated tests

• update wis2box-api to read/write new collection

• update plugins in wis2box-api to apply replacement-conditions

• update wis2box-webapp to input replacement-conditions