Skip to content

GTC-3288: Use Multiple Source URIs when Creating a Table Asset #696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 9, 2025

Conversation

gtempus
Copy link
Contributor

@gtempus gtempus commented May 8, 2025

It is possible that the first source URI passed in the creation_options could be file with only a header and no data.
Now, we send up to five source URIs when creating a schema.

Pull request checklist

Please check if your PR fulfills the following requirements:

  • Make sure you are requesting to pull a topic/feature/bugfix branch (right side). Don't request your master!
  • Make sure you are making a pull request against the develop branch (left side). Also you should start your branch off our develop.
  • Check the commit's or even all commits' message styles matches our requested structure.
  • Check your code additions will fail neither code linting checks nor unit test.

Pull request type

Please check the type of change your PR introduces:

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Documentation content changes
  • Other (please describe):

What is the current behavior?

If the first source URI has no data, no schema is created. Therefore, no table is created.
Seems reasonable. However, we expect the norm to be that source URIs have data so that a schema can be inferred.

Issue Number: GTC-3288

What is the new behavior?

Up to five source URIs are sent to create_tabular_schema.
Then, each source URI is tried.
Once a schema SQL is created, we break out of the loop and continue the job.

Does this introduce a breaking change?

  • Yes
  • No

Test Procedure in Dev Environment

Test Prep

I copied the first three csv files from the originally failing task inproduction and uploaded them to dev:

  • (just a header - no data) s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00000-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv
  • s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00011-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv
  • s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00020-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv

Dataset

Request:
PUT http://gfw-data-api-elb-shared-dev-lb-10091095.us-east-1.elb.amazonaws.com:30253/dataset/gtc_3288_multi_src_tabular_schema/v1

Response:
201 Created

Version

Request:
PUT http://gfw-data-api-elb-shared-dev-lb-10091095.us-east-1.elb.amazonaws.com:30253/dataset/gtc_3288_multi_src_tabular_schema/v1

Payload:

{
    "creation_options": {
        "delimiter": "\t",
        "source_type": "table",
        "source_driver": "text",
        "source_uri": [
            "s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00000-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv",
            "s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00011-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv",
            "s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00020-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv"
        ]
    }
}

Response:
202 Accepted

Assets

Request:
GET http://gfw-data-api-elb-shared-dev-lb-10091095.us-east-1.elb.amazonaws.com:30253/dataset/gtc_3288_multi_src_tabular_schema/v1/assets

Response:
NOTE: The asset failed because I didn't include schema type hints in the creation options. Therefore, confidence__cat was inferred as boolean. In the production creation_options, confidence__cat is explicitly set to TEXT.

{
    "data": [
        {
            "created_on": "2025-05-09T16:34:44.571686",
            "updated_on": "2025-05-09T16:39:23.293459",
            "asset_id": "2e15a153-44b9-4a45-bdab-f24137a4d733",
            "dataset": "gtc_3288_multi_src_tabular_schema",
            "version": "v1",
            "asset_type": "Database table",
            "asset_uri": "http://gfw-data-api-elb-shared-dev-lb-10091095.us-east-1.elb.amazonaws.com:30253/dataset/gtc_3288_multi_src_tabular_schema/v1/query",
            "status": "failed",
            "is_managed": true,
            "is_downloadable": true,
            "metadata": {
                "tags": null,
                "fields": [
                    {
                        "name": "iso",
                        "alias": "iso",
                        "description": null,
                        "data_type": "character varying",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "adm1",
                        "alias": "adm1",
                        "description": null,
                        "data_type": "numeric",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "adm2",
                        "alias": "adm2",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "longitude",
                        "alias": "longitude",
                        "description": null,
                        "data_type": "numeric",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "latitude",
                        "alias": "latitude",
                        "description": null,
                        "data_type": "numeric",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "alert__date",
                        "alias": "alert__date",
                        "description": null,
                        "data_type": "date",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "alert__time_utc",
                        "alias": "alert__time_utc",
                        "description": null,
                        "data_type": "numeric",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "confidence__cat",
                        "alias": "confidence__cat",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "bright_ti4__K",
                        "alias": "bright_ti4__K",
                        "description": null,
                        "data_type": "numeric",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "bright_ti5__K",
                        "alias": "bright_ti5__K",
                        "description": null,
                        "data_type": "numeric",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "frp__MW",
                        "alias": "frp__MW",
                        "description": null,
                        "data_type": "numeric",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "alert__count",
                        "alias": "alert__count",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "wdpa_protected_areas__iucn_cat",
                        "alias": "wdpa_protected_areas__iucn_cat",
                        "description": null,
                        "data_type": "character varying",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "wdpa_protected_area__iucn_cat",
                        "alias": "wdpa_protected_area__iucn_cat",
                        "description": null,
                        "data_type": "character varying",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "umd_tree_cover_density_2000__threshold",
                        "alias": "umd_tree_cover_density_2000__threshold",
                        "description": null,
                        "data_type": "numeric",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__umd_regional_primary_forest_2001",
                        "alias": "is__umd_regional_primary_forest_2001",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__birdlife_alliance_for_zero_extinction_sites",
                        "alias": "is__birdlife_alliance_for_zero_extinction_sites",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__birdlife_key_biodiversity_areas",
                        "alias": "is__birdlife_key_biodiversity_areas",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__landmark_indigenous_and_community_lands",
                        "alias": "is__landmark_indigenous_and_community_lands",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "gfw_planted_forests__type",
                        "alias": "gfw_planted_forests__type",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__gfw_mining_concessions",
                        "alias": "is__gfw_mining_concessions",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__gfw_managed_forests",
                        "alias": "is__gfw_managed_forests",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "rspo_oil_palm__certification_status",
                        "alias": "rspo_oil_palm__certification_status",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__gfw_wood_fiber",
                        "alias": "is__gfw_wood_fiber",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__gfw_peatlands",
                        "alias": "is__gfw_peatlands",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__idn_forest_moratorium",
                        "alias": "is__idn_forest_moratorium",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__gfw_oil_palm",
                        "alias": "is__gfw_oil_palm",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "idn_forest_area__class",
                        "alias": "idn_forest_area__class",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "per_forest_concessions__type",
                        "alias": "per_forest_concessions__type",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__gfw_oil_gas",
                        "alias": "is__gfw_oil_gas",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__gmw_global_mangrove_extent_2020",
                        "alias": "is__gmw_global_mangrove_extent_2020",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__ifl_intact_forest_landscapes_2016",
                        "alias": "is__ifl_intact_forest_landscapes_2016",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "ibge_bra_biomes__name",
                        "alias": "ibge_bra_biomes__name",
                        "description": null,
                        "data_type": "character varying",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "sbtn_natural_forests__class",
                        "alias": "sbtn_natural_forests__class",
                        "description": null,
                        "data_type": "character varying",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "umd_tree_cover_density__threshold",
                        "alias": "umd_tree_cover_density__threshold",
                        "description": null,
                        "data_type": "numeric",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__birdlife_alliance_for_zero_extinction_site",
                        "alias": "is__birdlife_alliance_for_zero_extinction_site",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__birdlife_key_biodiversity_area",
                        "alias": "is__birdlife_key_biodiversity_area",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__landmark_land_right",
                        "alias": "is__landmark_land_right",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "gfw_plantation__type",
                        "alias": "gfw_plantation__type",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__gfw_mining",
                        "alias": "is__gfw_mining",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__gfw_managed_forest",
                        "alias": "is__gfw_managed_forest",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__peatland",
                        "alias": "is__peatland",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "idn_forest_area__type",
                        "alias": "idn_forest_area__type",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "per_forest_concession__type",
                        "alias": "per_forest_concession__type",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__gmw_mangroves_2020",
                        "alias": "is__gmw_mangroves_2020",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "is__ifl_intact_forest_landscape_2016",
                        "alias": "is__ifl_intact_forest_landscape_2016",
                        "description": null,
                        "data_type": "boolean",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    },
                    {
                        "name": "bra_biome__name",
                        "alias": "bra_biome__name",
                        "description": null,
                        "data_type": "character varying",
                        "unit": null,
                        "is_feature_info": true,
                        "is_filter": true
                    }
                ],
                "id": "5f53bd26-83a4-4998-bffb-0f42a0cf64ca"
            }
        }
    ],
    "status": "success"
}

Tasks

Request:
GET http://gfw-data-api-elb-shared-dev-lb-10091095.us-east-1.elb.amazonaws.com:30253/asset/2e15a153-44b9-4a45-bdab-f24137a4d733/tasks

Response:
NOTE: create_table succeeds. The only reason the last tasks fails is because of the incorrect schema data type mentioned above.

{
    "data": [
        {
            "created_on": "2025-05-09T16:34:44.843729",
            "updated_on": "2025-05-09T16:38:44.830731",
            "task_id": "f1b4ae74-64b3-419d-b1a2-94ee459a5b09",
            "asset_id": "2e15a153-44b9-4a45-bdab-f24137a4d733",
            "change_log": [
                {
                    "date_time": "2025-05-09T16:34:44.838069",
                    "status": "pending",
                    "message": "Scheduled job create_table",
                    "detail": "Job ID: f1b4ae74-64b3-419d-b1a2-94ee459a5b09"
                },
                {
                    "date_time": "2025-05-09T16:38:44",
                    "status": "success",
                    "message": "Successfully ran command [ create_tabular_schema.sh -d gtc_3288_multi_src_tabular_schema -v v1 -D \\t -s s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00000-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv -s s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00011-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv -s s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00020-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv ]",
                    "detail": ""
                }
            ]
        },
        {
            "created_on": "2025-05-09T16:34:45.009561",
            "updated_on": "2025-05-09T16:38:52.628818",
            "task_id": "9e4fafa8-f5f2-4377-a860-acc4aadae92d",
            "asset_id": "2e15a153-44b9-4a45-bdab-f24137a4d733",
            "change_log": [
                {
                    "date_time": "2025-05-09T16:34:45.007318",
                    "status": "pending",
                    "message": "Scheduled job load_tabular_data_0",
                    "detail": "Job ID: 9e4fafa8-f5f2-4377-a860-acc4aadae92d, parents: [{'jobId': 'f1b4ae74-64b3-419d-b1a2-94ee459a5b09', 'type': 'SEQUENTIAL'}]"
                },
                {
                    "date_time": "2025-05-09T16:38:52",
                    "status": "success",
                    "message": "Successfully ran command [ load_tabular_data.sh -d gtc_3288_multi_src_tabular_schema -v v1 -D \\t -s s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00000-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv ]",
                    "detail": ""
                }
            ]
        },
        {
            "created_on": "2025-05-09T16:34:45.168805",
            "updated_on": "2025-05-09T16:38:52.389750",
            "task_id": "005e798e-0e4d-45ef-875c-24b06f8fb591",
            "asset_id": "2e15a153-44b9-4a45-bdab-f24137a4d733",
            "change_log": [
                {
                    "date_time": "2025-05-09T16:34:45.167153",
                    "status": "pending",
                    "message": "Scheduled job load_tabular_data_1",
                    "detail": "Job ID: 005e798e-0e4d-45ef-875c-24b06f8fb591, parents: [{'jobId': 'f1b4ae74-64b3-419d-b1a2-94ee459a5b09', 'type': 'SEQUENTIAL'}]"
                },
                {
                    "date_time": "2025-05-09T16:38:52",
                    "status": "success",
                    "message": "Successfully ran command [ load_tabular_data.sh -d gtc_3288_multi_src_tabular_schema -v v1 -D \\t -s s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00011-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv ]",
                    "detail": ""
                }
            ]
        },
        {
            "created_on": "2025-05-09T16:34:45.337974",
            "updated_on": "2025-05-09T16:39:23.281283",
            "task_id": "08e86877-7fed-4e18-8b27-87630c441174",
            "asset_id": "2e15a153-44b9-4a45-bdab-f24137a4d733",
            "change_log": [
                {
                    "date_time": "2025-05-09T16:34:45.334511",
                    "status": "pending",
                    "message": "Scheduled job load_tabular_data_2",
                    "detail": "Job ID: 08e86877-7fed-4e18-8b27-87630c441174, parents: [{'jobId': 'f1b4ae74-64b3-419d-b1a2-94ee459a5b09', 'type': 'SEQUENTIAL'}]"
                },
                {
                    "date_time": "2025-05-09T16:39:23",
                    "status": "failed",
                    "message": "Command [ load_tabular_data.sh -d gtc_3288_multi_src_tabular_schema -v v1 -D \\t -s s3://gfw-pipelines-dev/geotrellis/results/v20250424/gadm/v20250424/firealerts_viirs_20250424_1928/all/part-00020-2ae56bce-e3c6-4eee-9699-3a3da509ef31-c000.csv ] encountered errors",
                    "detail": "/tmp/08e86877-7fed-4e18-8b27-87630c441174 /\nNOTICE:  column \"geom\" of relation \"temp_5e6a8d9a8b0e4ebbbacd710f1e33163e\" does not exist, skipping\nBEGIN\nNOTICE:  column \"geom_wm\" of relation \"temp_5e6a8d9a8b0e4ebbbacd710f1e33163e\" does not exist, skipping\nCREATE TABLE\nALTER TABLE\nALTER TABLE\nERROR:  invalid input syntax for type boolean: \"h\"\nCONTEXT:  COPY temp_5e6a8d9a8b0e4ebbbacd710f1e33163e, line 3, column confidence__cat: \"h\"\n"
                }
            ]
        }
    ],
    "status": "success"
}

AWS Batch Log

batch_load_schema

@codecov-commenter
Copy link

codecov-commenter commented May 8, 2025

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.55%. Comparing base (932805d) to head (6afc74f).

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #696      +/-   ##
===========================================
- Coverage    76.56%   76.55%   -0.02%     
===========================================
  Files          143      143              
  Lines         6704     6700       -4     
===========================================
- Hits          5133     5129       -4     
  Misses        1571     1571              
Flag Coverage Δ
unittests 76.55% <100.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment on lines +6 to +28
#### GitHub Container Registry (GHCR) Access Setup

To authenticate Docker with GitHub Container Registry (`ghcr.io`) for pulling/pushing images, follow these steps:

##### 1. Create a GitHub Personal Access Token (PAT)

1. Navigate to: GitHub → Settings → Developer Settings → Personal Access Tokens → Tokens (Classic)
2. Click **Generate new token (Classic)**
3. Configure:
- **Note**: `docker-ghcr-access` (descriptive name)
- **Expiration**: Set duration (or "No expiration" for CI/CD)
- **Scopes**:
- `read:packages` (required for pull)
- `write:packages` (required for push)
4. Click **Generate token** and copy the token value

##### 2. Authenticate with Docker

```bash
echo "YOUR_GHCR_TOKEN" | docker login ghcr.io -u GITHUB_USERNAME --password-stdin
```

#### Proceed with Setup
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the docs update!

Copy link
Member

@dmannarino dmannarino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!
Testing these Batch scripts is difficult without doing an end-to-end test. Have you tested in dev though?

@gtempus
Copy link
Contributor Author

gtempus commented May 9, 2025

I'm testing it in dev now, @dmannarino 👍

@gtempus
Copy link
Contributor Author

gtempus commented May 9, 2025

Success! Please check out my dev test procedure in the description above, @dmannarino ☝️

gtempus added 4 commits May 9, 2025 15:29
This is needed for docker images that are stored in ghcr.io
This is needed to ensure a schema can be created. It can happen that
the first source URI has only a header and no data.
@gtempus gtempus force-pushed the gtc-3288-use-mult-src-uris branch from c0bb341 to 6afc74f Compare May 9, 2025 20:29
@gtempus gtempus merged commit 31887df into develop May 9, 2025
2 checks passed
@gtempus gtempus deleted the gtc-3288-use-mult-src-uris branch May 9, 2025 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants