Allow schema-free import

[geoextra]

It would be useful to allow the user to also include columns which are not in the template of the importer. Especially if the use case is a generic data set or table upload.

This would have a positive side effect: The template of an importer can act as a data annotation tool rather than just a mapping. If there is a non-required property "address" in my template, the user can use it to "annotate" address columns so the underlying application can make sense of it.

An option to enable this behavior on an importer-level would be great. I can put some work into it, once I have more time.

[ciminelli]

Hey @geoextra thanks for creating the issue. We're planning on supporting this with a new importer setting that would not have any column mapping, or have the SDK define the column mapping so it can be updated programmatically by your application.

This would have a positive side effect: The template of an importer can act as a data annotation tool rather than just a mapping.

Can you expand on this? I'm not sure I understand it fully.

[geoextra]

Fantastic, great to hear!

I indeed might have formulated it in a convoluted way. For instance in my current use case it would be great to have schema-free user upload as the baseline but also let the user introduce some structure to the just-uploaded data. If there is for instance a column contain unstructured addresses, the user could map it to an address property defined in my template (like currently). An application could then still benefit from knowing that there exists a certain column in the data and could, for instance, geocode these addresses to coordinates.

On a second look this would of course imply that many-to-one or many-to-many relationship between CSV attributes and template proporties would be possible ...

[ciminelli]

@geoextra gotcha, I think I understand better now! Maybe for your use case a solve could be having an import schema defined on your end (either in the admin app or in the SDK) but still allow for users to add columns that aren't in your schema? That way they could map the address column but also add other columns. Does that sound like it could work?

[geoextra]

Exactly, that would be a solution!

[geoextra]

I think this issue has been at least partially resolved.
Just my "mapping approach" with uploading schemaless data and then using the Tableflow schema columns as annotations is not supported yet. However, this would be probably better suited for another issue.

[geoextra]

The schemafree option seemed to be removed during the migration. Is there a technical reason for it? I would reopen the issue for the sake of completion.
Related to #182.

[ciminelli]

Thanks for re-opening @geoextra, we'll work on adding back support for this!