Feature: Allow for specifying IDs via the API

[nickchomey]

Feature description

The examples in the swagger UI suggest that we can specify an ID when we Create/Post a new Pipeline/Connector/Processor. eg.

However, the underlying API and orchestrators don't seem to actually allow for doing so - it just sets a new UUID.

Proposal

It would be helpful if the API and orchestrators could accept an optional ID field. If it exists, use it. If not, create a UUID.

This would both align it with the yaml config file behaviour, as well as make it vastly easier to deal with configurations since we can know exactly what the IDs are/should be rather than needing to List them all, parse the response, extract the assigned (UU)IDs, then use them.

Most importantly, for me, it would allow for aligning the ID with specific variables in the broader system - eg a server or client number.

[lovromazgon]

Hmm, where did you get that example? It's not what I'm seeing in Swagger:

We made a decision soon at the start of the project to let Conduit handle the IDs, so this was intentionally designed this way. That said, once we introduced pipeline configuration files, we had to give the user the ability to specify the ID, as that's how we identify deleted resources when reloading the file. So this is a discrepancy we are aware about and we would like to fix.

We have a bit of a bigger change in mind though, as we would also want to refactor the API paths a bit to better match the components in pipeline configuration files. This would also allow us to finally tackle the implementation of the import and export endpoints. So, rest assured, this is on our roadmap, but I can't really give you a definite timeline on when we'll get to it.

Meanwhile, if specifying the ID on the POST endpoints is of importance to you, this shouldn't be too hard of a change and I invite you to open a PR, we'll help you with reviewing and shipping it!

[nickchomey]

I'm pretty sure just from localhost:8000. Odd that we see different things! Not sure what else to say about it.

Anyway, thanks for the clarification! I'll try to make a basic PR to allow specifying ids something in the near future.