Description
Feature description
This feature request originated here ConduitIO/conduit-connector-postgres#168.
Some resources (databases) support a truncate operation to delete all records from a collection. In this case, we don't get the full list of records that were deleted, but rather just the information that the collection was truncated. With the current set of operations (snapshot
, create
, update
, delete
) it's impossible to represent this action. We would have to spawn a delete
record for every row that was truncated, but the connector doesn't have the information about which rows existed before. Even without this limitation, this approach wouldn't really be performant on big collections.
One solution is to create a truncate
operation that has to contain the metadata field for which collection is affected. The destination connector could apply this operation as best it can.
We need to figure out what side effects we could experience if we introduce such an operation. For instance, processors working on single records would likely fail to process a truncate
operation, since the payload and key would likely be empty.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status