Skip to content

Data Extension Extract Failing #91

@realityryan

Description

@realityryan

2023-01-05 14:49:29,713Z tap - INFO Making RETRIEVE call to 'DataExtension' endpoint with filters '{'Property': 'CustomerKey', 'Value': 'Account_Salesforce_1', 'SimpleOperator': 'equals'}'.
2023-01-05 14:49:29,713Z target - INFO Serializing batch with 7 messages for table data_extension.Complaint
2023-01-05 14:49:29,713Z target - INFO Sending batch of 4130 bytes to https://api.stitchdata.com/v2/import/batch
2023-01-05 14:49:29,715Z target - INFO replicated 7 records from "data_extension.Complaint" endpoint
2023-01-05 14:49:29,796Z tap - INFO Got 1 results from DataExtension endpoint.
2023-01-05 14:50:38,698Z tap - INFO Got 2500 results from DataExtensionObject endpoint.
2023-01-05 14:50:39,921Z target - INFO Serializing batch with 1260 messages for table data_extension.Account_Salesforce_1
2023-01-05 14:50:39,960Z target - INFO Sending batch of 3994190 bytes to https://api.stitchdata.com/v2/import/batch
2023-01-05 14:50:39,965Z target - INFO replicated 1260 records from "data_extension.Account_Salesforce_1" endpoint
2023-01-05 14:50:41,089Z tap - INFO Getting more results from 'DataExtensionObject' endpoint
2023-01-05 14:50:41,090Z target - INFO Serializing batch with 1240 messages for table data_extension.Account_Salesforce_1
2023-01-05 14:50:41,134Z target - INFO Sending batch of 3996107 bytes to https://api.stitchdata.com/v2/import/batch
2023-01-05 14:50:41,138Z target - INFO replicated 1240 records from "data_extension.Account_Salesforce_1" endpoint
2023-01-05 14:50:51,805Z target - INFO Requests complete, stopping loop
2023-01-05 14:50:51,847Z main - INFO Target exited normally with status 0
2023-01-05 14:50:51,851Z main - INFO No tunnel subprocess to tear down
2023-01-05 14:50:51,851Z main - INFO Exit status is: Discovery succeeded. Tap failed with code -9. Target succeeded.

Above is the error message I receive. The errors seem to only occur on larger objects with hundreds of thousands of records. The above fails after successfully importing about 55,000 records. I'm not sure what a failure with code -9 means but as the error isn't being generated from the Python (as far as I can tell), I believe it has to do with a limitation on the API endpoint.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions