-
Notifications
You must be signed in to change notification settings - Fork 5
Open
Labels
featureNew feature or requestNew feature or request
Description
Feature description
We already had a PR for this but it had many problems {all these are just things specific for acceptance tests to work on S3, they are not S3 related bugs}:
#44
-
The acceptance tests expect that the destination writes a record, and the source will retrieve the same record containing the same key and payload.. but this is not the case here because the key would be a random number.. and the payload would be the whole record {S3 destination has a batch size.. so the file name at the destination bucket can't have the same key.. so it's a random number.json}
-
the
createdAttimestamp in the record is an int timestamp: ex: 1655131862580631000, and when we try to parse the file body intosdk.Recordwe get this error:
"CreatedAt":1655131862580631000
err: parsing time "1655131862580631000" as "\"2006-01-02T15:04:05Z07:00\"": cannot parse "1655131862580631000" as "\""
- if the payload has any special characters, a back dash will be added to the string, which will make it not match, ex:
want: "Payload":{"'8B2\u0026U
got : "Payload":"{\"'8B2\\u0026U
- the
isEqualRecordsmethod in acceptance tests assumes that payloads are unique, but in our case, the payload is the whole record.
hariso
Metadata
Metadata
Assignees
Labels
featureNew feature or requestNew feature or request
Type
Projects
Status
No status