Skip to content

Commit

Permalink
Merge branch 'develop' of github.com:rudderlabs/rudder-transformer in…
Browse files Browse the repository at this point in the history
…to chore.test-migration-TS
  • Loading branch information
utsabc committed Nov 27, 2024
2 parents 0a1fbeb + ed9fe55 commit 15e8cb3
Show file tree
Hide file tree
Showing 132 changed files with 8,668 additions and 3,794 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/dt-test-and-report-code-coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ jobs:
- name: SonarCloud Scan
if: always()
uses: SonarSource/sonarcloud-github-action@v3.0.0
uses: SonarSource/sonarcloud-github-action@v3.1.0
env:
GITHUB_TOKEN: ${{ secrets.PAT }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
Expand Down
12 changes: 11 additions & 1 deletion .github/workflows/verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,19 @@ jobs:
with:
token: ${{ secrets.GITHUB_TOKEN }}

- name: Filter JS/TS Files
run: |
changed_files=$(echo "${{ steps.files.outputs.added_modified }}" | tr ' ' '\n' | grep -E '\.(js|ts|jsx|tsx)$' || true)
if [ -z "$changed_files" ]; then
echo "No JS/TS files to format or lint."
exit 0
fi
- name: Run format Checks
run: |
npx prettier ${{steps.files.outputs.added_modified}} --write
if [ -s changed_files.txt ]; then
npx prettier --write $(cat changed_files.txt)
fi
- run: git diff --exit-code

Expand Down
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ dist

# Stores VSCode versions used for testing VSCode extensions
.vscode-test
.vscode

# yarn v2
.yarn/cache
Expand All @@ -133,9 +134,9 @@ dist
# Others
**/.DS_Store
.dccache

.python-version
.idea

# component test report
test_reports/
temp/
temp/
58 changes: 58 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,64 @@

All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.

### [1.85.1](https://github.com/rudderlabs/rudder-transformer/compare/v1.85.0...v1.85.1) (2024-11-21)


### Bug Fixes

* braze subscription batch size ([#3897](https://github.com/rudderlabs/rudder-transformer/issues/3897)) ([ca71a31](https://github.com/rudderlabs/rudder-transformer/commit/ca71a318e4d8d098116fe539964b699254f58617))
* stringifying session ID for airship ([#3896](https://github.com/rudderlabs/rudder-transformer/issues/3896)) ([bb0b9dc](https://github.com/rudderlabs/rudder-transformer/commit/bb0b9dc1e5a56e8141c6cb56e89835ba61ee7761))

## [1.85.0](https://github.com/rudderlabs/rudder-transformer/compare/v1.84.0...v1.85.0) (2024-11-18)


### Features

* added support to eu/us2 datacenter for gainsight px destination ([#3871](https://github.com/rudderlabs/rudder-transformer/issues/3871)) ([12ac3de](https://github.com/rudderlabs/rudder-transformer/commit/12ac3de6e7cc91a6cd52c33bc342f74bbaa8a631))
* iterable EUDC ([#3828](https://github.com/rudderlabs/rudder-transformer/issues/3828)) ([1c134f8](https://github.com/rudderlabs/rudder-transformer/commit/1c134f84601aaea78581078137cb9955de576f9e))
* iterable EUDC deleteUsers ([#3881](https://github.com/rudderlabs/rudder-transformer/issues/3881)) ([becb4fa](https://github.com/rudderlabs/rudder-transformer/commit/becb4fa54e9093ed69779f54c36864cb9d28d321))
* moved userSchema to connection config in GARL vdmv2 ([#3870](https://github.com/rudderlabs/rudder-transformer/issues/3870)) ([640a11e](https://github.com/rudderlabs/rudder-transformer/commit/640a11eb3dca5735fed3ad9ad5bd058974b069d6))
* now getting consent related fields from connection config from retl for GARL ([#3877](https://github.com/rudderlabs/rudder-transformer/issues/3877)) ([51bbc02](https://github.com/rudderlabs/rudder-transformer/commit/51bbc02d5b00ce1b8fe8c91b4a7041e926bae9bd))
* onboard linkedin audience destination ([#3857](https://github.com/rudderlabs/rudder-transformer/issues/3857)) ([f3ff409](https://github.com/rudderlabs/rudder-transformer/commit/f3ff4092d455508dd3354ffb22d345fa97f4d1f2))
* onboarding intercom v2 retl support ([#3843](https://github.com/rudderlabs/rudder-transformer/issues/3843)) ([3d7db73](https://github.com/rudderlabs/rudder-transformer/commit/3d7db7366e30df31c37cc473e344da82b49ed885))
* sources v2 spec support along with adapters ([04c0694](https://github.com/rudderlabs/rudder-transformer/commit/04c069486bdd3c101906fa6c621e983090fcab25))
* sources v2 spec support along with adapters ([#3810](https://github.com/rudderlabs/rudder-transformer/issues/3810)) ([c51cfbb](https://github.com/rudderlabs/rudder-transformer/commit/c51cfbb4664a8531dce23b2d06fe40997f95697e))
* update pinterest_tag single product events with new mapping ([#3858](https://github.com/rudderlabs/rudder-transformer/issues/3858)) ([8520278](https://github.com/rudderlabs/rudder-transformer/commit/85202781de3464bd46fe910159d2b143cd4209e8))


### Bug Fixes

* adding logger for undefined source event ([#3879](https://github.com/rudderlabs/rudder-transformer/issues/3879)) ([79e5979](https://github.com/rudderlabs/rudder-transformer/commit/79e597907eee126b4187e4534b2aa2253d1431da))
* adding uuid transformation for airship ([#3884](https://github.com/rudderlabs/rudder-transformer/issues/3884)) ([a80f874](https://github.com/rudderlabs/rudder-transformer/commit/a80f87486dc93b423e4fe6efbee6f4cb8330ba02))
* handling invalid timestamp for adjust source ([#3866](https://github.com/rudderlabs/rudder-transformer/issues/3866)) ([d57f48e](https://github.com/rudderlabs/rudder-transformer/commit/d57f48e989d18d469bea0de94293bc685300945b))
* revert gaec changes ([#3885](https://github.com/rudderlabs/rudder-transformer/issues/3885)) ([0aeaa39](https://github.com/rudderlabs/rudder-transformer/commit/0aeaa391b025fc68de6e3d63a6721f067c5be318))

## [1.84.0](https://github.com/rudderlabs/rudder-transformer/compare/v1.83.2...v1.84.0) (2024-11-11)


### Features

* gaec migration ([#3855](https://github.com/rudderlabs/rudder-transformer/issues/3855)) ([7a26459](https://github.com/rudderlabs/rudder-transformer/commit/7a264590b61d3d31d5559c8ac53fd572b40cddec))
* **GARL:** support vdm next for GARL ([#3835](https://github.com/rudderlabs/rudder-transformer/issues/3835)) ([f4b38eb](https://github.com/rudderlabs/rudder-transformer/commit/f4b38eba3ca8dff602915853fda5cd7ca284bba3))
* update on twitter_ads ([#3856](https://github.com/rudderlabs/rudder-transformer/issues/3856)) ([adc8976](https://github.com/rudderlabs/rudder-transformer/commit/adc8976990fa98c5b874472aee180cadfabb0088))


### Bug Fixes

* adding throttled status code for server unavailable error in salesforce ([#3862](https://github.com/rudderlabs/rudder-transformer/issues/3862)) ([fa93f09](https://github.com/rudderlabs/rudder-transformer/commit/fa93f0917d4f75fc197a6ea4c574d37faa0a3f77))
* linkedin ads conversionValue object as well as price is not mandatory ([#3860](https://github.com/rudderlabs/rudder-transformer/issues/3860)) ([bfd7edc](https://github.com/rudderlabs/rudder-transformer/commit/bfd7edc5608c60a39644a1d4ad6e15e5dbcbea0e))
* marketo bulk upload handle special chars ([#3859](https://github.com/rudderlabs/rudder-transformer/issues/3859)) ([f959a7d](https://github.com/rudderlabs/rudder-transformer/commit/f959a7dc2487dc7e36377f5f2e265014f692f476))
* unsafe property getting set via set value library ([#3853](https://github.com/rudderlabs/rudder-transformer/issues/3853)) ([80d7b41](https://github.com/rudderlabs/rudder-transformer/commit/80d7b417be7a0e459de49caca25aba43ffdba337))

### [1.83.2](https://github.com/rudderlabs/rudder-transformer/compare/v1.83.1...v1.83.2) (2024-11-05)


### Bug Fixes

* update gaec destination with config validation ([#3847](https://github.com/rudderlabs/rudder-transformer/issues/3847)) ([e5c5b0a](https://github.com/rudderlabs/rudder-transformer/commit/e5c5b0a28070ff5ca89a274c3998b96780139149))

### [1.83.1](https://github.com/rudderlabs/rudder-transformer/compare/v1.83.0...v1.83.1) (2024-11-01)

## [1.83.0](https://github.com/rudderlabs/rudder-transformer/compare/v1.82.2...v1.83.0) (2024-10-25)


Expand Down
29 changes: 17 additions & 12 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "rudder-transformer",
"version": "1.83.0",
"version": "1.85.1",
"description": "",
"homepage": "https://github.com/rudderlabs/rudder-transformer#readme",
"bugs": {
Expand Down Expand Up @@ -64,7 +64,7 @@
"@koa/router": "^12.0.0",
"@ndhoule/extend": "^2.0.0",
"@pyroscope/nodejs": "^0.2.9",
"@rudderstack/integrations-lib": "^0.2.10",
"@rudderstack/integrations-lib": "^0.2.13",
"@rudderstack/json-template-engine": "^0.18.0",
"@rudderstack/workflow-engine": "^0.8.13",
"@shopify/jest-koa-mocks": "^5.1.1",
Expand Down Expand Up @@ -95,7 +95,7 @@
"koa": "^2.15.3",
"koa-bodyparser": "^4.4.0",
"koa2-swagger-ui": "^5.7.0",
"libphonenumber-js": "^1.11.1",
"libphonenumber-js": "^1.11.12",
"lodash": "^4.17.21",
"match-json": "^1.3.5",
"md5": "^2.3.0",
Expand All @@ -118,7 +118,7 @@
"truncate-utf8-bytes": "^1.0.2",
"ua-parser-js": "^1.0.37",
"unset-value": "^2.0.1",
"uuid": "^9.0.0",
"uuid": "^9.0.1",
"valid-url": "^1.0.9",
"zod": "^3.22.4"
},
Expand Down Expand Up @@ -152,7 +152,7 @@
"eslint-plugin-unicorn": "^46.0.1",
"glob": "^10.3.3",
"http-terminator": "^3.2.0",
"husky": "^9.1.1",
"husky": "^9.1.6",
"jest": "^29.5.0",
"jest-sonar": "^0.2.16",
"jest-when": "^3.5.2",
Expand Down
6 changes: 2 additions & 4 deletions src/cdk/v2/destinations/linkedin_ads/utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ function checkIfPricePresent(properties) {
}

const calculateConversionObject = (message) => {
const { properties, event } = message;
const { properties } = message;

const calculateAmount = () => {
if (properties?.products && properties.products.length > 0) {
Expand All @@ -107,9 +107,7 @@ const calculateConversionObject = (message) => {
};
return conversionObject;
}
throw new InstrumentationError(
`[LinkedIn Conversion API]: Cannot map price for event ${event}. Aborting`,
);
return null;
};

const deduceConversionRules = (trackEventName, destConfig) => {
Expand Down
9 changes: 5 additions & 4 deletions src/cdk/v2/destinations/linkedin_ads/utils.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,13 @@ describe('formatEmail', () => {
});

describe('calculateConversionObject', () => {
// Returns a conversion object with currency code 'USD' and amount 0 when message properties are empty
it('should throw instrumentation error when message properties are empty', () => {
// Returns empty object when message properties are empty
it('should return empty object when message properties are empty', () => {
const message = { properties: {} };
expect(() => {
fetchUserIds(calculateConversionObject(message));
}).toThrow(InstrumentationError);
const conversionObject = calculateConversionObject(message);
expect(conversionObject).toEqual({});
});
});

// Returns a conversion object with currency code 'USD' and amount 0 when message properties price is defined but quantity is 0
Expand Down
10 changes: 10 additions & 0 deletions src/cdk/v2/destinations/linkedin_audience/config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
export const SUPPORTED_EVENT_TYPE = 'record';
export const ACTION_TYPES = ['insert', 'delete'];
export const BASE_ENDPOINT = 'https://api.linkedin.com/rest';
export const USER_ENDPOINT = '/dmpSegments/audienceId/users';
export const COMPANY_ENDPOINT = '/dmpSegments/audienceId/companies';
export const FIELD_MAP = {
sha256Email: 'SHA256_EMAIL',
sha512Email: 'SHA512_EMAIL',
googleAid: 'GOOGLE_AID',
};
89 changes: 89 additions & 0 deletions src/cdk/v2/destinations/linkedin_audience/procWorkflow.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
bindings:
- path: ./config
exportAll: true
- path: ./utils
exportAll: true
- name: defaultRequestConfig
path: ../../../../v0/util

steps:
- name: validateInput
description: Validate input, if all the required fields are available or not.
template: |
const config = .connection.config.destination;
const secret = .metadata.secret;
let messageType = .message.type;
$.assertConfig(config.audienceId, "Audience Id is not present. Aborting");
$.assertConfig(secret.accessToken, "Access Token is not present. Aborting");
$.assertConfig(config.audienceType, "audienceType is not present. Aborting");
$.assert(messageType, "Message Type is not present. Aborting message.");
$.assert(messageType.toLowerCase() === $.SUPPORTED_EVENT_TYPE, `Event type ${.message.type.toLowerCase()} is not supported. Aborting message.`);
$.assert(.message.fields, "`fields` is not present. Aborting message.");
$.assert(.message.identifiers, "`identifiers` is not present inside properties. Aborting message.");
$.assert($.containsAll([.message.action], $.ACTION_TYPES), "Unsupported action type. Aborting message.")
- name: getConfigs
description: This step fetches the configs from different places and combines them.
template: |
const config = .connection.config.destination;
{
audienceType: config.audienceType,
audienceId: config.audienceId,
accessToken: .metadata.secret.accessToken,
isHashRequired: config.isHashRequired,
}
- name: prepareUserTypeBasePayload
condition: $.outputs.getConfigs.audienceType === 'user'
steps:
- name: prepareUserIds
description: Prepare user ids for user audience type
template: |
const identifiers = $.outputs.getConfigs.isHashRequired === true ?
$.hashIdentifiers(.message.identifiers) :
.message.identifiers;
$.prepareUserIds(identifiers)
- name: preparePayload
description: Prepare base payload for user audiences
template: |
const payload = {
'elements': [
{
'action': $.generateActionType(.message.action),
'userIds': $.outputs.prepareUserTypeBasePayload.prepareUserIds,
....message.fields
}
]
}
payload;
- name: prepareCompanyTypeBasePayload
description: Prepare base payload for company audiences
condition: $.outputs.getConfigs.audienceType === 'company'
template: |
const payload = {
'elements': [
{
'action': $.generateActionType(.message.action),
....message.identifiers,
....message.fields
}
]
}
payload;
- name: buildResponseForProcessTransformation
description: build response depending upon batch size
template: |
const response = $.defaultRequestConfig();
response.body.JSON = {...$.outputs.prepareUserTypeBasePayload, ...$.outputs.prepareCompanyTypeBasePayload};
response.endpoint = $.generateEndpoint($.outputs.getConfigs.audienceType, $.outputs.getConfigs.audienceId);
response.headers = {
"Authorization": "Bearer " + $.outputs.getConfigs.accessToken,
"Content-Type": "application/json",
"X-RestLi-Method": "BATCH_CREATE",
"X-Restli-Protocol-Version": "2.0.0",
"LinkedIn-Version": "202409"
};
response;
Loading

0 comments on commit 15e8cb3

Please sign in to comment.