Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(web-api): add request interceptor and HTTP adapter config to WebClient [ISSUE-2073] #2076

Merged
merged 10 commits into from
Oct 16, 2024
Merged
59 changes: 59 additions & 0 deletions docs/content/packages/web-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -662,6 +662,65 @@ const web = new WebClient(token, { agent: proxy });

---

### Modify outgoing requests with a request interceptor

The client allows you to customize a request
[`interceptor`](https://axios-http.com/docs/interceptors) to modify outgoing requests.
Using this option allows you to modify outgoing requests to conform to the requirements of a proxy, which is a common requirement in many corporate settings.

For example you may want to wrap the original request information within a POST request:

```javascript
const { WebClient } = require('@slack/web-api');

const token = process.env.SLACK_TOKEN;

const webClient = new WebClient(token, {
requestInterceptor: (config) => {
config.headers['Content-Type'] = 'application/json';

config.data = {
method: config.method,
base_url: config.baseURL,
path: config.url,
body: config.data ?? {},
query: config.params ?? {},
headers: structuredClone(config.headers),
test: 'static-body-value',
};

return config;
}
});
```

---

### Using a pre-configured http client to handle outgoing requests

The client allows you to specify an
[`adapter`](https://github.com/axios/axios/blob/v1.x/README.md?plain=1#L586) to handle outgoing requests.
Using this option allows you to use a pre-configured http client, which is a common requirement in many corporate settings.

For example you may want to use an HTTP client which is already configured with logging capabilities, desired timeouts, etc.

```javascript
const { WebClient } = require('@slack/web-api');
const { CustomHttpClient } = require('@company/http-client')

const token = process.env.SLACK_TOKEN;

const customClient = CustomHttpClient();

const webClient = new WebClient(token, {
adapter: (config: RequestConfig) => {
return customClient.request(config);
}
});
```

---

### Rate limits

When your app calls API methods too frequently, Slack will politely ask (by returning an error) the app to slow down,
Expand Down
122 changes: 121 additions & 1 deletion packages/web-api/src/WebClient.spec.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
import fs from 'node:fs';
import axios, { type InternalAxiosRequestConfig } from 'axios';
import { assert, expect } from 'chai';
import nock from 'nock';
import sinon from 'sinon';
import { type WebAPICallResult, WebClient, WebClientEvent, buildThreadTsWarningMessage } from './WebClient';
import {
type RequestConfig,
type WebAPICallResult,
WebClient,
WebClientEvent,
buildThreadTsWarningMessage,
} from './WebClient';
import { ErrorCode, type WebAPIRequestError } from './errors';
import {
buildGeneralFilesUploadWarning,
Expand Down Expand Up @@ -964,6 +971,119 @@ describe('WebClient', () => {
});
});

describe('requestInterceptor', () => {
function configureMockServer(expectedBody: () => Record<string, unknown>) {
nock('https://slack.com/api', {
reqheaders: {
test: 'static-header-value',
'Content-Type': 'application/json',
},
})
.post(/method/, (requestBody) => {
expect(requestBody).to.deep.equal(expectedBody());
return true;
})
.reply(200, (_uri, requestBody) => {
expect(requestBody).to.deep.equal(expectedBody());
return { ok: true, response_metadata: requestBody };
});
}

it('can intercept out going requests, synchronously modifying the request body and headers', async () => {
let expectedBody: Record<string, unknown>;

const client = new WebClient(token, {
requestInterceptor: (config: RequestConfig) => {
expectedBody = Object.freeze({
method: config.method,
base_url: config.baseURL,
path: config.url,
body: config.data ?? {},
query: config.params ?? {},
headers: structuredClone(config.headers),
test: 'static-body-value',
});
config.data = expectedBody;

config.headers.test = 'static-header-value';
config.headers['Content-Type'] = 'application/json';

return config;
},
});

configureMockServer(() => expectedBody);

await client.apiCall('method');
});

it('can intercept out going requests, asynchronously modifying the request body and headers', async () => {
let expectedBody: Record<string, unknown>;

const client = new WebClient(token, {
requestInterceptor: async (config: RequestConfig) => {
expectedBody = Object.freeze({
method: config.method,
base_url: config.baseURL,
path: config.url,
body: config.data ?? {},
query: config.params ?? {},
headers: structuredClone(config.headers),
test: 'static-body-value',
});

config.data = expectedBody;

config.headers.test = 'static-header-value';
config.headers['Content-Type'] = 'application/json';

return config;
},
});

configureMockServer(() => expectedBody);

await client.apiCall('method');
});
});

describe('adapter', () => {
it('allows for custom handling of requests with preconfigured http client', async () => {
nock('https://slack.com/api', {
reqheaders: {
'User-Agent': 'custom-axios-client',
},
})
.post(/method/)
.reply(200, (_uri, requestBody) => {
return { ok: true, response_metadata: requestBody };
});

const customLoggingInterceptor = (config: InternalAxiosRequestConfig) => {
// client with custom logging behaviour
return config;
};
const customLoggingSpy = sinon.spy(customLoggingInterceptor);

const customAxiosClient = axios.create();
customAxiosClient.interceptors.request.use(customLoggingSpy);

const customClientRequestSpy = sinon.spy(customAxiosClient, 'request');

const client = new WebClient(token, {
adapter: (config: RequestConfig) => {
config.headers['User-Agent'] = 'custom-axios-client';
return customAxiosClient.request(config);
},
});

await client.apiCall('method');

expect(customLoggingSpy.calledOnce).to.be.true;
expect(customClientRequestSpy.calledOnce).to.be.true;
});
});

it('should throw an error if the response has no retry info', async () => {
// @ts-expect-error header values cannot be undefined
const scope = nock('https://slack.com').post(/api/).reply(429, {}, { 'retry-after': undefined });
Expand Down
79 changes: 65 additions & 14 deletions packages/web-api/src/WebClient.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,17 @@
import type { Agent } from 'node:http';
import { basename } from 'node:path';
import { stringify as qsStringify } from 'node:querystring';
import type { Readable } from 'node:stream';
import type { SecureContextOptions } from 'node:tls';
import { TextDecoder } from 'node:util';
import zlib from 'node:zlib';

import axios, { type AxiosHeaderValue, type AxiosInstance, type AxiosResponse } from 'axios';
import axios, {
type InternalAxiosRequestConfig,
type AxiosHeaderValue,
type AxiosInstance,
type AxiosResponse,
type AxiosAdapter,
} from 'axios';
import FormData from 'form-data';
import isElectron from 'is-electron';
import isStream from 'is-stream';
Expand Down Expand Up @@ -90,6 +95,20 @@
* @default true
*/
attachOriginalToWebAPIRequestError?: boolean;
/**
* Custom function to modify outgoing requests. See {@link https://axios-http.com/docs/interceptors Axios interceptor documentation} for more details.
* @type {Function | undefined}
* @default undefined
*/
requestInterceptor?: RequestInterceptor;
filmaj marked this conversation as resolved.
Show resolved Hide resolved
/**
* Custom functions for modifing and handling outgoing requests.
* Useful if you would like to manage outgoing request with a custom http client.
mtjandra marked this conversation as resolved.
Show resolved Hide resolved
* See {@link https://github.com/axios/axios/blob/v1.x/README.md?plain=1#L586 Axios adapter documentation} for more information.
* @type {Function | undefined}
* @default undefined
*/
adapter?: AdapterConfig;
}

export type TLSOptions = Pick<SecureContextOptions, 'pfx' | 'key' | 'passphrase' | 'cert' | 'ca'>;
Expand Down Expand Up @@ -130,6 +149,24 @@
? A
: never;

/**
* An alias to {@link https://github.com/axios/axios/blob/v1.x/index.d.ts#L367 Axios' `InternalAxiosRequestConfig`} object,
* which is the main parameter type provided to Axios interceptors and adapters.
*/
export type RequestConfig = InternalAxiosRequestConfig;
filmaj marked this conversation as resolved.
Show resolved Hide resolved

/**
* An alias to {@link https://github.com/axios/axios/blob/v1.x/index.d.ts#L489 Axios' `AxiosInterceptorManager<InternalAxiosRequestConfig>` onFufilled} method,
* which controls the custom request interceptor logic
*/
export type RequestInterceptor = (config: RequestConfig) => RequestConfig | Promise<RequestConfig>;

/**
* An alias to {@link https://github.com/axios/axios/blob/v1.x/index.d.ts#L112 Axios' `AxiosAdapter`} interface,
* which is the contract required to specify an adapter
*/
export type AdapterConfig = AxiosAdapter;

/**
* A client for Slack's Web API
*
Expand Down Expand Up @@ -196,6 +233,9 @@

/**
* @param token - An API token to authenticate/authorize with Slack (usually start with `xoxp`, `xoxb`)
* @param {Object} [webClientOptions] - Configuration options.
* @param {Function} [webClientOptions.requestInterceptor] - An interceptor to mutate outgoing requests. See {@link https://axios-http.com/docs/interceptors Axios interceptors}
* @param {Function} [webClientOptions.adapter] - An adapter to allow custom handling of requests. Useful if you would like to use a pre-configured http client. See {@link https://github.com/axios/axios/blob/v1.x/README.md?plain=1#L586 Axios adapter}
*/
public constructor(
token?: string,
Expand All @@ -212,6 +252,8 @@
headers = {},
teamId = undefined,
attachOriginalToWebAPIRequestError = true,
requestInterceptor = undefined,
adapter = undefined,
}: WebClientOptions = {},
) {
super();
Expand Down Expand Up @@ -240,12 +282,12 @@
if (this.token && !headers.Authorization) headers.Authorization = `Bearer ${this.token}`;

this.axios = axios.create({
adapter: adapter ? (config: InternalAxiosRequestConfig) => adapter({ ...config, adapter: undefined }) : undefined,
timeout,
baseURL: slackApiUrl,
headers: isElectron() ? headers : { 'User-Agent': getUserAgent(), ...headers },
httpAgent: agent,
httpsAgent: agent,
transformRequest: [this.serializeApiCallOptions.bind(this)],
validateStatus: () => true, // all HTTP status codes should result in a resolved promise (as opposed to only 2xx)
maxRedirects: 0,
// disabling axios' automatic proxy support:
Expand All @@ -254,9 +296,16 @@
// protocols), users of this package should use the `agent` option to configure a proxy.
proxy: false,
});
// serializeApiCallOptions will always determine the appropriate content-type
// serializeApiCallData will always determine the appropriate content-type
this.axios.defaults.headers.post['Content-Type'] = undefined;

// request interceptors have reversed execution order
filmaj marked this conversation as resolved.
Show resolved Hide resolved
// see: https://github.com/axios/axios/blob/v1.x/test/specs/interceptors.spec.js#L88
if (requestInterceptor) {
this.axios.interceptors.request.use(requestInterceptor, null);
}
this.axios.interceptors.request.use(this.serializeApiCallData.bind(this), null);

this.logger.debug('initialized');
}

Expand Down Expand Up @@ -667,18 +716,16 @@
* a string, used when posting with a content-type of url-encoded. Or, it can be a readable stream, used
* when the options contain a binary (a stream or a buffer) and the upload should be done with content-type
* multipart/form-data.
* @param options - arguments for the Web API method
* @param headers - a mutable object representing the HTTP headers for the outgoing request
* @param config - The Axios request configuration object
*/
private serializeApiCallOptions(
options: Record<string, unknown>,
headers?: Record<string, string>,
): string | Readable {
private serializeApiCallData(config: InternalAxiosRequestConfig): InternalAxiosRequestConfig {
const { data, headers } = config;

// The following operation both flattens complex objects into a JSON-encoded strings and searches the values for
// binary content
let containsBinaryData = false;
// biome-ignore lint/suspicious/noExplicitAny: call options can be anything
const flattened = Object.entries(options).map<[string, any] | []>(([key, value]) => {
// biome-ignore lint/suspicious/noExplicitAny: HTTP request data can be anything
const flattened = Object.entries(data).map<[string, any] | []>(([key, value]) => {
if (value === undefined || value === null) {
return [];
}
Expand Down Expand Up @@ -730,21 +777,25 @@
headers[header] = value;
}
}
return form;
config.data = form;
config.headers = headers;
return config;

Check warning on line 782 in packages/web-api/src/WebClient.ts

View check run for this annotation

Codecov / codecov/patch

packages/web-api/src/WebClient.ts#L780-L782

Added lines #L780 - L782 were not covered by tests
}

// Otherwise, a simple key-value object is returned
if (headers) headers['Content-Type'] = 'application/x-www-form-urlencoded';
// biome-ignore lint/suspicious/noExplicitAny: form values can be anything
const initialValue: { [key: string]: any } = {};
return qsStringify(
config.data = qsStringify(
flattened.reduce((accumulator, [key, value]) => {
if (key !== undefined && value !== undefined) {
accumulator[key] = value;
}
return accumulator;
}, initialValue),
);
config.headers = headers;
return config;
}

/**
Expand Down