-
Notifications
You must be signed in to change notification settings - Fork 38.8k
feat: Add token to sendAndWait operations links to walidate in webhook #17566
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
feat: Add token to sendAndWait operations links to walidate in webhook #17566
Conversation
…unprotected-human-in-the-loop-webhooks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cubic analysis
2 issues found across 21 files • Review in cubic
React with 👍 or 👎 to teach cubic. You can also tag @cubic-dev-ai
to give feedback, ask questions, or re-run the review.
packages/core/src/execution-engine/node-execution-context/webhook-context.ts
Outdated
Show resolved
Hide resolved
❌ 1 Tests Failed:
View the top 1 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
|
||
const expectedToken = this.getExecutionWaitingToken(); | ||
|
||
const valid = crypto.timingSafeEqual(Buffer.from(token), Buffer.from(expectedToken)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we make this backwards compatible so that executions created before the introduction of the HMAC mechanism can still be resumed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll check, but for now it doesn't look like it will be possible
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it could be doable by storing the current largest exec id (eg using a migration) and require the signature only for executions that are newer than the oldest one before this change. Requires some effort but prolly doable
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or we persist the expected signature as part of execution data and compare on resume…
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe also by using timestamps of execution
question is do we want have code for that included in n8n and affect every single execution farther
breaking will be only single execution of workflow started before instance upgrade, so probably not a lot
packages/core/src/constants.ts
Outdated
@@ -19,3 +19,5 @@ export const CREDENTIAL_ERRORS = { | |||
INVALID_JSON: 'Decrypted credentials data is not valid JSON.', | |||
INVALID_DATA: 'Credentials data is not in a valid format.', | |||
}; | |||
|
|||
export const WAITING_TOKEN_QUERY_PARAM = 'n8nwaitingtoken'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should call the query param sig
or signature
which is commonly used name for a param like this
@@ -178,6 +180,22 @@ export abstract class NodeExecutionContext implements Omit<FunctionsBase, 'getCr | |||
return this.instanceSettings.instanceId; | |||
} | |||
|
|||
protected getExecutionWaitingToken() { | |||
const token = crypto | |||
.createHmac('sha256', this.instanceSettings.encryptionKey) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should not use the instance encryption key directly for this as it’s not safe. Instead we should let it be configurable and by default derive a nee key from the encryption key. See eg the binary file signing key
* Secret for creating publicly-accesible signed URLs for binary data. |
protected getExecutionWaitingToken() { | ||
const token = crypto | ||
.createHmac('sha256', this.instanceSettings.encryptionKey) | ||
.update(this.getExecutionId()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signing only the execution id is not secure. It allows changing any query param and the signature will still be valid. It also makes any subsequent wait URL have the same signature. Instead we need to sugn all critical parts, at least path and all query params.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we could not include query params, as we expect them be different, depending of users choice
I think we could add stringified node's parameters to update, this would be unique per waiting node and we could decode it later, WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh so users can provide arbitrary input to the hitl nodes using query params? I was thinking of signing the URL but I guess we could also sign the details that identify the 1) execution 2) node 3) input (unless arbitrary)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
currently it's execution id + stringified node(INode), I think this would be unique and secure
|
||
const expectedToken = this.getExecutionWaitingToken(); | ||
|
||
const valid = crypto.timingSafeEqual(Buffer.from(token), Buffer.from(expectedToken)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comment from cubic is valid. Don’t ignore it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, had to wrapped in trycatch block, thought I did this, thanks
…unprotected-human-in-the-loop-webhooks
@@ -225,6 +228,14 @@ export class InstanceSettings { | |||
.digest('hex'); | |||
} | |||
|
|||
private getOrGenerateHmacSignatureSecret() { | |||
const hmacSignatureSecretFromEnv = process.env.N8N_HMAC_SIGNATURE_SECRET; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just for my understanding, why would using the encryptionKey (or a hash of it) be considered insecure? Introducing new env vars adds configuration complexity and I think we should avoid it unless really necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
having separate hmacSignatureSecret
make sense to me
I would say we could skip adding new env and always derive hmacSignatureSecret
from encryption key
@tomi what do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah ideally I would like us to have only single signing secret, but we have named the binary one very specifically, so we can’t reuse it (without breaking change). As long as we derive a nee key from encryption key it should be okay.
Using it directly is not a good idea because
- They have different purposes and different risks. It’s used for encryption while signing key only for signing messages. If one is leaked the other still remains secret
- Using the same key can be used to probe details (eg bits) about the key by crafting input to the hmac validation and observing the answer.
More details here
…unprotected-human-in-the-loop-webhooks
…unprotected-human-in-the-loop-webhooks
const isTokenValid = this.validateExecutionWaitingToken(); | ||
|
||
if (!isTokenValid) { | ||
throw new Error('Invalid waiting token'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be a NodeOperationError
or another class?
Could validateExecutionWaitingToken
do the throw?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be a NodeOperationError or another class?
Yes, maybe a UserError or NodeApiErro would be more appropriate.
Could validateExecutionWaitingToken handle the throw?
I considered that, but I think it's better if this helper simply returns whether the token is valid. The calling logic can then decide how to proceed. Maybe we should rename it to isExecutionWaitingTokenValid
to reflect that behavior.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
UserError
probably makes more sense here as it's invalid input from the user, tho it should never be invalid unless there's a bug or it's malicious input from a user, so NodeApiError
might do as well. Don't really mind which one it is, as long as it results in a 403 response for the webhook.
@@ -357,6 +357,12 @@ export async function sendAndWaitWebhook(this: IWebhookFunctions) { | |||
const res = this.getResponseObject(); | |||
const req = this.getRequestObject(); | |||
|
|||
const isTokenValid = this.validateExecutionWaitingToken(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be possible to do this in core, maybe in waiting-webhooks
, for any waiting execution that is resumed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as for now doing this will break other waiting execution, because updates in nodes will be needed, this PR deals only with HiTL operations
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ideally we should do this in core and provide necessary methods for nodes. I think it’s ok to do it for hitl nodes first
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It feels wrong to have this in the core.
Currently, this is a complete feature that allows obtaining a signing token and validating it at the node level.
If we move it to the core, we would need to check what kind of node it is and what is current operation. This logic would likely need to be modified later.
We should also consider community nodes - there’s already interest in adding send and wait operations. In my opinion, it’s better to provide tools for node developers rather than handling this at the core level.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking if we could always add the signature to $execution.resumeUrl
and verify the signature of all (not specific nodes) incoming requests on waiting-webhook
.
Then node code wouldn't have to be changed and $execution.resumeUrl
would keep working for existing workflows.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we could, but it would increase the scope
we’d have to ensure that the signature is always included in all possible waiting executions
changes to the nodes would still be required - for example like in this PR for including url in service's message
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Exactly what Elias said. Every node using resume URL should have this. But like I said, let's keep this scoped to HITL nodes for now.
@@ -480,8 +486,6 @@ export function getSendAndWaitConfig(context: IExecuteFunctions): SendAndWaitCon | |||
.replace(/\\n/g, '\n') | |||
.replace(/<br>/g, '\n'); | |||
const subject = escapeHtml(context.getNodeParameter('subject', 0, '') as string); | |||
const resumeUrl = context.evaluateExpression('{{ $execution?.resumeUrl }}', 0) as string; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean that $execution.resumeUrl
will no longer work because it would be missing the signature?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, unfortunately, this is breaking change for responses for executions started before update, still looking if it's possible to avoid
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But wouldn't it also be breaking for new executions, if your workflow uses $execution.resumeUrl
in an expression?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
resumeUrl
is only used in the Wait node for webhook mode
In other waiting nodes, such as sendAndWait
operations and forms, we provide the FE with a way to resume execution
it would not be breaking, this PR does not modify resumeUrl
sendAndWait
operations currently use resumeUrl
+ nodeId
Forms use resumeFormUrl
…unprotected-human-in-the-loop-webhooks
…unprotected-human-in-the-loop-webhooks
…unprotected-human-in-the-loop-webhooks
@@ -357,6 +357,12 @@ export async function sendAndWaitWebhook(this: IWebhookFunctions) { | |||
const res = this.getResponseObject(); | |||
const req = this.getRequestObject(); | |||
|
|||
const isTokenValid = this.validateExecutionWaitingToken(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Exactly what Elias said. Every node using resume URL should have this. But like I said, let's keep this scoped to HITL nodes for now.
const isTokenValid = this.validateExecutionWaitingToken(); | ||
|
||
if (!isTokenValid) { | ||
throw new Error('Invalid waiting token'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
UserError
probably makes more sense here as it's invalid input from the user, tho it should never be invalid unless there's a bug or it's malicious input from a user, so NodeApiError
might do as well. Don't really mind which one it is, as long as it results in a 403 response for the webhook.
if (this.runExecutionData) { | ||
this.runExecutionData.validateSignature = true; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a bit odd that a get*
method does a side effect. Could we do this somewhere else?
Also, what if there is no execution data?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a bit odd that a get* method does a side effect. Could we do this somewhere else?
moved to constructor
Also, what if there is no execution data?
then we do not need to set anything as context not used in execution
…unprotected-human-in-the-loop-webhooks
…unprotected-human-in-the-loop-webhooks
…unprotected-human-in-the-loop-webhooks
…unprotected-human-in-the-loop-webhooks
@tomi updated as suggested, could you please review? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks mostly good! Few minor comments
if (this.runExecutionData) this.runExecutionData.validateSignature = true; | ||
} | ||
|
||
getSignedResumeUrl(parameters: Record<string, string> = {}) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add docs for this
const { webhookWaitingBaseUrl, executionId } = this.additionalData; | ||
|
||
if (typeof executionId !== 'string') { | ||
throw new ApplicationError('Execution id is missing'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ApplicationError
is deprecated. Is this an expected situation? If not, we could use UnexpectedError
here
const baseURL = new URL(`${webhookWaitingBaseUrl}/${executionId}/${this.node.id}`); | ||
|
||
for (const [key, value] of Object.entries(parameters)) { | ||
baseURL.searchParams.set(key, value); | ||
} | ||
|
||
const token = this.getExecutionWaitingToken(baseURL.toString()); | ||
|
||
baseURL.searchParams.set(WAITING_TOKEN_QUERY_PARAM, token); | ||
|
||
return baseURL.toString(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
@@ -200,6 +202,38 @@ export abstract class NodeExecutionContext implements Omit<FunctionsBase, 'getCr | |||
return this.instanceSettings.instanceId; | |||
} | |||
|
|||
getExecutionWaitingToken(url: string) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this used outside this class? Should make it private/protected if not
@@ -172,4 +174,28 @@ export class WebhookContext extends NodeExecutionContext implements IWebhookFunc | |||
itemIndex, | |||
); | |||
} | |||
|
|||
validateExecutionWaitingToken() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's document this method
|
||
if (typeof token !== 'string') return false; | ||
|
||
const parsedUrl = new URL(req.url, `http://${req.headers.host}`); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we assume the URL is http
? Couldn't it be https as well? Do we need to define the base
?
@@ -163,7 +163,7 @@ export function createSendAndWaitMessageBody(context: IExecuteFunctions) { | |||
const config = getSendAndWaitConfig(context); | |||
|
|||
const buttons: string[] = config.options.map( | |||
(option) => `*<${`${config.url}?approved=${option.value}`}|${option.label}>*`, | |||
(option) => `*<${`${option.url}`}|${option.label}>*`, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are actually cleaner now that they don't build URLs anymore 👍
Summary
Links in sendAndWait operations will include a token, which will be validated upon receipt in the webhook method
Related Linear tickets, Github issues, and Community forum posts
https://linear.app/n8n/issue/NODE-3300/unprotected-human-in-the-loop-webhooks
Review / Merge checklist
release/backport
(if the PR is an urgent fix that needs to be backported)