chore(core): Move scopes and roles into database in preparation for custom roles #17226

afitzek · 2025-07-11T09:30:08Z

Summary

We move scopes and roles into the database and use the database as the source of truth for these. This is to
prepare the system for custom roles, which can be stored in the database.

Related Linear tickets, Github issues, and Community forum posts

closes https://linear.app/n8n/issue/PAY-3054/move-role-definitions-into-the-database

Review / Merge checklist

PR title and summary are descriptive. (conventions)
Docs updated or follow-up ticket created.
Tests included.
PR Labeled with release/backport (if the PR is an urgent fix that needs to be backported)

codecov · 2025-07-11T09:42:22Z

❌ 1631 Tests Failed:

Tests completed	Failed	Passed	Skipped
9231	1631	7600	0

View the top 3 failed test(s) by shortest run time

--deleteWorkflowsAndCredentials resets LDAP settings

Stack Traces | 0s run time

QueryFailedError: SQLITE_ERROR: near ":member": syntax error
    at handler (.../n8n/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]_pg@.../driver/sqlite/SqliteQueryRunner.ts:137:29)
    at replacement (.../n8n/node_modules/.pnpm/[email protected]..../sqlite3/lib/trace.js:25:27)
    at Statement.errBack (.../n8n/node_modules/.pnpm/[email protected]..../sqlite3/lib/sqlite3.js:15:21)

PATCH /projects/:projectId member management should not add or remove users from a personal project

Stack Traces | 0s run time

QueryFailedError: SQLITE_ERROR: near ":member": syntax error
    at handler (.../n8n/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]_pg@.../driver/sqlite/SqliteQueryRunner.ts:137:29)
    at replacement (.../n8n/node_modules/.pnpm/[email protected]..../sqlite3/lib/trace.js:25:27)
    at Statement.errBack (.../n8n/node_modules/.pnpm/[email protected]..../sqlite3/lib/sqlite3.js:15:21)

POST /projects/:projectId/folders should create folder in personal project

Stack Traces | 0s run time

QueryFailedError: SQLITE_ERROR: near ":member": syntax error
    at handler (.../n8n/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]_pg@.../driver/sqlite/SqliteQueryRunner.ts:137:29)
    at replacement (.../n8n/node_modules/.pnpm/[email protected]..../sqlite3/lib/trace.js:25:27)
    at Statement.errBack (.../n8n/node_modules/.pnpm/[email protected]..../sqlite3/lib/sqlite3.js:15:21)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

bundlemon · 2025-07-29T10:02:37Z

BundleMon

Files added (2)

Status	Path	Size	Limits
✅	WASM Dependencies _{tree-sitter-bash.wasm}	+181.26KB	-
✅	WASM Dependencies _{tree-sitter.wasm}	+74.47KB	-

Total files change +255.73KB

Groups added (2)

Status	Path	Size	Limits
✅	*/.js	+5.35MB	-
✅	*/.css	+188.05KB	-

Final result: ✅

View report in BundleMon website ➡️

Current branch size history

cubic-dev-ai

cubic analysis

10 issues found across 26 files • Review in cubic

_{React with 👍 or 👎 to teach cubic. You can also tag @cubic-dev-ai to give feedback, ask questions, or re-run the review.}

packages/@n8n/db/src/entities/role.ts

packages/@n8n/db/src/entities/scope.ts

packages/@n8n/permissions/src/types.ee.ts

cubic-dev-ai · 2025-07-29T11:29:21Z

packages/@n8n/db/src/migrations/common/1750252139168-LinkRoleToUserTable.ts

+					systemRole: true,
+				},
+			);
+		} catch (error) {


Catching and ignoring every error hides real migration failures; consider only suppressing duplicate-key violations.

Prompt for AI agents

Address the following comment on packages/@n8n/db/src/migrations/common/1750252139168-LinkRoleToUserTable.ts at line 30: <comment>Catching and ignoring every error hides real migration failures; consider only suppressing duplicate-key violations.</comment> <file context> @@ -0,0 +1,78 @@ +import type { MigrationContext, ReversibleMigration } from '../migration-types'; + +/* + * This migration + */ + +export class LinkRoleToUserTable1750252139168 implements ReversibleMigration { + async up({ + schemaBuilder: { addForeignKey, addColumns, column }, </file context>

Agree with the bot here 🤣

packages/@n8n/db/src/migrations/common/1750252139168-LinkRoleToUserTable.ts

packages/cli/src/auth/auth.roles.service.ts

packages/@n8n/permissions/src/scope-information.ts

packages/@n8n/db/src/repositories/user.repository.ts

guillaumejacquart

some minor comments

packages/@n8n/db/src/entities/role.ts

guillaumejacquart · 2025-07-29T14:53:38Z

packages/@n8n/db/src/entities/types-db.ts

@@ -105,7 +105,7 @@ export interface PublicUser {
 	passwordResetToken?: string;
 	createdAt: Date;
 	isPending: boolean;
-	role?: GlobalRole;
+	role?: string;


Could we do this on followup PR ?

packages/@n8n/permissions/src/scope-information.ts

guillaumejacquart · 2025-07-30T06:47:47Z

packages/@n8n/db/src/migrations/common/1750252139168-LinkRoleToUserTable.ts

+					systemRole: true,
+				},
+			);
+		} catch (error) {


Agree with the bot here 🤣

guillaumejacquart · 2025-07-30T06:48:57Z

packages/@n8n/db/src/migrations/common/1750252139168-LinkRoleToUserTable.ts

+		const systemRoleColumn = escape.columnName('systemRole');
+
+		// Make sure that the global roles that we need exist
+		try {


DO you think it would make sense to have a loop here (of ['global:owner', ''global:admin', 'global:member']) ?

guillaumejacquart

Some additional minor comments

packages/@n8n/db/src/entities/role.ts

packages/@n8n/db/src/migrations/common/1750252139167-AddRolesTables.ts

packages/@n8n/db/src/migrations/common/1750252139168-LinkRoleToUserTable.ts

guillaumejacquart · 2025-07-30T13:08:13Z

packages/@n8n/db/src/migrations/common/1750252139168-LinkRoleToUserTable.ts

+		for (const role of ['global:owner', 'global:admin', 'global:member']) {
+			if (dbType === 'sqlite') {
+				await runQuery(
+					`INSERT OR REPLACE INTO ${tableName} (${slugColumn}, ${roleTypeColumn}, ${systemRoleColumn}) VALUES (:slug, :roleType, :systemRole)`,


Sqlite supports upsert now: https://sqlite.org/lang_upsert.html
So it could have the same syntax as postgres I think.
Plus currently, your sqlite/mysql query replace in case of conflict, while the postgres one ignore on conflict

You are right, I was able to change the insert queries to all ignore on conflict, but still all 3 db queries are different, the sqlite one needs an INSERT OR REPLACE, which postgres doesn't recognize.

I was under the impression that Sqlite did not need INSERT OR REPLACE (see examples here): https://sqlite.org/lang_upsert.html#examples

Ah you are right, I miss read 🙈

guillaumejacquart · 2025-07-30T13:14:25Z

packages/cli/src/auth/auth.roles.service.ts

+			};
+			return [slug, info.displayName, info.description ?? null] as const;
+		})
+			.filter(([slug, displayName, description]) => {


(completely optional)
I'm questioning the relevance of splitting filter with map, because we need to check existing scope exists in both steps.

What about:

const scopesToSave = ALL_SCOPES.map((slug) => { const info = scopeInformation[slug] ?? { displayName: slug, description: null }; const existing = existingScopes.find(scope => scope.slug === slug); // Create new scope if doesn't exist if (!existing) { const newScope = new Scope(); newScope.slug = slug; newScope.displayName = info.displayName; newScope.description = info.description ?? null; return newScope; } // Update existing scope if needed const needsUpdate = existing.displayName !== info.displayName || existing.description !== (info.description ?? null); if (needsUpdate) { existing.displayName = info.displayName; existing.description = info.description ?? null; return existing; } return null; }).filter(Boolean);

guillaumejacquart

looks good to me

packages/@n8n/db/src/migrations/common/1750252139168-LinkRoleToUserTable.ts

currents-bot · 2025-07-31T08:52:53Z

✅

E2E Tests: n8n tests passed after 5m 29.4s

🟢 508 · 🔴 0 · ⚪️ 0

View Run Details

Run Details

Project: n8n
Groups: 1
Framework: Currents
Run Status: Passed
Commit: 680c013
Spec files: 105
Overall tests: 508
Duration: 5m 29.4s
Parallelization: 1

This message was posted automatically by currents.dev | Integration Settings

tomi · 2025-07-31T12:33:38Z

packages/@n8n/db/src/entities/role.ts

+		default: false,
+		name: 'systemRole',
+	})
+	systemRole: boolean; // Indicates if the role is managed by the system and cannot be edited


Can we move the comments here to JSDoc so they are discoverable via intellisense?

tomi · 2025-08-01T05:53:28Z

packages/@n8n/db/src/migrations/common/1750252139166-AddScopeTables.ts

+	async up({ schemaBuilder: { createTable, column } }: MigrationContext) {
+		await createTable('scope').withColumns(
+			column('slug').varchar(128).primary.notNull,
+			column('displayName').text.default(null),


This seems to be not null in the entity definition. Probably should be here as well? Same applies to the other migrations

tomi · 2025-08-01T05:53:55Z

packages/@n8n/db/src/migrations/common/1750252139166-AddScopeTables.ts

+ * slug        | Text | Unique identifier of the scope for example: 'project:create'
+ * displayName | Text | Name used to display in the UI
+ * description | Text | Text describing the scope in more detail of users


Could we add these descriptions as comments to the columns with .comment(...). Same applies to the other migrations

tomi · 2025-08-01T05:54:11Z

packages/@n8n/db/src/migrations/common/1750252139166-AddScopeTables.ts

Any reason why we have 3 separate migrations instead of 1?

I wanted to keep the migrations small and simple and specific to do one thing.

Personally I'd prefer just one, since every migration is extra overhead in instance startup. One migration can also be composed into smaller functions. But I can understand your reasoning as well.

tomi · 2025-08-01T05:55:06Z

packages/@n8n/db/src/migrations/common/1750252139167-AddRolesTables.ts

+			column('slug').varchar(128).primary.notNull,
+			column('displayName').text.default(null),
+			column('description').text.default(null),
+			column('roleType').text.default(null),


Could we have a constraint that enforces the valid values for this column?

We opted to not validate the values in the database in there, because we might want to add more values here later on, at which point we would need to do another migration to extend the the type.

tomi · 2025-08-01T06:18:29Z

packages/@n8n/permissions/src/scope-information.ts

+	return resourceScopes;
+}
+
+export const ALL_SCOPES = buildResourceScopes();


What do you think about having a test case that does expect(ALL_SCOPES).toMatchSnapshot() so a) we have a list of all scopes and b) one needs explicit action if they are changed?

tomi · 2025-08-01T06:20:28Z

packages/cli/src/auth/auth.roles.service.ts

+				description: null,
+			};
+
+			const existingScope = availableScopes.find((scope) => scope.slug === slug);


nit: This is a linear search. The array is probably not big, but it all adds up. Using e.g. Map for availableScopes would be more approriate

tomi · 2025-08-01T06:21:48Z

packages/cli/src/auth/auth.roles.service.ts

+		for (const roleNamespace of Object.keys(ALL_ROLES) as Array<keyof typeof ALL_ROLES>) {
+			const rolesToUpdate = ALL_ROLES[roleNamespace]
+				.map((role) => {
+					const existingRole = existingRoles.find((r) => r.slug === role.role);


tomi · 2025-08-01T06:29:09Z

packages/cli/src/auth/auth.roles.service.ts

+			await this.scopeRepository.save(scopesToUpdate);
+			this.logger.info('Scopes updated successfully.');
+		} else {
+			this.logger.info('No scopes to update.');


Would debug be more approriate here? Not sure if this is needs to be logged on every startup

tomi · 2025-08-01T06:29:18Z

packages/cli/src/auth/auth.roles.service.ts

+				await this.roleRepository.save(rolesToUpdate);
+				this.logger.info(`${roleNamespace} roles updated successfully.`);
+			} else {
+				this.logger.info(`No ${roleNamespace} roles to update.`);


tomi · 2025-08-01T06:30:37Z

packages/cli/src/auth/auth.roles.service.ts

Very clean and easy to follow code. Nice job 👏

despairblue

I only looked at the migrations and they look solid. 👍🏾

tomi

Thank you for addressing the feedback 💘 LGTM now. Just one question regarding multi-main mode

tomi · 2025-08-04T18:37:45Z

packages/@n8n/db/src/services/auth.roles.service.ts

+		}
+	}
+
+	async init() {


Does this still behave correctly when we have multiple mains starting up simultaneously and all racing to run this?

afitzek · 2025-08-04T20:51:42Z

Thank you for addressing the feedback 💘 LGTM now. Just one question regarding multi-main mode

Thanks, that is a good question:
I think with multi-main it should still be good, because if all mains are running the same version, they try to run the same query, worst case would be that each main runs the same update, but this shouldn't lead to any problems. On the other hand, if only the leader runs this update, with a zero down time update, there can be multiple different versions running at the same time, so we wouldn't be able to ensure that a leader runs this during start up, we would need to run it on each leader change, which would also lead to multiple executions of the query in the end. But if this would become an issue, I think we could look into synchronizing this.

n8n-assistant bot added core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team labels Jul 11, 2025

afitzek force-pushed the pay-3054-move-roles-into-database branch from 4d9a6d0 to 6602f56 Compare July 16, 2025 08:09

afitzek force-pushed the pay-3054-move-roles-into-database branch from 49e3965 to 14920c9 Compare July 29, 2025 09:25

afitzek marked this pull request as ready for review July 29, 2025 11:20

afitzek requested a review from a team as a code owner July 29, 2025 11:20

cubic-dev-ai bot reviewed Jul 29, 2025

View reviewed changes

guillaumejacquart reviewed Jul 30, 2025

View reviewed changes

afitzek requested a review from guillaumejacquart July 30, 2025 12:14

guillaumejacquart reviewed Jul 30, 2025

View reviewed changes

guillaumejacquart previously approved these changes Jul 30, 2025

View reviewed changes

afitzek dismissed guillaumejacquart’s stale review via e978b3e July 30, 2025 14:04

guillaumejacquart reviewed Jul 30, 2025

View reviewed changes

packages/@n8n/db/src/migrations/common/1750252139168-LinkRoleToUserTable.ts Show resolved Hide resolved

afitzek requested a review from a team July 31, 2025 06:18

afitzek force-pushed the pay-3054-move-roles-into-database branch 2 times, most recently from f6c05ad to 680c013 Compare July 31, 2025 07:13

afitzek requested a review from guillaumejacquart July 31, 2025 07:27

guillaumejacquart previously approved these changes Jul 31, 2025

View reviewed changes

tomi reviewed Aug 1, 2025

View reviewed changes

packages/cli/src/auth/auth.roles.service.ts Outdated

Copy link

Collaborator

tomi Aug 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very clean and easy to follow code. Nice job 👏

despairblue previously approved these changes Aug 1, 2025

View reviewed changes

guillaumejacquart dismissed stale reviews from despairblue and themself via 3bdc588 August 1, 2025 15:12

guillaumejacquart force-pushed the pay-3054-move-roles-into-database branch from 680c013 to 3bdc588 Compare August 1, 2025 15:12

afitzek added 3 commits August 4, 2025 11:48

Create scope table and sync scopes to db

8c57148

Add roles tables

43e9c6f

Link user table to global roles

3f71e79

afitzek added 15 commits August 4, 2025 11:48

Fix role syncing

3d56981

Adapt DB

c1f90c5

Rename migration

cef6f2d

Fix

f8d7c57

Adaptions

faa59a1

Adaptions

de82bca

Adaptions

fe722b3

Change migration

c1b873d

Add roles tables

541fa7c

Address AI comments

3d3be7b

Address comments

d9bc071

Address comments

6a9423f

Address comments

144ab8b

Address review

fe66d74

Address review

55e8cc9

afitzek force-pushed the pay-3054-move-roles-into-database branch from 99f21b5 to 55e8cc9 Compare August 4, 2025 09:48

Address comments

16df029

afitzek requested review from tomi and guillaumejacquart August 4, 2025 11:41

tomi reviewed Aug 4, 2025

View reviewed changes

chore(core): Move scopes and roles into database in preparation for custom roles #17226

Are you sure you want to change the base?

chore(core): Move scopes and roles into database in preparation for custom roles #17226

Conversation

afitzek commented Jul 11, 2025

Summary

Related Linear tickets, Github issues, and Community forum posts

Review / Merge checklist

Uh oh!

codecov bot commented Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 1631 Tests Failed:

Uh oh!

bundlemon bot commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

BundleMon

Uh oh!

cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

cubic analysis

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cubic-dev-ai bot Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

guillaumejacquart left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

guillaumejacquart left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

guillaumejacquart left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

currents-bot bot commented Jul 31, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

codecov bot commented Jul 11, 2025 •

edited

Loading

bundlemon bot commented Jul 29, 2025 •

edited

Loading