feat: add new table of end user oauth

[CourTeous33]

Important

1. Make sure you have read our contribution guidelines
2. Ensure there is an associated issue and you have been assigned to it
3. Use the correct syntax to link this PR: Fixes #<issue number>.

Summary

Added a new table for end user auth:

id: Unique identifier for each authentication entry (auto-generated)
name: A human-readable name for the credentials (defaults to "API KEY 1")
tenant_id: Which tenant/organization this belongs to
end_user_id: Which specific end user owns these credentials
provider: The name of the tool/service provider (e.g., "github", "google", etc.)
encrypted_credentials: The actual credentials stored in encrypted JSON format
credential_type: What kind of authentication this is (defaults to "api-key", but could be "oauth2", etc.)
created_at: When the credentials were first added
updated_at: When they were last modified
expires_at: When the credentials expire (defaults to -1, meaning no expiration)

Screenshots

Before	After
...	...

Checklist

• This change requires a documentation update, included: Dify Document
• I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• I've updated the documentation accordingly.
• I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

[gemini-code-assist]

Summary of Changes

Hello @CourTeous33, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new database model, EndUserAuthenticationProvider, designed to securely store and manage authentication credentials for individual end users within the system's tool integration framework. This addition provides a granular approach to user authentication, allowing for specific credentials to be associated with and managed by each end user, thereby enhancing flexibility and security in how users interact with external tools.

Highlights

• New Database Table: Introduced EndUserAuthenticationProvider to store authentication credentials specifically for end users.
• User-Specific Authentication: This table enables the management of authentication details at the individual end-user level, distinct from tenant-level providers.
• Schema Definition: The table includes fields such as id, name, tenant_id, end_user_id, provider, encrypted_credentials, credential_type, and timestamps, with sensible defaults for name ("API KEY 1"), credential_type ("api-key"), and expires_at (-1 for no expiration).
• Data Integrity: Unique constraints ensure that each end-user authentication entry is distinct, and indexes are added for efficient querying by tenant_id and end_user_id.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new SQLAlchemy model EndUserAuthenticationProvider to store authentication credentials for end-users in tools. The changes look good and follow the existing patterns in the codebase. I've made a few suggestions to improve database portability and consistency by using SQLAlchemy's backend-agnostic functions and adding a missing Python-level default. Overall, a solid addition.

[QuantumGhost]

IMO, enforcing name uniqueness for a (provider, end_user_id) pair creates more overhead than value.

Ideally, we should rely on IDs as the primary identifier and treat names merely as user-facing hints. Therefore, strict uniqueness shouldn't be required. If a user creates duplicate names, they should bear the responsibility for any ambiguity.

Enforcing name uniqueness has caused implementation issues in the past, such as with credential management in the EE version. Furthermore, it complicates name generation.

That said, I suggest discussing this with the PM. I’ve skimmed the PRD and found no explicit requirement for name uniqueness.

[CourTeous33]

Agreed, should we remove the unique constraint at all? Since ID is enough for uniqueness

[QuantumGhost]

This would break MySQL support.

[CourTeous33]

Will edit the script at once after all comments got resolved at the new provider

[asukaminato0721]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a new table, tool_enduser_authentication_providers, for managing end-user authentication credentials, which is a valuable addition. The implementation includes a new Alembic migration and updates to the SQLAlchemy models. However, my review found several critical and high-severity inconsistencies between the model definition in api/models/tools.py and the database schema defined in the migration file. These discrepancies relate to unique constraints, column types, nullability, default values, and database indexes. It is crucial to resolve these mismatches to ensure data integrity, prevent potential runtime errors, and maintain a consistent schema. I have provided specific comments and code suggestions to address each of these issues.

[gemini-code-assist]

The unique constraint in the model definition is ("end_user_id", "provider", "name"), but the corresponding Alembic migration defines it as ("tenant_id", "provider", "end_user_id", "name"). This inconsistency can lead to unexpected database behavior and ORM errors. The constraint in the migration seems more correct as it includes tenant_id, ensuring uniqueness across the whole system. Please update the model to match the migration.

Suggested change

	__table_args__ = (
	sa.UniqueConstraint("end_user_id", "provider", "name"),
	)
	__table_args__ = (
	sa.UniqueConstraint("tenant_id", "provider", "end_user_id", "name"),
	)

[gemini-code-assist]

The provider column is defined as sa.String(length=256), but the corresponding model EndUserAuthenticationProvider defines it as LongText. This can cause data truncation if a provider name exceeds 256 characters. Please use models.types.LongText() for consistency with the model and to avoid potential data loss.

Suggested change

	sa.Column("provider", sa.String(length=256), nullable=False),
	sa.Column("provider", models.types.LongText(), nullable=False),

[gemini-code-assist]

There are several inconsistencies between the model definition and the Alembic migration regarding indexes:

1. name: The model specifies index=True, but the migration does not create a separate index for this column.
2. provider: The model specifies index=True, but the migration does not create a separate index.
3. tenant_id: The migration creates an index on this column, but the model does not specify index=True.

These discrepancies can lead to performance issues and schema drift. Please ensure the model definition and the migration are synchronized. It's recommended to make the model the source of truth and adjust the migration accordingly.

[gemini-code-assist]

The encrypted_credentials column is defined as non-nullable (nullable=False) with a Mapped[str] type hint, but the migration file defines it as nullable. This is inconsistent. To align with the database schema and avoid potential runtime errors, it should be defined as nullable. The BuiltinToolProvider model uses Mapped[str | None] and nullable=True, which is a good pattern to follow.

Suggested change

	encrypted_credentials: Mapped[str] = mapped_column(LongText, nullable=False, default="")
	encrypted_credentials: Mapped[str | None] = mapped_column(LongText, nullable=True, default=None)

[gemini-code-assist]

The id column uses default=uuidv7, while the migration uses uuid_generate_v4(). Furthermore, other models in this file, like BuiltinToolProvider, use uuid4. For consistency within the codebase and with the database schema, please use a consistent UUID generation method. Using uuid4 would align with other models in this file.

Suggested change

	id: Mapped[str] = mapped_column(StringUUID, primary_key=True, default=uuidv7, init=False)
	id: Mapped[str] = mapped_column(StringUUID, primary_key=True, default=lambda: str(uuid4()), init=False)

[gemini-code-assist]

The created_at column uses default=datetime.now. This can lead to issues with timezones as it may not be UTC-aware. It's better to use a consistent, timezone-aware UTC datetime. The BuiltinToolProvider model uses server_default=func.current_timestamp(), which is a good practice for ensuring database-level timestamping in UTC. Please update this to be consistent and avoid potential timezone bugs.

Suggested change

	created_at: Mapped[datetime] = mapped_column(
	sa.DateTime, nullable=False, default=datetime.now, init=False
	)
	created_at: Mapped[datetime] = mapped_column(
	sa.DateTime, nullable=False, server_default=func.current_timestamp(), init=False
	)