Skip to content

feat(tests): Add E2E test data creator script for comprehensive testing #10368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 15 commits into
base: dev
Choose a base branch
from

Conversation

Abhi1992002
Copy link
Contributor

Changes

  • Introduced a new script to generate test data for end-to-end (E2E) tests using API functions, ensuring compatibility with future model changes.
  • The script creates test users, agent blocks, graphs, profiles, library agents, presets, API keys, and store submissions.
  • Utilizes external services for image and video URLs, and includes error handling for data creation processes.
  • Provides a summary of created data upon completion, enhancing the testing framework for the AutoGPT platform.

Checklist 📋

For code changes:

  • I have clearly listed my changes in the PR description
  • I have made a test plan
  • I have tested my changes according to the test plan:
    • Test scripts are working perfectly and not breaking anything. Data is also correctly visible in the database.

- Introduced a new script to generate test data for end-to-end (E2E) tests using API functions, ensuring compatibility with future model changes.
- The script creates test users, agent blocks, graphs, profiles, library agents, presets, API keys, and store submissions.
- Utilizes external services for image and video URLs, and includes error handling for data creation processes.
- Provides a summary of created data upon completion, enhancing the testing framework for the AutoGPT platform.
@Abhi1992002 Abhi1992002 requested a review from a team as a code owner July 14, 2025 12:04
@Abhi1992002 Abhi1992002 requested review from 0ubbe and Swiftyos and removed request for a team July 14, 2025 12:04
@github-project-automation github-project-automation bot moved this to 🆕 Needs initial review in AutoGPT development kanban Jul 14, 2025
Copy link

netlify bot commented Jul 14, 2025

Deploy Preview for auto-gpt-docs-dev failed.

Name Link
🔨 Latest commit a419f21
🔍 Latest deploy log https://app.netlify.com/projects/auto-gpt-docs-dev/deploys/68773602ba32d00008163555

@Abhi1992002 Abhi1992002 marked this pull request as draft July 14, 2025 12:04
@github-actions github-actions bot added platform/backend AutoGPT Platform - Back end size/xl labels Jul 14, 2025
Copy link

netlify bot commented Jul 14, 2025

Deploy Preview for auto-gpt-docs failed.

Name Link
🔨 Latest commit a419f21
🔍 Latest deploy log https://app.netlify.com/projects/auto-gpt-docs/deploys/687736029c65030008056183

Copy link

You are nearing your monthly Qodo Merge usage quota. For more information, please visit here.

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 No relevant tests
🔒 Security concerns

Credential exposure:
The script contains hardcoded test passwords ("testpassword123") and creates API keys with real permissions (EXECUTE_GRAPH, READ_GRAPH). While intended for testing, this could pose security risks if the script is run in production environments or if test data persists beyond testing phases. Additionally, the script bypasses normal authentication flows by directly creating users in Supabase, which could be exploited if not properly secured.

⚡ Recommended focus areas for review

Security Risk

Hardcoded test password and potential credential exposure in test data creation. The script uses a fixed password for all test users and may create API keys with real permissions.

password = "testpassword123"  # Standard test password
name = faker.name()
Error Handling

Inconsistent error handling patterns throughout the script. Some methods continue on errors while others may fail silently, potentially leaving the database in an inconsistent state.

except Exception as supabase_error:
    print(f"Supabase user creation failed for {email}, using fallback: {supabase_error}")
    # Fall back to direct database creation
Resource Management

Missing proper cleanup mechanisms for created test data and potential database connection leaks if errors occur during execution.

try:
    creator = TestDataCreator()
    await creator.create_all_test_data()
finally:
    # Disconnect from database
    await prisma.disconnect()

Copy link

deepsource-io bot commented Jul 14, 2025

Here's the code health analysis summary for commits fde3533..a419f21. View details on DeepSource ↗.

Analysis Summary

AnalyzerStatusSummaryLink
DeepSource JavaScript LogoJavaScript✅ SuccessView Check ↗
DeepSource Python LogoPython✅ Success
❗ 11 occurences introduced
View Check ↗

💡 If you’re a repository administrator, you can configure the quality gates from the settings.

- Updated the test data creator to save generated data to JSON files in the frontend .test-data directory.
- Introduced a summary file that includes metadata about the generated test data.
- Adjusted user and agent creation limits to ensure consistent test data generation.
- Improved error handling during data saving processes to enhance reliability.
- This update aims to streamline the E2E testing framework and facilitate easier data management.
… data generation

- Added steps to set up Python 3.11 and install Poetry 1.6.1 in the CI workflow.
- Included a step to install backend dependencies using Poetry.
- Implemented a step to create E2E test data by running a dedicated script, improving the testing framework.
- Updated .gitignore to exclude the new .test-data directory in the frontend.
…2E test data script

- Eliminated the direct database creation of test profiles to streamline the test data generation process.
- This change simplifies the code and focuses on creating test library agents, enhancing maintainability.
… saving logic

- Removed the functionality to save generated test data to JSON files, simplifying the test data creation process.
- Adjusted user creation logic to ensure the first user has a predefined email for testing.
- Updated graph creation to include specific node types and links, enhancing the structure of generated graphs.
- This refactor aims to improve maintainability and focus on in-memory data generation for E2E tests.
- Introduced a new step in the CI workflow to generate E2E test data by executing a dedicated script within the backend container.
- This addition enhances the testing process by ensuring that relevant test data is available during the CI pipeline execution.
- Changed the backend container name from 'agpt-backend' to 'rest_server-1' in the CI workflow for generating E2E test data.
- This update ensures the correct container is referenced during the CI process, improving the reliability of test data generation.
…ment

- Added steps to set up Python 3.11 and install Poetry in the CI workflow.
- Implemented caching for Poetry dependencies to speed up the installation process.
- Updated the E2E test data generation step to ensure it runs within the backend directory.
- This enhancement improves the reliability and efficiency of the CI pipeline for backend testing.
…rom CI workflow

- Eliminated the steps for setting up Python 3.11 and installing Poetry in the CI workflow.
- Removed caching for Poetry dependencies and the installation of Python dependencies.
- This refactor streamlines the CI process by focusing on essential steps, improving overall efficiency.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🆕 Needs initial review
Status: No status
Development

Successfully merging this pull request may close these issues.

1 participant