perf(backend/db): Optimize StoreAgent and Creator views with database indexes and materialized views

[Swiftyos]

Summary

Performance optimization for the platform's store and creator functionality by adding targeted database indexes and implementing materialized views to reduce query execution time.

Changes 🏗️

Database Performance Optimizations:

• Added strategic database indexes for StoreListing, StoreListingVersion, StoreListingReview, AgentGraphExecution, and Profile tables
• Implemented materialized views (mv_agent_run_counts, mv_review_stats) to cache expensive aggregation queries
• Optimized StoreAgent and Creator views to use materialized views and improved query patterns
• Added automated refresh function with 15-minute scheduling for materialized views (when pg_cron extension is available)

Key Performance Improvements:

• Filtered indexes on approved store listings to speed up marketplace queries
• GIN index on categories for faster category-based searches
• Composite indexes for common query patterns (e.g., listing + version lookups)
• Pre-computed agent run counts and review statistics to eliminate expensive aggregations

Checklist 📋

For code changes:

• I have clearly listed my changes in the PR description
• I have made a test plan
• I have tested my changes according to the test plan:
  • Verified migration runs successfully without errors
  • Confirmed materialized views are created and populated correctly
  • Tested StoreAgent and Creator view queries return expected results
  • Validated automatic refresh function works properly
  • Confirmed rollback migration successfully removes all changes

For configuration changes:

• .env.example is updated or already compatible with my changes
• docker-compose.yml is updated or already compatible with my changes
• I have included a list of my configuration changes in the PR description (under Changes)

Note: No configuration changes were required as this is purely a database schema optimization.

[github-actions]

This PR targets the master branch but does not come from dev or a hotfix/* branch.

Automatically setting the base branch to dev.

[qodo-code-review]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Potential Bug The wait=False parameter in the client.run() call may cause issues as the function doesn't handle asynchronous responses properly. The code expects an immediate result but the API call is configured as non-blocking. output: FileOutput | list[FileOutput] = client.run( # type: ignore model_name, input=input_params, wait=False, ) Incomplete Implementation The NodeKeyValueInput component was refactored to use NodeGenericInputField for values, but the updateKeyValuePairs function doesn't properly handle the new value structure, potentially causing data loss or incorrect updates. handleInputChange={(_, newValue) => updateKeyValuePairs( keyValuePairs.toSpliced(index, 1, { key: key, value: newValue, }), ) } Missing Variable In the _file_to_data_uri function, the mime_type variable is used but not defined after the refactoring. It should be using the get_mime_type function that was added. def _file_to_data_uri(path: Path) -> str: mime_type = get_mime_type(str(path)) b64 = base64.b64encode(path.read_bytes()).decode("utf-8") return f"data:{mime_type};base64,{b64}"

[deepsource-io]

Here's the code health analysis summary for commits 243400e..0b87663. View details on DeepSource ↗.

Analysis Summary

Analyzer	Status	Summary	Link
JavaScript	✅ Success		View Check ↗
Python	✅ Success	❗ 3 occurences introduced 🎯 1 occurence resolved	View Check ↗

---

💡 If you’re a repository administrator, you can configure the quality gates from the settings.

[AutoGPT-Agent]

Thanks for your work on optimizing the database views! Before this can be merged, there are a few issues that need to be addressed:

1. PR Description: Please add a clear description of what these changes are doing and why they're needed. Your PR makes significant database optimizations, but there's no explanation of the problem you're solving or how these changes help.

2. Checklist: The PR checklist is completely empty. Please fill out the appropriate sections of the checklist or remove sections that don't apply. Since this involves database schema changes, please include your test plan.

3. PR Title: The PR title needs to follow our conventional commit format. For this type of change, something like perf(backend): optimize store agent and creator views would be appropriate.

The technical changes look solid - creating indexes, materialized views, and optimizing the Creator and StoreAgent views should improve query performance. I especially like the addition of the refresh function with concurrency handling. Just need the PR documentation to match the quality of the code changes.

[AutoGPT-Agent]

Thank you for submitting this PR with database optimizations. Before we can merge it, there are several issues that need to be addressed:

1. PR Title Format: Please update the title to follow our conventional commit format, such as fix(backend): optimize store agent and creator views

2. Missing PR Description: Please complete the PR description explaining:

   • Why these database optimizations are needed
   • What performance improvements are expected
   • Any risks or considerations with these changes

3. Incomplete Checklist: Please check off all the relevant items in the checklist or update it to match what you've done to validate these changes

4. Test Plan: For database changes like this, include how you've tested that these optimizations work correctly and don't break existing functionality

The changes themselves look like good performance improvements with:

• Creating appropriate indexes
• Adding materialized views for frequently accessed data
• Implementing refresh functions with concurrency handling
• Properly handling rollback scenarios

However, we need more context about the purpose and validation of these changes before merging.

[netlify]

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	0b87663
🔍 Latest deploy log	https://app.netlify.com/projects/auto-gpt-docs/deploys/686fd27e852909000824db76

[Swiftyos]

!deploy

[github-actions]

🚀 Deploying PR #10084 to development environment...

[Swiftyos]

❌ Preview Environment Deployment Failed

🚨 The preview environment deployment encountered critical errors and has been rolled back.

Cleanup Completed:

• ☸️ Kubernetes namespace deleted
• 🗃️ Database schema dropped
• 🔌 All services terminated

Please check the workflow logs for details and try again.

[ntindle]

Double check this won’t break the admin pages

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[github-actions]

🧹 Auto-undeploying: PR closed with active deployment. Cleaning up development environment for PR #10084.

[Swiftyos]

🧹 Preview Environment Cleaned Up

All resources for PR #10084 have been removed:

• ☸️ Kubernetes namespace deleted
• 🗃️ Database schema pr_10084 dropped

Cleanup completed successfully.