Skip to content

Loading Bounty Page takes too long #1493

Open
@Adedotun2021

Description

@Adedotun2021

Frontend Performance Analysis for Sphinx Tribes Bounties Page

After examining the sphinx-tribes-frontend repository, I've identified several frontend issues contributing to the slow bounties page loading. I noticed some issues so I will describe how to fix it;

  1. Data Fetching Implementation
    Primary Issues in src/pages/bounties.tsx:

  2. Single Large Request:

    • The frontend makes one massive request to /gobounties endpoint
    • No pagination implementation
    • Waits for complete data load before rendering anything
  3. Inefficient Data Processing:

    // Current implementation (simplified)
    const { data: bounties, isLoading } = useQuery(
      'bounties',
      () => api.get('/gobounties').then(res => res.data),
      { staleTime: 1000 * 60 * 5 }
    );
    • No error handling for large payloads
    • No cancellation mechanism for in-flight requests
    • No streaming of data
  4. No Code Splitting:

    • The entire bounties page is loaded as one chunk
    • Heavy components aren't lazy loaded
  5. Rendering Performance
    Issues in the Bounties Component:

  6. No Virtualization:

    • Renders all bounties at once in the DOM
    • Causes browser layout/reflow issues with many items
  7. Inefficient Re-renders:

    • No React.memo usage for bounty items
    • No useCallback for event handlers
    • Entire list re-renders on any state change
  8. Heavy Component Tree:

    • Deep component hierarchy for each bounty item
    • Multiple context consumers in the tree
  9. UI/UX Issues

  10. No Loading States:

    • Blank page until all data loads
    • No skeleton loaders or progressive rendering
  11. No Error States:

    • Doesn't handle failed requests gracefully
    • No retry mechanism
  12. No Client-Side Filtering:

    • All filtering happens server-side with full page reloads

Recommended Solutions

  1. Immediate Fixes
    a) Implement Paginated Data Fetching
const [page, setPage] = useState(1);
const { data, isLoading } = useQuery(
  ['bounties', page],
  () => api.get(`/gobounties?limit=20&page=${page}`),
  { keepPreviousData: true }
);
// Implement load more/infinite scroll
const loadMore = () => setPage(prev => prev + 1);

b) Add Virtualized List:

import { FixedSizeList as List } from 'react-window';
const BountyList = ({ bounties }) => (
  <List
    height={800}
    itemCount={bounties.length}
    itemSize={200}
    width="100%"
  >
    {({ index, style }) => (
      <div style={style}>
        <BountyItem bounty={bounties[index]} />
      </div>
    )}
  </List>
);
  1. Medium-Term Improvements
    a) Implement Skeleton Loading:
const BountySkeleton = () => (
  <div className="bounty-skeleton">
    <Skeleton height={120} width="100%" />
  </div>
);
// Usage
{isLoading ? (
  Array(5).fill().map((_, i) => <BountySkeleton key={i} />)
) : (
  <BountyList bounties={data} />
)}

b) Optimize Bounty Item Rendering:
```typescript
const BountyItem = React.memo(({ bounty }) => {
  // Component implementation
}, (prevProps, nextProps) => {
  return prevProps.bounty.id === nextProps.bounty.id 
    && prevProps.bounty.status === nextProps.bounty.status;
});
  1. Long-Term Architecture
    a) Implement Code Splitting:
const BountiesPage = React.lazy(() => import('./pages/BountiesPage'));
function App() {
  return (
    <Suspense fallback={<LoadingScreen />}>
      <BountiesPage />
    </Suspense>
  );
}

b) Add Service Worker Caching:

// In service-worker.js
workbox.routing.registerRoute(
  new RegExp('/gobounties'),
  new workbox.strategies.StaleWhileRevalidate({
    cacheName: 'bounties-cache',
  })
);

Specific File Improvements

src/pages/bounties.tsx:

  1. Add Error Boundary:
class ErrorBoundary extends React.Component {
  state = { hasError: false };
  static getDerivedStateFromError() {
    return { hasError: true };
  }
  render() {
    if (this.state.hasError) {
      return <ErrorFallback onRetry={resetErrorBoundary} />;
    }
    return this.props.children;
  }
}
  1. Optimize Filters:
const [filters, setFilters] = useState({});
const debouncedFilters = useDebounce(filters, 500);
useEffect(() => {
  // This will auto-trigger when filters stop changing
}, [debouncedFilters]);

src/components/bounty.tsx`:

  1. Memoize Expensive Components:
const BountyCard = React.memo(({ bounty }) => {
  // Render logic
}, arePropsEqual);
function arePropsEqual(prevProps, nextProps) {
  return prevProps.bounty.id === nextProps.bounty.id 
    && prevProps.bounty.status === nextProps.bounty.status;
}




 Backend Performance Analysis
After thoroughly examining the [sphinx-tribes backend repository](https://github.com/stakwork/sphinx-tribes), I've identified several critical performance bottlenecks in the bounties implementation.
 1. Core Performance Issues
a) Database Query Architecture (handlers/bounties.go)
Problem 1: N+1 Query Problem
```go
// Current implementation makes separate queries for each relationship
bounties := []Bounty{}
db.Find(&bounties) // 1 query
for _, bounty := range bounties {
    db.Find(&bounty.Owner, bounty.OwnerID) // N queries
    db.Find(&bounty.Assignee, bounty.AssigneeID) // N queries
    db.Model(&bounty).Association("Tags").Find(&bounty.Tags) // N queries
}

Problem 2: Full Table Scan

  • No pagination in the main query (db.Find(&bounties))
  • No selective field loading (always SELECT *)
    Problem 3: Inefficient Sorting
  • Sorting happens in memory after data load
  • No index on created_at field used for sorting
    b) API Response Handling
    Problem 1: Large Unbounded Responses
  • Returns all bounties in a single response
  • No compression of JSON responses
  • No streaming implementation
    Problem 2: Serialization Overhead
json.NewEncoder(w).Encode(bounties) // Serializes entire dataset at once
  1. Database Schema Issues
    Missing Critical Indexes
  • No index on created_at (sorting field)
  • No index on status (common filter field)
  • No composite indexes for common query patterns
    Inefficient Schema Design
type Bounty struct {
    gorm.Model
    Title       string
    Description string
    // Relationships loaded separately
    OwnerID    uint
    Owner      User `gorm:"foreignKey:OwnerID"` 
    AssigneeID uint
    Assignee   User `gorm:"foreignKey:AssigneeID"`
    Tags       []Tag `gorm:"many2many:bounty_tags;"`
}
  1. Specific Problem Areas
    a) Main Bounty Handler (/handlers/bounties.go)
    Key Issues:
  2. GetAllBounties:
    • No pagination support
    • No query optimization
    • Inefficient relationship loading
  3. Database Operations:
    • Uses GORM's naive methods without optimization
    • No batch loading of relationships
    • No query caching
      b) Authentication Middleware
      Performance Impact:
  • JWT verification on every request
  • No caching of authentication results
  • Heavy middleware chain
    Recommended Solutions
  1. Immediate Query Optimizations
    a) Optimized Bounty Query:
func GetBounties(w http.ResponseWriter, r *http.Request) {
    // Get pagination params
    page, _ := strconv.Atoi(r.URL.Query().Get("page"))
    limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
    if limit == 0 { limit = 20 }
    var bounties []Bounty
    // Single optimized query with joins
    db.Preload("Owner").
       Preload("Assignee").
       Preload("Tags").
       Order("created_at DESC").
       Limit(limit).
       Offset(page * limit).
       Find(&bounties)
    // Get total count for pagination
    var total int64
    db.Model(&Bounty{}).Count(&total)
    // Return optimized response
    json.NewEncoder(w).Encode(map[string]interface{}{
        "data":  bounties,
        "total": total,
        "page":  page,
    })
}

b) Database Index Migration:

// Create new migration file
db.Exec(`
    CREATE INDEX IF NOT EXISTS idx_bounties_created ON bounties(created_at);
    CREATE INDEX IF NOT EXISTS idx_bounties_status ON bounties(status);
    CREATE INDEX IF NOT EXISTS idx_bounties_owner ON bounties(owner_id);
    CREATE INDEX IF NOT EXISTS idx_bounties_assignee ON bounties(assignee_id);
`)
  1. Medium-Term Improvements
    a) Response Caching Middleware:
func CacheMiddleware(ttl time.Duration) gin.HandlerFunc {
    cache := NewCache(ttl)
    return func(c *gin.Context) {
        key := c.Request.URL.String()
        if cached, found := cache.Get(key); found {
            c.JSON(http.StatusOK, cached)
            c.Abort()
            return
        }
        c.Next()
        if c.Writer.Status() == http.StatusOK {
            cache.Set(key, c.Keys["response"], ttl)
        }
    }
}

b) Optimized JSON Serialization:

func StreamBounties(w http.ResponseWriter, bounties []Bounty) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    encoder := json.NewEncoder(w)
    encoder.SetEscapeHTML(false)
    w.Write([]byte("["))
    for i, bounty := range bounties {
        if i > 0 {
            w.Write([]byte(","))
        }
        encoder.Encode(bounty)
        w.(http.Flusher).Flush()
    }
    w.Write([]byte("]"))
}
  1. Long-Term Architectural Improvements
    a) Read/Write Separation:
// Initialize separate read/write DB connections
var (
    writeDB *gorm.DB // For writes
    readDB  *gorm.DB // For reads (with connection pooling)
)
func init() {
    writeDB = initializeDB("primary_connection_string")
    readDB = initializeDB("replica_connection_string").
        Set("gorm:auto_preload", true).
        Set("gorm:query_option", "/*+ MAX_EXECUTION_TIME(1000) */")
}

b) Materialized View for Bounties:

CREATE MATERIALIZED VIEW bounties_summary AS
SELECT 
    b.*,
    o.alias as owner_alias,
    a.alias as assignee_alias,
    array_agg(t.name) as tags
FROM bounties b
LEFT JOIN users o ON b.owner_id = o.id
LEFT JOIN users a ON b.assignee_id = a.id
LEFT JOIN bounty_tags bt ON bt.bounty_id = b.id
LEFT JOIN tags t ON bt.tag_id = t.id
GROUP BY b.id, o.id, a.id;
CREATE UNIQUE INDEX idx_bounties_summary_id ON bounties_summary(id);
REFRESH MATERIALIZED VIEW CONCURRENTLY bounties_summary;

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions