Loading Bounty Page takes too long

[Adedotun2021]

Frontend Performance Analysis for Sphinx Tribes Bounties Page

After examining the sphinx-tribes-frontend repository, I've identified several frontend issues contributing to the slow bounties page loading. I noticed some issues so I will describe how to fix it;

1. Data Fetching Implementation
   Primary Issues in src/pages/bounties.tsx:

2. Single Large Request:

   • The frontend makes one massive request to /gobounties endpoint
   • No pagination implementation
   • Waits for complete data load before rendering anything

3. Inefficient Data Processing:

   // Current implementation (simplified)
   const { data: bounties, isLoading } = useQuery(
     'bounties',
     () => api.get('/gobounties').then(res => res.data),
     { staleTime: 1000 * 60 * 5 }
   );

   • No error handling for large payloads
   • No cancellation mechanism for in-flight requests
   • No streaming of data

4. No Code Splitting:

   • The entire bounties page is loaded as one chunk
   • Heavy components aren't lazy loaded

5. Rendering Performance
   Issues in the Bounties Component:

6. No Virtualization:

   • Renders all bounties at once in the DOM
   • Causes browser layout/reflow issues with many items

7. Inefficient Re-renders:

   • No React.memo usage for bounty items
   • No useCallback for event handlers
   • Entire list re-renders on any state change

8. Heavy Component Tree:

   • Deep component hierarchy for each bounty item
   • Multiple context consumers in the tree

9. UI/UX Issues

10. No Loading States:

    • Blank page until all data loads
    • No skeleton loaders or progressive rendering

11. No Error States:

    • Doesn't handle failed requests gracefully
    • No retry mechanism

12. No Client-Side Filtering:

    • All filtering happens server-side with full page reloads

Recommended Solutions

1. Immediate Fixes
   a) Implement Paginated Data Fetching

const [page, setPage] = useState(1);
const { data, isLoading } = useQuery(
  ['bounties', page],
  () => api.get(`/gobounties?limit=20&page=${page}`),
  { keepPreviousData: true }
);
// Implement load more/infinite scroll
const loadMore = () => setPage(prev => prev + 1);

b) Add Virtualized List:

import { FixedSizeList as List } from 'react-window';
const BountyList = ({ bounties }) => (
  <List
    height={800}
    itemCount={bounties.length}
    itemSize={200}
    width="100%"
  >
    {({ index, style }) => (
      <div style={style}>
        <BountyItem bounty={bounties[index]} />
      </div>
    )}
  </List>
);

2. Medium-Term Improvements
   a) Implement Skeleton Loading:

const BountySkeleton = () => (
  <div className="bounty-skeleton">
    <Skeleton height={120} width="100%" />
  </div>
);
// Usage
{isLoading ? (
  Array(5).fill().map((_, i) => <BountySkeleton key={i} />)
) : (
  <BountyList bounties={data} />
)}

b) Optimize Bounty Item Rendering:
```typescript
const BountyItem = React.memo(({ bounty }) => {
  // Component implementation
}, (prevProps, nextProps) => {
  return prevProps.bounty.id === nextProps.bounty.id 
    && prevProps.bounty.status === nextProps.bounty.status;
});

3. Long-Term Architecture
   a) Implement Code Splitting:

const BountiesPage = React.lazy(() => import('./pages/BountiesPage'));
function App() {
  return (
    <Suspense fallback={<LoadingScreen />}>
      <BountiesPage />
    </Suspense>
  );
}

b) Add Service Worker Caching:

// In service-worker.js
workbox.routing.registerRoute(
  new RegExp('/gobounties'),
  new workbox.strategies.StaleWhileRevalidate({
    cacheName: 'bounties-cache',
  })
);

Specific File Improvements

src/pages/bounties.tsx:

1. Add Error Boundary:

class ErrorBoundary extends React.Component {
  state = { hasError: false };
  static getDerivedStateFromError() {
    return { hasError: true };
  }
  render() {
    if (this.state.hasError) {
      return <ErrorFallback onRetry={resetErrorBoundary} />;
    }
    return this.props.children;
  }
}

2. Optimize Filters:

const [filters, setFilters] = useState({});
const debouncedFilters = useDebounce(filters, 500);
useEffect(() => {
  // This will auto-trigger when filters stop changing
}, [debouncedFilters]);

src/components/bounty.tsx`:

1. Memoize Expensive Components:

const BountyCard = React.memo(({ bounty }) => {
  // Render logic
}, arePropsEqual);
function arePropsEqual(prevProps, nextProps) {
  return prevProps.bounty.id === nextProps.bounty.id 
    && prevProps.bounty.status === nextProps.bounty.status;
}




 Backend Performance Analysis
After thoroughly examining the [sphinx-tribes backend repository](https://github.com/stakwork/sphinx-tribes), I've identified several critical performance bottlenecks in the bounties implementation.
 1. Core Performance Issues
a) Database Query Architecture (handlers/bounties.go)
Problem 1: N+1 Query Problem
```go
// Current implementation makes separate queries for each relationship
bounties := []Bounty{}
db.Find(&bounties) // 1 query
for _, bounty := range bounties {
    db.Find(&bounty.Owner, bounty.OwnerID) // N queries
    db.Find(&bounty.Assignee, bounty.AssigneeID) // N queries
    db.Model(&bounty).Association("Tags").Find(&bounty.Tags) // N queries
}

Problem 2: Full Table Scan

• No pagination in the main query (db.Find(&bounties))
• No selective field loading (always SELECT *)
  Problem 3: Inefficient Sorting
• Sorting happens in memory after data load
• No index on created_at field used for sorting
  b) API Response Handling
  Problem 1: Large Unbounded Responses
• Returns all bounties in a single response
• No compression of JSON responses
• No streaming implementation
  Problem 2: Serialization Overhead

json.NewEncoder(w).Encode(bounties) // Serializes entire dataset at once

2. Database Schema Issues
   Missing Critical Indexes

• No index on created_at (sorting field)
• No index on status (common filter field)
• No composite indexes for common query patterns
  Inefficient Schema Design

type Bounty struct {
    gorm.Model
    Title       string
    Description string
    // Relationships loaded separately
    OwnerID    uint
    Owner      User `gorm:"foreignKey:OwnerID"` 
    AssigneeID uint
    Assignee   User `gorm:"foreignKey:AssigneeID"`
    Tags       []Tag `gorm:"many2many:bounty_tags;"`
}

3. Specific Problem Areas
   a) Main Bounty Handler (/handlers/bounties.go)
   Key Issues:
4. GetAllBounties:
   • No pagination support
   • No query optimization
   • Inefficient relationship loading
5. Database Operations:
   • Uses GORM's naive methods without optimization
   • No batch loading of relationships
   • No query caching
     b) Authentication Middleware
     Performance Impact:

• JWT verification on every request
• No caching of authentication results
• Heavy middleware chain
  Recommended Solutions

1. Immediate Query Optimizations
   a) Optimized Bounty Query:

func GetBounties(w http.ResponseWriter, r *http.Request) {
    // Get pagination params
    page, _ := strconv.Atoi(r.URL.Query().Get("page"))
    limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
    if limit == 0 { limit = 20 }
    var bounties []Bounty
    // Single optimized query with joins
    db.Preload("Owner").
       Preload("Assignee").
       Preload("Tags").
       Order("created_at DESC").
       Limit(limit).
       Offset(page * limit).
       Find(&bounties)
    // Get total count for pagination
    var total int64
    db.Model(&Bounty{}).Count(&total)
    // Return optimized response
    json.NewEncoder(w).Encode(map[string]interface{}{
        "data":  bounties,
        "total": total,
        "page":  page,
    })
}

b) Database Index Migration:

// Create new migration file
db.Exec(`
    CREATE INDEX IF NOT EXISTS idx_bounties_created ON bounties(created_at);
    CREATE INDEX IF NOT EXISTS idx_bounties_status ON bounties(status);
    CREATE INDEX IF NOT EXISTS idx_bounties_owner ON bounties(owner_id);
    CREATE INDEX IF NOT EXISTS idx_bounties_assignee ON bounties(assignee_id);
`)

2. Medium-Term Improvements
   a) Response Caching Middleware:

func CacheMiddleware(ttl time.Duration) gin.HandlerFunc {
    cache := NewCache(ttl)
    return func(c *gin.Context) {
        key := c.Request.URL.String()
        if cached, found := cache.Get(key); found {
            c.JSON(http.StatusOK, cached)
            c.Abort()
            return
        }
        c.Next()
        if c.Writer.Status() == http.StatusOK {
            cache.Set(key, c.Keys["response"], ttl)
        }
    }
}

b) Optimized JSON Serialization:

func StreamBounties(w http.ResponseWriter, bounties []Bounty) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    encoder := json.NewEncoder(w)
    encoder.SetEscapeHTML(false)
    w.Write([]byte("["))
    for i, bounty := range bounties {
        if i > 0 {
            w.Write([]byte(","))
        }
        encoder.Encode(bounty)
        w.(http.Flusher).Flush()
    }
    w.Write([]byte("]"))
}

3. Long-Term Architectural Improvements
   a) Read/Write Separation:

// Initialize separate read/write DB connections
var (
    writeDB *gorm.DB // For writes
    readDB  *gorm.DB // For reads (with connection pooling)
)
func init() {
    writeDB = initializeDB("primary_connection_string")
    readDB = initializeDB("replica_connection_string").
        Set("gorm:auto_preload", true).
        Set("gorm:query_option", "/*+ MAX_EXECUTION_TIME(1000) */")
}

b) Materialized View for Bounties:

CREATE MATERIALIZED VIEW bounties_summary AS
SELECT 
    b.*,
    o.alias as owner_alias,
    a.alias as assignee_alias,
    array_agg(t.name) as tags
FROM bounties b
LEFT JOIN users o ON b.owner_id = o.id
LEFT JOIN users a ON b.assignee_id = a.id
LEFT JOIN bounty_tags bt ON bt.bounty_id = b.id
LEFT JOIN tags t ON bt.tag_id = t.id
GROUP BY b.id, o.id, a.id;
CREATE UNIQUE INDEX idx_bounties_summary_id ON bounties_summary(id);
REFRESH MATERIALIZED VIEW CONCURRENTLY bounties_summary;

[Adedotun2021]

@tomastiminskas please review