🏀 SwishAI - Basketball Shot Analysis & Tracking

SwishAI is a Computer Vision application designed to analyze basketball shots from video footage automatically. Powered by YOLOv11 and a modern React + FastAPI stack, it tracks players, basketballs, and hoops in real-time, calculating shooting percentage and visualizing successful shots with dynamic overlays.

Support my work

If you find SwishAI useful, please consider supporting the development! 💙

✨ Key Features

Advanced Object Detection: Detects 5 distinct classes: Ball, Ball in Basket, Player, Basket, and Player Shooting.
Smart Scoring System: Automatically counts shots attempted vs. made to calculate real-time Field Goal Percentage (FG%).
Visual FX: Renders dynamic "pulse" animations and overlays when a basket is scored.
Configurable AI: Users can adjust Confidence Thresholds for each class via the UI to adapt to different lighting conditions.
Processing Modes - the video can be processed with 3 different modes:
- Full Tracking: Bounding boxes + Visual Effects + HUD.
- Stats & Effects: Clean view with only scoring effects and HUD.
- Stats Only: Minimalist overlay.
Performance Optimization: Includes a "Test Mode" (processes only the first 15s) and auto-cleanup mechanisms to manage server storage.

📸 Screenshots

Main Interface

Upload interface with processing mode selection

Advanced Model Settings

Fine-tune confidence thresholds for each detection class

Ready to Process

Video uploaded and ready for analysis

Processing in Progress

Real-time progress tracking with live statistics updates

Processing Complete

Analysis finished - ready to download the processed video

Output Example

Example of processed video with tracking overlays and statistics

🎥 Sample Output Video

Watch a full demonstration of SwishAI in action:

Click the image above to watch the full video output on YouTube

▶️ Watch on YouTube

🚀 Usage Guide

Open the App: Go to the local frontend URL.
Upload Video: Click the upload area to select a basketball video (MP4, MOV, AVI).
Configure Settings (Optional):
- Processing Mode: Choose between Full Tracking, Stats & Effects or Stats Only.
- Advanced Settings: Click "Advanced" to tweak confidence thresholds for specific classes (e.g., lower "Ball" threshold for dark videos).
- Test Mode: Check "Test Mode" to process only the first 15 seconds. Useful to check if the model works properly.
Start Analysis: Click the Start Analysis button. You will see a real-time progress bar and stats updating as the backend processes the video.
Download: Once complete, download the rendered video with overlays.

🛠 Tech Stack

Computer Vision & AI Core

YOLOv11s (Ultralytics): Latest SOTA object detection model
PyTorch: Deep learning framework
OpenCV (cv2): Video processing and drawing logic
NumPy: Matrix operations for game logic

Backend (BE)

FastAPI: High-performance, asynchronous Python web framework
Uvicorn: ASGI server implementation
Threading: Background video processing handling

Frontend (FE)

React 18: UI Library
Vite: Next-generation frontend tooling
Tailwind CSS: Utility-first CSS framework for styling
Lucide React: Modern icon set

📂 Project Architecture

SwishAI/
├── BE/                           # Backend (Python)
│   ├── basketball_training/      # Training scripts & artifacts
│   │   ├── weights/              # Stores best.pt after training
│   │   └── data_basketball.yaml  # Dataset configuration
│   ├── processed/                # Output videos (Auto-cleaned)
│   ├── uploads/                  # Raw user uploads (Auto-cleaned)
│   ├── app.py                    # FastAPI Server & Business Logic
│   ├── train_model.py            # YOLO Training Script
│   └── requirements.txt          # Python dependencies
├── FE/                           # Frontend (React)
│   ├── src/                      # Source code
│   │   └── App.jsx               # Main UI Component
│   ├── public/                   # Static assets
│   ├── tailwind.config.js        # Tailwind configuration
│   └── package.json              # Node dependencies
└── README.md

⚙️ Installation & Setup

Prerequisites

Python 3.10+
Node.js 18+ & npm
NVIDIA GPU (Optional but highly recommended for training/inference)

1. Backend Setup

Navigate to the backend directory:

cd BE

Create and activate a virtual environment:

python -m venv venv

# Windows:
.\venv\Scripts\activate

# Mac/Linux:
source venv/bin/activate

Install dependencies:

pip install -r requirements.txt

Start the API server:

python app.py

Server will run at http://localhost:8000.

2. Frontend Setup

Navigate to the frontend directory:

cd FE

Install packages:

npm install

Start the development server:

npm run dev

Client will run at http://localhost:5173.

3. Dataset & Model Training

Before running the app, you need to train the model (or use pre-trained weights).

Download Dataset: Get the basketball detection dataset from Roboflow Universe.

Configure Training:

Ensure the dataset is extracted into BE/basketball-detection-srfkd-1
Check train_model.py config class (the script is tailored for my hardware: GTX 1060 6GB - i7 6700k - 16gb ram):

DATASET_DIR = Path("basketball-detection-srfkd-1")
EPOCHS = 200
BATCH_SIZE = 8  # Adjust based on your VRAM

Run Training:

cd BE
python train_model.py

This script handles auto-validation, GPU checks, and custom augmentation.

🧠 Technical Logic & AI

Physics & Cooldowns

To prevent double-counting, the system implements physics-based cooldown logic (app.py):

Shot Cooldown (1.5s): Prevents the model from registering multiple shots for a single throwing motion.
Basket Cooldown (2.0s): Ensures the ball swishing through the net isn't counted twice in consecutive frames.

Custom Augmentation Strategy

The training script (train_model.py) uses specialized augmentation for sports footage:

HSV Saturation (0.7): Helps detect orange balls in varied lighting.
Shear (2.0°): Improves robustness against camera angles.
Mixup (0.15): Helps separating players in crowded scenes.

Detection Classes

ID	Class Name	Default Confidence
0	Ball	0.60
1	Ball in Basket	0.25
2	Player	0.70
3	Basket	0.70
4	Player Shooting	0.77

📊 Model Training & Performance

Dataset Information

SwishAI uses a custom-trained YOLOv11s model trained on the Basketball Detection Dataset from Roboflow Universe.

Dataset Source: Roboflow Universe - Basketball Detection
Dataset Size: ~10,000 annotated images
License: CC BY 4.0
Classes: 5 (Ball, Ball in Basket, Player, Basket, Player Shooting)
Training Date: March 17, 2025

Training Configuration

The model was trained using a custom training script (train_model.py) specifically optimized for consumer-grade hardware:

Hardware Specifications:

- GPU: NVIDIA GTX 1060 6GB
- CPU: Intel i7-6700K
- RAM: 16GB DDR4
- Training Duration: ~48 hours continuous

Training Parameters:

Model: YOLOv11s (small variant for efficiency)
Epochs: 200
Batch Size: 8 (optimized for 6GB VRAM)
Image Size: 640x640
Optimizer: AdamW
Learning Rate: 0.001 (with cosine decay)

Model Performance Metrics

Training & Validation Loss Curves

Confusion Matrix

Normalized Confusion Matrix

Key Performance Indicators (Final Model - Epoch 200):

mAP50: 0.909 (Mean Average Precision at IoU 0.5)
mAP50-95: 0.623 (Mean Average Precision from IoU 0.5 to 0.95)
Overall Precision: 0.878
Overall Recall: 0.861

Per-Class Performance:

Class	Precision	Recall	mAP50
Ball	0.80	0.88	0.847
Ball in Basket	0.51	0.36	0.932
Player	0.86	0.85	0.928
Basket	0.91	0.91	0.966
Player Shooting	0.76	0.34	0.873

Precision-Recall Curves

Training Script Details

The train_model.py script includes:

Hardware-specific optimizations for GTX 1060 6GB (batch size, workers, memory management)
Custom augmentation pipeline tailored for basketball footage (HSV saturation, shear, mixup)
Automatic validation split (80/20 train/val)
Early stopping and checkpoint saving
Mixed precision training (FP16) for faster training on lied VRAM

Note on Model Improvement: The current model achieves strong performance for real-time basketball analysis. However, further improvements are possible through:

Extended training (300+ epochs)

Larger model variants (YOLOv11m or YOLOv11l)

Additional data augmentation techniques

Fine-tuning on specific court environments

Ensemble methods or model fusion

Users with more powerful hardware (RTX 3080+, 24GB+ VRAM) can modify the training parameters in train_model.py to achieve higher accuracy.

📜 Credits

Developer: sPappalard
Dataset: Roboflow Universe - Basketball Detection
Frameworks: Ultralytics YOLO, FastAPI, React

Dataset Citation

@misc{
    basketball-detection-srfkd_dataset,
    title = { Basketball detection Dataset },
    type = { Open Source Dataset },
    author = { basketball },
    howpublished = { \url{ https://universe.roboflow.com/basketball-6vyfz/basketball-detection-srfkd } },
    url = { https://universe.roboflow.com/basketball-6vyfz/basketball-detection-srfkd },
    journal = { Roboflow Universe },
    publisher = { Roboflow },
    year = { 2025 },
    month = { mar },
    note = { visited on 2025-11-30 },
}

📄 License

This project is released under the GNU Affero General Public License v3.0 (AGPL-3.0).

Why AGPL-3.0? This project integrates Ultralytics YOLO, which is licensed under AGPL-3.0. As a derivative work, RotoAI inherits this license to ensure full compliance with the open-source terms of its dependencies.

What this means for you:

✅ Use: You can use this software for personal, research, or commercial purposes.
✅ Modify: You can modify the source code.
🔄 Share: If you distribute this software or host it as a network service (SaaS), you must disclose the source code of your modified version under the same AGPL-3.0 license.

Name		Name	Last commit message	Last commit date
Latest commit History 28 Commits
BE		BE
FE		FE
LICENSE		LICENSE
README.md		README.md

License

sPappalard/SwishAI

Folders and files

Latest commit

History

Repository files navigation