Skip to content

Robot Controller UI

Jump to bottom
Shushi Huang edited this page Jul 5, 2025 · 2 revisions

Robot Controller UI

A Next.js-based web interface for controlling and monitoring the McGill Robotics rover. This application provides real-time control capabilities for the rover's drive system, robotic arm, and camera feeds through an intuitive web interface.

Project Architecture

This is a Next.js 15 application built with TypeScript, Tailwind CSS, and React 19. The UI communicates with backend services through WebRTC for camera streams and WebSocket/HTTP APIs for robot control.

Technology Stack

  • Frontend: Next.js 15, React 19, TypeScript
  • Styling: Tailwind CSS 4
  • Charts: Recharts
  • Icons: Lucide React
  • HTTP Client: Axios
  • Real-time Communication: WebRTC, WebSockets

Folder Structure

src/
├── app/                          # Next.js App Router pages
│   ├── globals.css              # Global styles
│   ├── layout.tsx               # Root layout with navbar
│   ├── page.tsx                 # Home page (redirects to /status)
│   ├── arm/                     # Arm control page
│   ├── drive/                   # Drive control page
│   └── status/                  # System status page
│
├── components/                   # Reusable UI components
│   ├── icons/                   # Custom icon components
│   ├── layout/                  # Layout components (navbar, etc.)
│   ├── sections/                # Page-specific component sections
│   │   ├── arm/                 # Arm-related components
│   │   │   ├── control/         # Arm control interface & PS4 controller
│   │   │   ├── info/            # Arm telemetry & joint information
│   │   │   └── view/            # 3D arm visualizer
│   │   └── drive/               # Drive-related components
│   │       ├── camera/          # Camera feed & pan-tilt control
│   │       └── mobility/        # Drive controls & diagnostics
│   │           ├── control/     # Drive controls, headlights, speed
│   │           ├── info/        # Motor diagnostics, speedometer, charts
│   │           └── navigation/  # GPS panel
│   └── ui/                      # Generic UI components (buttons, etc.)
│
├── config/                      # Configuration files
│   ├── camera.ts               # Camera device configurations
│   └── network.ts              # Network settings (IP addresses)
│
├── context/                     # React contexts
│   └── NavbarContext.tsx       # Navbar state management
│
├── hooks/                       # Custom React hooks
│   ├── useBandwidthStats.ts    # Network bandwidth monitoring
│   ├── useCameraList.ts        # Camera device enumeration
│   └── useWebRTCStreams.ts     # WebRTC video streaming
│
├── lib/                        # Utility libraries
│   └── utils.ts                # Common utility functions
│
└── assets/                     # Static assets
    └── images/                 # Image files

public/                         # Static files served by Next.js
├── mcgillRobotics.svg         # Logo/favicon
└── gamepads/                  # PS4 controller button icons
    ├── base.svg, bumper.svg, circle.svg, etc.

Application Features

Drive Control (/drive)

  • Camera Feed: Real-time WebRTC video streaming from rover cameras
  • Pan-Tilt Control: D-pad interface for camera positioning
  • Mobility Controls:
    • Drive control interface
    • Headlight toggle
    • Maximum speed adjustment knob
  • Diagnostics:
    • Real-time speedometer cluster
    • Motor diagnostic panels with telemetry
    • Performance charts and graphs
  • Navigation: GPS panel for location tracking

Arm Control (/arm)

  • Visualizer: Real-time representation of arm position
  • PS4 Controller Interface: Visual gamepad for arm control
  • Joint Information:
    • Individual joint position and status
    • Coordinate display system
    • Speed control sliders
  • Command Log: History of sent commands
  • Performance Graphs: Joint movement and performance charts

Status Page (/status)

  • System health monitoring
  • Connection status indicators
  • Overall rover diagnostics

How the Website Works

Frontend Architecture

  1. Next.js App Router: Modern file-based routing system
  2. Component Architecture: Modular, reusable components organized by feature
  3. Real-time Communication:
    • WebRTC for low-latency camera streams
    • WebSocket connections for control commands
    • HTTP APIs for configuration and status

Camera System

  • WebRTC Streaming: Direct peer-to-peer video streaming
  • Device Detection: Automatic camera enumeration and selection
  • Bandwidth Monitoring: Real-time network performance tracking
  • Auto-reconnection: Automatic stream recovery on connection loss

Control Systems

  • Gamepad Integration: PS4 controller support for control
  • Real-time Feedback: Live telemetry and status updates
  • Safety Features: Speed limiting and emergency stops

Getting Started

Prerequisites

  • Node.js 18+
  • npm or yarn
  • Python 3.8+ (for backend services)

Installation

  1. Install frontend dependencies:
cd teleop/robot-controller-ui
npm install
  1. Install backend service dependencies:
# Install Python dependencies for services
pip install -r requirements.txt

Running the Application

1. Start Backend Services

Camera Service (WebRTC Server):

cd teleop/services/camera
python webrtc.py
  • Runs on port 8081
  • Provides WebRTC signaling and camera device enumeration
  • Handles video streaming from V4L2 devices

ROS Services:

# Start ROS2 nodes for robot control
cd teleop/services/ros
python ros_manager.py

2. Start Frontend Development Server

cd teleop/robot-controller-ui
npm run dev
  • Runs on port 3000 by default
  • Hot-reload enabled for development
  • Access at http://localhost:3000

3. Production Build

# Build for production
npm run build

# Start production server
npm start

Service Integration

Camera Service (teleop/services/camera/webrtc.py)

  • WebRTC Signaling Server: Handles peer connection establishment
  • Device Management: Lists and manages V4L2 camera devices
  • Bandwidth Monitoring: Provides real-time streaming statistics
  • CORS Support: Enables cross-origin requests from the web UI

ROS Integration (teleop/services/ros/)

  • ROS2 Bridge: Connects web interface to ROS2 ecosystem
  • Publisher/Subscriber Nodes: Handle command transmission and telemetry
  • Message Interfaces: Custom message types for rover communication

Network Configuration

Development Setup

  • Frontend: localhost:3000
  • WebRTC Service: localhost:8081
  • Camera IP: Configured in src/config/network.ts

Production Setup

Update src/config/network.ts to point to your rover's IP address:

export const CAMERAIP = "192.168.1.100" // Replace with rover IP

Supported Camera Devices

The application supports these camera types (configured in src/config/camera.ts):

  • USB 2.0 Camera
  • Logitech HD Webcam
  • VGA USB Camera
  • CC HD webcam

Development

Adding New Features

  1. Components: Add to appropriate section in src/components/sections/or in src/components/ui/ for reusable components
  2. Pages: Create new routes in src/app/
  3. Hooks: Add custom hooks in src/hooks/
  4. Services: Extend backend services in teleop/services/

Code Style

  • TypeScript: Strict type checking enabled
  • ESLint: Code linting with Next.js configuration
  • Tailwind CSS: Utility-first styling approach

Troubleshooting

Camera Issues

  • Ensure camera devices are not in use by other applications
  • Check V4L2 device permissions: ls -la /dev/video*
  • Verify WebRTC service is running on port 8081

Connection Issues

  • Check network configuration in src/config/network.ts
  • Ensure all backend services are running
  • Verify firewall settings for required ports

Performance Issues

  • Monitor bandwidth usage in the drive interface
  • Check browser console for WebRTC connection errors
  • Ensure adequate network bandwidth for video streaming

License

This project is part of the McGill Robotics rover system. Please refer to the main project license for usage terms.

Clone this wiki locally