fix: the mount on runpod must be "/workspace/data"

Author: TimPietruskyRunPod

docker-compose.yml

@@ -18,11 +18,12 @@ services:
       - "8888:8888" # Jupyter Lab (from base image)
       - "2222:22" # SSH access (from base image)
 
-    # Environment variables for training configuration
+
+        # Environment variables for training configuration
     environment:
       # Required credentials
       - HF_TOKEN=${HF_TOKEN}
-      # - WANDB_API_KEY=${WANDB_API_KEY}
+      - WANDB_API_KEY=${WANDB_API_KEY}
 
       # Training configuration (examples - customize as needed)
       - AXOLOTL_BASE_MODEL=TinyLlama/TinyLlama_v1.1
@@ -44,6 +45,7 @@ services:
       # - PUBLIC_KEY=${PUBLIC_KEY}
 
     # Volume mounts for persistent data
+    # IMPORTANT: Mount to /workspace/data, NOT /workspace (which overwrites everything!)
     volumes:
       - ./outputs:/workspace/data/axolotl-artifacts
       - ./configs:/workspace/fine-tuning/configs

docs/conventions.md

@@ -314,6 +314,34 @@ export AXOLOTL_GRADIENT_ACCUMULATION_STEPS="16"
 
 ## 🔧 Troubleshooting
 
+### Critical RunPod Issue
+
+#### ⚠️ **NEVER Mount Volumes to `/workspace`**
+
+```bash
+# ❌ WRONG - This will overwrite the entire Docker image structure
+volumes:
+  - ./data:/workspace
+
+# ✅ CORRECT - Mount to subdirectories only
+volumes:
+  - ./data:/workspace/data/axolotl-artifacts
+```
+
+**Why**: RunPod volume mounts to `/workspace` will completely overwrite:
+
+- `/workspace/axolotl/` (Axolotl installation)
+- `/workspace/fine-tuning/` (Your scripts)
+- All symlinks and directory structure
+
+**Symptoms**:
+
+- `ln: failed to create symbolic link '/workspace/axolotl/outputs': No such file or directory`
+- `/root/cloud-entrypoint.sh: line 93: /workspace/fine-tuning/autorun.sh: No such file or directory`
+- Infinite restart loop
+
+**Solution**: Always mount to `/workspace/data/` or other subdirectories, never `/workspace` directly.
+
 ### Common Issues
 
 #### 1. **Environment Variables Not Applied**