🚀 MCP Apache Spark History Server v0.1.0 Release
🚀 Initial Release Highlights
We're excited to announce the first official release of the MCP Apache Spark History Server! This groundbreaking tool enables AI-powered debugging and optimization of Apache Spark jobs through natural language interactions.
🤖 AI Integration
• Claude Desktop Integration - Seamless AI-powered Spark job analysis
• Amazon Q CLI Support - Native AWS AI assistant integration
• LangGraph Example with Ollama - Local AI model integration capabilities
⚙️ Configuration & Flexibility
• Configurable MCP Server Settings - Customizable server parameters
• Environment Variable Support - .env file configuration with default values
• Configurable Listen Address & NodePort - Flexible networking options
• SSL Verification Toggle - Support for self-signed certificates
🏗️ Development & Testing
• End-to-End Testing Suite - Comprehensive test coverage
• Local Kind Cluster Support - Easy local development setup
• Taskfile Build System - Streamlined development workflow
• Pre-commit Security Guardrails - Prevents hardcoded credentials
🔗 Platform Support
• EMR Persistent UI Support - Enhanced AWS EMR integration
• Glue SHS Compatibility - AWS Glue Spark History Server support
• KIRO Integration - Complete integration instructions
📦 Distribution
• PyPI Publishing - Easy installation via pip install mcp-apache-spark-history-server
• Docker Image Builds - Automated container publishing
• Helm Chart Support - Kubernetes deployment ready
👥 Core Contributors and Maintainers
• @vara-bonthu
• @nabuskey
• @andrewdkim47
📚 Documentation & Examples
• Comprehensive README with demo videos
• Step-by-step setup guides for multiple AI platforms
• Complete integration examples and use cases
Get Started: pip install mcp-apache-spark-history-server
What's Changed
- feat: SHS MCP Optimizaiton by @vara-bonthu in #1
- feat: Migrated to Taskfile by @vara-bonthu in #4
- feat: Claude desktop and Amazon Q CLI integration by @vara-bonthu in #6
- make mcp server settings configurable by @nabuskey in #8
- Update env var keys by @nabuskey in #9
- make run tasks interactive by @nabuskey in #11
- use task in workflows by @nabuskey in #13
- add e2e tests by @nabuskey in #12
- update documentation for q configuration by @nabuskey in #14
- Make listen address configurable by @nabuskey in #16
- add tasks for local kind deployment by @nabuskey in #17
- make service nodeport configurable by @nabuskey in #20
- add task to deploy shs to kind cluster with examples by @nabuskey in #21
- Updating Q CLI Setup Readme, and removing suggestion to --trust-all-tools by @andrewdkim47 in #22
- add complete kind setup by @nabuskey in #24
- break up task file by @nabuskey in #25
- pin dependencies to major versions by @nabuskey in #31
- add repo lint files by @nabuskey in #33
- ensure script fails when there's an error by @nabuskey in #35
- Adding guidelines for Glue SHS compatibility: Adding verify_ssl flag to allow user to disable SSL verification for SHS URLs that are self-signed by @andrewdkim47 in #34
- add demo video link by @nabuskey in #38
- ensure background tasks start by @nabuskey in #37
- feat: Langgraph example with SHS MCP and Ollama Models by @vara-bonthu in #40
- fix: Readme update with correct tools and added missing doc string by @vara-bonthu in #41
- Adding EMR persistent UI support by @andrewdkim47 in #43
- Restructuring files by @andrewdkim47 in #47
- Fixing EMR base url to include shs, upgrading pymodule versions, removing duplicate emr client by @andrewdkim47 in #48
- Adding wait for EMR Persistent UI Client status STARTING by @andrewdkim47 in #51
- Updating spark_id to app_id by @andrewdkim47 in #49
- Adding spark_history_server_mcp_launcher.sh to see MCP server output logs by @andrewdkim47 in #52
- Updating get to list when retrieving mutiple items, following native spark standards by @andrewdkim47 in #50
- update lock file and project name by @nabuskey in #60
- reflect repo name change in readme by @nabuskey in #57
- update dockerfile to run the mcp server correctly by @nabuskey in #62
- Update README.md by @vara-bonthu in #61
- rename helm chart to reflect repo name changes by @nabuskey in #64
- Updating KIRO integration instructions by @andrewdkim47 in #65
- Updating diagram and youtube video demo in README by @andrewdkim47 in #63
- Fix Q CLI doc by @nabuskey in #69
- Adding pre-commit guardrail for hardcoded username, password, or authtoken by @andrewdkim47 in #71
- add docker build for sem var tag by @nabuskey in #73
- add configurations for publishing to pypi by @nabuskey in #75
- update gitignore by @nabuskey in #76
- Support dot env file by @nabuskey in #72
- add default config values by @nabuskey in #77
- add pypi index by @nabuskey in #78
- use default pypi settings by @nabuskey in #79
New Contributors
- @vara-bonthu made their first contribution in #1
- @nabuskey made their first contribution in #8
- @andrewdkim47 made their first contribution in #22
Full Changelog: https://github.com/DeepDiagnostix-AI/mcp-apache-spark-history-server/commits/v0.1.0
