feat(core): Deploy v0.2.1 core services with KEDA and PV data source

[rebodutch]

Summary

This pull request deploys the core application services for version 0.2.1, activating the complete asynchronous data processing pipeline. Building upon the new chart architecture, this feature introduces the new Go-based worker, enables event-driven autoscaling with KEDA, and implements a dynamic data loading mechanism for the simulator using PersistentVolumes.

This marks a major milestone, bringing the write-pipeline from simulation to a fully functional, observable, and scalable deployment within the Kubernetes environment.

Key Changes

This feature branch implements the following key functionalities:

1. Upgraded Core Service Interfaces to v0.2.1

• The deployments for existing services (simulator, ingestor, analytics-api) have been updated to use the v0.2.1 container images and corresponding environment variables/configurations.

2. Introduced the Go Worker and KEDA Autoscaling

• New worker Deployment: A new Deployment for the Go-based Kafka consumer (worker) has been added to the write-pipeline.
• KEDA ScaledObject Integration: A ScaledObject is now deployed alongside the worker, configured to monitor the Kafka topic's consumer lag. This enables the worker deployment to automatically scale from minReplicas to maxReplicas based on real-time load, making the data pipeline resilient to traffic bursts.

3. Adopted the Modernized values.yaml Interface

• The safezone-core chart and its subcharts have been updated to consume variables from the new, centralized values.yaml structure (e.g., global.kafka, global.database).

4. Implemented Dynamic Data Loading via PV/PVC

• The simulator's data source is no longer static. It now mounts a PersistentVolumeClaim.
• This PVC uses a dynamic selector, which binds to different PersistentVolumes based on a type parameter passed from the environment-specific values.yaml (e.g., type: "smoke-test" or type: "full-data"). This allows for easy switching of simulation scenarios between environments.

5. Renamed Subcharts for Clarity

• To better reflect the data flow, the subcharts within safezone-core have been renamed from the generic producer/consumer to the more descriptive write-pipeline and read-pipeline.

Overall Impact

With this change, the entire write-pipeline is now fully deployed and operational. The system is no longer just a collection of individual services but a cohesive, event-driven architecture that can autonomously respond to load. This lays the final foundation for end-to-end testing and validation of the SafeZone application.