Malcolm v25.04.0

[mmguero]

Malcolm v25.04.0 contains new features and improvements, component version updates, bug fixes, and other great stuff.

v25.03.1...v25.04.0

• ✨ Features and enhancements

  • add option to use external NetBox instance (Use external NetBox instance #597)

  • add -q/--quiet option for start/restart (Option for Quiet Start/Restart #656)

  • handle non-HTTPS arkime case (handle non-HTTPS arkime case #629)

  • lots of improvements to control.py and install.py for Kubernetes deployment

    • improved start/stop/wipe control script behavior
    • allow providing resource requests in manifests via YML file and command-line argument

    ...
    Kubernetes:
      -n, --namespace <string>
                            Kubernetes namespace
      --skip-persistent-volume-checks [SKIPPERVOLCHECKS]
                            Skip checks for PersistentVolumes/PersistentVolumeClaims in manifests (only for "start" operation with Kubernetes)
      --no-capture-pods [NOCAPTUREPODSSTART]
                            Do not deploy pods for traffic live capture/analysis (only for "start" operation with Kubernetes)
      --no-capabilities [NOCAPABILITIES]
                            Do not specify modifications to container capabilities (only for "start" operation with Kubernetes)
      --inject-resources [INJECTRESOURCES]
                            Inject container resources from kubernetes-container-resources.yml (only for "start" operation with Kubernetes)
      --image-source <string>
                            Source for container images (e.g., "ghcr.io/idaholab/malcolm"; only for "start" operation with Kubernetes)
      --image-tag <string>  Tag for container images (e.g., "25.04.0"; only for "start" operation with Kubernetes)
      --delete-namespace [DELETENAMESPACE]
                            Delete Kubernetes namespace (only for "wipe" operation with Kubernetes)
    ...
    

  • improvements to Malcolm's vanilla Kubernetes manifests

    • lowered the amount of storage for the persistent volumes in the AWS EFS example
    • replaced name label with app label for deployments in accordance with best practices

  • improve links on landing page for NetBox and auth to accurately reflect what Malcolm is using

  • added more smarts to the NGINX startup script to dynamically set up upstreams that may or may not exist based on enabled or disabled Malcolm features

  • fixed a minor issue in the script setting up Zeek intelligence updates where it would remove its own lockfile

• ✅ Component version updates
  • Alpine Linux v3.21
  • Arkime v5.6.3
  • Keycloak v26.2
  • NetBox v4.2.8
  • netbox-initializers v4.2.0
  • netbox-topology v4.2.1
  • Fluent Bit to v4.0.1
• 🐛 Bug fixes
  • API tokens created in NetBox still require authentication through NGINX reverse proxy (API tokens created in NetBox still require authentication through NGINX reverse proxy #383)
  • adjust Logstash health check so K8s liveness probe doesn't kill it (adjust logstash health check so kubernetes liveness probe doesn't kill it #630)
  • be more resilient in zeekctl status checks in zeekdeploy.sh (be more resilient in zeekctl status checks in zeekdeploy.sh #652)
  • in deployments with multiple zeek-live containers, each container's restarting causes the others to restart zeek (in deployments with multiple zeek-live containers, each container's restarting causes the others to restart zeek #651)
• 📄 Configuration changes (in environment variables in ./config/) for Malcolm and in control_vars.conf for Hedgehog Linux
  • replaced NETBOX_DISABLED with NETBOX_MODE in netbox-common.env for Use external NetBox instance #597
  • added NETBOX_URL in netbox-common.env for Use external NetBox instance #597
  • added NETBOX_TOKEN in netbox-secret.env for Use external NetBox instance #597
  • removed unused NETBOX_CRON variable from netbox-common.env
  • added LOGIN_REQUIRED, LOGIN_PERSISTENCE, and ISOLATED_DEPLOYMENT in netbox.env
  • added MALCOLM_NETWORK_INDEX_DEFAULT_PIPELINE, MALCOLM_NETWORK_INDEX_LIFECYCLE_NAME, MALCOLM_NETWORK_INDEX_LIFECYCLE_ROLLOVER_ALIAS, MALCOLM_OTHER_INDEX_DEFAULT_PIPELINE, MALCOLM_OTHER_INDEX_LIFECYCLE_NAME, MALCOLM_OTHER_INDEX_LIFECYCLE_ROLLOVER_ALIAS in opensearch.env for integrate customizations from Malcolm-Helm as options in vanilla Malcolm (part 1) #642; these are used to support customizations in the index templates, primarily for when using a remote Elasticsearch instance as the backing document store
  • added EXTRACTED_FILE_ENABLE_VTOT in zeek.env rather than just relying on the presence of VTOT_API2_KEY in zeek-secret.env
• 🧹 Code and project maintenance
  • various minor documenation improvements
  • improvements to build and appliance packaging scripts (Environment variable migration, netbox restore and building 25.03.1 api #640)
  • document customizing Malcolm with an additional output pipeline (document customizing Malcolm with an adidtional output pipeline #643)
  • overhaul "deploying Malcolm on AWS" documentation (overhaul "deploying Malcolm on AWS" documentation #655)
  • integrate customizations from Malcolm-Helm as options in vanilla Malcolm (part 1) (integrate customizations from Malcolm-Helm as options in vanilla Malcolm (part 1) #642)
  • put in version pinning for Python packages (put in version pinning for Python packages #644)
  • remove redundant storage of URLs in documents as artifact of NetBox enrichment
  • removed references to AWS client access and secret keys from packer_vars.json.example and documentation for building AWS AMIs (for security, these variables are now passed in via environment variables on the command line in the examples)

Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

---

This discussion was created from the release Malcolm v25.04.0.