Skip to content

Releases: idaholab/Malcolm

Malcolm v6.4.3

06 Dec 21:44
@mmguero mmguero
v6.4.3
46af12c
Compare
Choose a tag to compare
Malcolm v6.4.3

Malcolm v6.4.3 is a minor release containing enhancements, component version updates and bug fixes.

v6.4.2...v6.4.3

  • Enhancements

    • Import the NetBox Device Type Library on NetBox first run to populate manufacturers, device types, models and modules
    • idaholab/Malcolm#127 have install.py --configure ask about other storage locations for PCAP, Zeek logs and OpenSearch indices
    • idaholab/Malcolm#128 have install.py --configure prompt for Arkime to manage uploaded PCAP files or not

  • Component version updates

  • Fixes

    • Fix some bad links in the documentation and other minor documentation improvements
    • Fix idaholab/Malcolm#126, suricata logs show up in Arkime as "notip" for the protocol
    • Fix idaholab/Malcolm#129, filtering by rootId in Arkime returns no results
    • Fix Docker health checks for NetBox and supporting containers
    • Fix "read-only" version of nginx.conf
    • Tweaks to install.py memory recommendations

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 6
Loading

Malcolm v6.4.2

17 Nov 19:33
@mmguero mmguero
v6.4.2
62518b1
Compare
Choose a tag to compare
Malcolm v6.4.2

Malcolm v6.4.2 is a minor release containing a few component version updates (some addressing component vulnerabilities) and other improvements.

v6.4.1...v6.4.2

  • Component version updates
    • Zeek to v5.0.3 (this release fixes several security vulnerabilities in Zeek itself)
    • OpenSearch and OpenSearch Dashboards to v2.4.0
    • Logstash to v8.4.0
    • FileBeat to v8.5.1
    • NetBox to v3.3.8
  • Bug Fixes
    • Fix unhandled exceptions in API when certain API calls are made before data is indexed
  • Improvements
    • Added Zeek plugin to detect vulnerability to and exploitation attempts of CVE-2022-3602
    • Minor documentation fixes
    • Minor improvements to Docker container debug logging
    • Implemented caching of entropy calculations for DNS requests and TLS hostnames

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 6
Loading

Malcolm v6.4.1

03 Nov 21:52
@mmguero mmguero
v6.4.1
92dde18
Compare
Choose a tag to compare
Malcolm v6.4.1

Malcolm v6.4.1 is a minor release containing a few bug fixes, component version updates and other improvements.

v6.4.0...v6.4.1

  • Bug fixes
    • Zeek log files that have been renamed and are in the process of moving not caught correctly by Logstash (#121)
    • Hedgehog Arkime viewer node should use TLS (#122)
    • Recent changes to Elastic Common Schema needed adjustment (map number data type to long)
  • Component version updates
  • Improvements
    • On Hedgehog Linux, allow configuration of Arkime capture to use PCAP compression if desired
    • Changes to GitHub Docker image and ISO workflows, updating deprecated actions and features
    • Create corresponding net-map.json/Host and Subnet Name Mapping items in NetBox on when applicable
    • Remove unnecessary linux-headers- package from Zeek Docker image

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 6
Loading

Malcolm v6.4.0

19 Oct 18:54
@mmguero mmguero
v6.4.0
fd2dc25
Compare
Choose a tag to compare
Malcolm v6.4.0

Malcolm v6.4.0 features refactored documentation, the initial integration of NetBox (a network infrastructure resource modeling tool), several component version updates and other improvements and bug fixes.

Note that some changes involved in this release require some modifications to files used by docker-compose. Please run ./scripts/auth_setup and ./scripts/install.py --configure to ensure the appropriate new environment variables are set.

v6.3.0...v6.4.0

  • New features
    • initial NetBox integration (development ongoing, see #17)
  • Improvements
    • Documentation reformat/refactor
    • Use tini for Docker image init
    • Added support for s7comm_upload_download.log
    • Surface more options in install.py --configure, as well as minor tweaks
    • Update documentation report for results of ISO hardening
  • Component version updates
    • Arkime v4.0.1
      • Allow (optional) PCAP compression on Hedgehog
    • OpenSearch and OpenSearch Dashboards v2.3.0
    • Fluent Bit v1.9.9
    • Zeek v5.0.2
  • Bug fixes
    • verify capa signature hits are still being parsed/inserted correctly (#120)
    • Handle long integers in parsing bacnet_discovery and bacnet_property
    • Better enrichment of network.direction based on source and destination IP addresses

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 6
Loading

Malcolm v6.3.0

07 Sep 22:28
@mmguero mmguero
v6.3.0
cc83826
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v6.3.0

Malcolm v6.3.0 is a feature release with a number of new features, bug fixes and improvements. Of particular note is Malcolm's ability to now use another OpenSearch instance or cluster in lieu of its own local instance.

Note that the changes involved in #10 require modifications to files used by docker-compose. Please run ./scripts/auth_setup and ./scripts/install.py --configure to ensure the appropriate new environment variables are set.

v6.2.0...v6.3.0

  • New Features

    • Support remote OpenSearch instance/cluster as alternative to local containerized instance (#10)
    • Documentation and convenience scripts for configuring ingestion of third-party logs, and basic parsing/normalizing of Fluent-Bit's Windows event logs
    • S7comm Plus support and replaced Amazon S7comm parser with icsnpp-s7comm (#99)

  • Version Bumps

    • OpenSearch and OpenSearch Dashboards to v2.2.1
    • Zeek to v5.0.1
    • Spicy to v1.5.1
    • spicy-plugin to v1.3.17
    • YARA to v4.2.3
    • Capa to v4.0.1

  • Improvements

    • Major improvements to OPC UA Binary parser and supporting dashboards
    • Ensure that all containers are provided the same information about trusted CA certificates
    • changed list of "sensitive countries" to match U.S. Department of Energy Sensitive Country List
    • Increased maximum fields from 3,000 to 5,000
    • Standardized configuration and authentication for primary and secondary remote OpenSearch instances, and make sure that index templates are created on secondary remote OpenSearch instances
    • Expand and fix normalization of network.direction in lieu of using tags
    • Various tweaks and improvements to the install.py script for enabling/disabling some features

  • Bugs Fixed

    • fields could be missing in Arkime due to a large number of concurrent requests (#115)
    • mapper_parsing_exception, TCP flag parsing problem (cisagov#214)

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/download/.

Assets 7
Loading

Malcolm v6.2.0

03 Aug 20:31
@mmguero mmguero
v6.2.0
c2c3df0
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v6.2.0

Malcolm v6.2.0 is a feature release with a number of bug fixes and improvements. Of particular note is a major reworking of how a standalone instance of Malcolm (i.e., when not receiving traffic from a network sensor) analyzes "live" traffic. See the README for more information.

Note that the changes around #109 and #110 require changes to the files used by docker-compose. Please run ./scripts/auth_setup and ./scripts/install.py --configure to ensure the appropriate new environment variables are set.

v6.1.0...v6.2.0

  • Improvements

    • #109: break Zeek/Suricata into two containers: one for "live" capture and one for uploaded PCAPs
      • give option to disable capture interface hardware offloading and adjust ring buffer sizes for standalone Malcolm capture
      • Zeek and Suricata images are now configured to not drop privileges at init (in order to be able to set permissions for network capture), but supervisord will drop privileges for all of its child processes before they execute to maintain security posture
      • include headers needed to build Zeek af_packet plugin in Zeek docker container
      • updated README to describe methods for capturing local traffic with standalone Malcolm
      • same images will be used for zeek and zeek-live containers, as well as for suricata and suricata-live containers, respectively
      • use the same scripts zeekdeploy.sh to configure and run Zeek on both Hedgehog and in the Malcolm zeek docker images
      • prevent "live" and "non-live" Zeek containers from both trying to update intel indicators at the same time
    • Speed up build time by getting official Debian suricata packages from backports rather than building from source
    • Added Suricata rule update cron jobs
    • Added documentation (in the form of comments) to all docker-compose file variables

  • Bugs

    • Fix #107: expand action/result meaning in DNP3 (and other?) dashboards
      • Clean up some Nul values that could appear in Zeek logs
      • improve mapping of BACnet actions
    • Fix #108: export PCAP not working from Arkime sessions without "Arkime Sessions"
    • Fix #110: SFTP upload broken due to dollar sign(s) in openssl-encrypted password
      • prompt in install.py --configure whether or not to expose this port to external hosts
    • Fix an issue that could prevent some zeek logs from being parsed correctly if they contained non-ASCII charactters

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/download/.

Assets 7
Loading

Malcolm v6.1.0

13 Jul 13:48
@mmguero mmguero
v6.1.0
e116d27
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v6.1.0

Malcolm v6.1.0 is a feature release with a number of updates and improvements.

v6.0.1...v6.1.0

  • Bugs fixed

    • Zeek logs get reingested after container restart - (#101)
    • Added IPsec fields that were not being parsed
    • Fixed some dashboards that should have been using ECS field names
    • Split the STUN attribute type field on comma during stun.log parsing

  • Improvements

    • Malcolm's OpenSearch index template is now composed upon initialization with elements from the latest Elastic Common Schema release.
    • Replaced most instances of beats on Hedgehog Linux (with the exception of the Apache-licensed 7.10.2 filebeat which is compatible with OpenSearch) with Fluent Bit (see #102) for resource utilization monitoring, etc. and recreated dashboards referencing these metrics
    • Replaced Auditbeat file integrity checking module with AIDE for Hedgehog Linux
    • Added an optionally exposed (disabled by default) a TCP input endpoint to Malcolm to allow easier ingestion of other third-party logs not natively supported by Malcolm
    • Improvements to APIs for listing fields and indices
    • Removed old environment variable-configured Index State Management code as the new OpenSearch v2.1.0 release has nice UIs for both index state management and snapshot management

  • Version bumps of note

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/download/.

Assets 7
Loading

Malcolm v6.0.1

25 May 17:36
@mmguero mmguero
v6.0.1
5d83ca7
Compare
Choose a tag to compare
Malcolm v6.0.1

Malcolm v6.0.1 is a minor release updating some of Malcolm's core components and adding a couple of Zeek plugins for detecting recent CVEs.

v6.0.0...v6.0.1

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/download/.

Assets 7
Loading

Malcolm v6.0.0

13 May 21:14
@mmguero mmguero
v6.0.0
5b8c189
Compare
Choose a tag to compare
Malcolm v6.0.0

Malcolm v6.0.0 is a major release which incorporates Suricata as a data source for network traffic analysis in Malcolm alongside Zeek and Arkime. A team at BYU (@piercema, @aglad-eng, @Jarscott1, @n8hacks) recently completed their work on Suricata integration for their capstone project. This release includes their changes as well as some additional work by Malcolm's developer in integrating Suricata in other ways not covered in the scope of their project. This release also includes other bug fixes and improvements.

v5.2.11...v6.0.0

As the Malcolm project uses semantic versioning when choosing version numbers. This release required some pretty extensive remapping of Zeek fields in order for Zeek and Suricata to target the same naming conventions for common fields. This backwards-compatibility breaking change is the reason for bumping the major version number from 5 to 6. It is not recommended to attempt an upgrade from a previous release; a fresh install is strongly encouraged.

  • Features

    • Incorporate Suricata as a data source for network traffic analysis in both Malcolm and Hedgehog Linux
    • Added support for the GENISYS protocol

  • Improvements

    • Minor tweaks to the GitHub workflows for building the Malcolm installer ISO
    • Better fingerprinting of events during Logstash parsing in order to create a unique but reproducible hash for events in the case that duplicate data is indexed into Malcolm
    • All data sources (Arkime, Zeek and Suricata) now specify the data source (stored as event.provider, arkime, zeek and suricata, respectively) and the log type (stored as event.dataset, e.g., session, conn, alert, etc.) in order to facilitate filtering among various types of network metadata
    • The Malcolm REST API was improved to support POST operations for all of the calls which can accept a filter argument to allow for easier representation of filters as JSON objects
    • Reworked several dashboards, including the Overview, Security Overview, Zeek Notices and Signatures dashboards
    • Leave packages in place on the ISO-installed Malcolm and Hedgehog Linux environments in order to support mounting SMB shares from the Thunar GUI

  • Bug fixes

    • Fix #94: docker-compose | "function" has no attribute "get" (ubuntu 20.04 install)
    • Fix #96: DNP3 dashboard has invalid saved search syntax
    • Fix #97: virustotal file scanning broken (AttributeError: 'Namespace' object has no attribute 'vtotReqLimit')
    • Fix #98: BSAP RDB data parsed incorrectly

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/download/.

Contributors

n8hacks, Jarscott1, and 2 other contributors
Assets 7
Loading

Malcolm v5.2.11

27 Apr 21:19
@mmguero mmguero
v5.2.11
3d88836
Compare
Choose a tag to compare
Malcolm v5.2.11

Malcolm v5.2.11 is a minor release with a few user experience improvements and component version updates (some of which resolve potential security issues).

v5.2.10...v5.2.11

  • Addressing security vulnerabilities

    • bump Zeek to v4.2.1 addressing a potential Zeek buffer overflow vulnerability
    • Deserilization of Untrusted YML data - cisagov#207

  • Version bumps

  • Improvements

    • Resolve performance degredation when we went to OpenSearch 1.3 by using the G1GC garbage collector - #91
    • improve workflow for configuring Malcolm to run behind another reverse proxy (Caddy, Traefik, etc.) - #92
    • assign and display both event.provider and event.dataset in Arkime - #89
    • only show the controls for PCAP download from session details if there is actually a PCAP backing the session document #90 - #90
    • increase timeouts related to filebeat (see https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html) to be a little more forgiving for log files that take a long time to process - mmguero-dev/Malcolm@04b6084
    • strip build status badges from deployed copy of README.md
    • The install.py script will make use of the pythondialog module for user interaction (on Linux) if it is available
    • added link to Dashboards in the footer of Arkime's interface

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/download/.

Assets 7
Loading