Releases: ntop/nDPI
Releases · ntop/nDPI
3.0 Stable
New Features
- nDPI now reports the protocol ASAP even when specific fields have not yet been dissected because such packets have not yet been observed. This is important for inline applications that can immediately act on traffic. Applications that need full dissection need to call the new API function ndpi_extra_dissection_possible() to check if metadata dissection has been completely performed or if there is more to read before declaring it completed.
- TLS (formerly identified as SSL in nDPI v2.x) is now dissected more deeply, certificate validity is extracted as well certificate SHA-1.
- nDPIreader can now export data in CSV format with option
-C - Implemented Sequence of Packet Length and Time (SPLT) and Byte Distribution (BD) as specified by Cisco Joy (https://github.com/cisco/joy). This allows malware activities on encrypted TLS streams. Read more at https://blogs.cisco.com/security/detecting-encrypted-malware-traffic-without-decryption
- Available as library and in
ndpiReaderwith option-J
- Available as library and in
- Promoted usage of protocol categories rather than protocol identifiers in order to classify protocols. This allows application protocols to be clustered in families and thus better managed by users/developers rather than using hundred of protocols unknown to most of the people.
- Added Inter-Arrival Time (IAT) calculation used to detect protocol misbehaviour (e.g. slow-DoS detection)
- Added data analysis features for computign metrics such as entropy, average, stddev, variance on a single and consistent place that will prevent when possible. This should ease traffic analysis on monitoring/security applications. New API calls have been implemented such as ndpi_data_XXX() to handle these calculations.
- Initial release of Python bindings available under nDPI/python.
- Implemented search of human readable strings for promoting data exfiltration detection
- Available as library and in
ndpiReaderwith option-e
- Available as library and in
- Fingerprints
- JA3 (https://github.com/salesforce/ja3)
- HASSH (https://github.com/salesforce/hassh)
- DHCP
- Implemented a library to serialize/deserialize data in both Type-Length-Value (TLV) and JSON format
- Used by nProbe/ntopng to exchange data via ZMQ
New Supported Protocols and Services
- DTLS (i.e. TLS over UDP)
- Hulu
- TikTok/Musical.ly
- WhatsApp Video
- DNSoverHTTPS
- Datasaver
- Line protocol
- Google Duo and Hangout merged
- WireGuard VPN
- IMO
- Zoom.us
Improvements
- TLS
- Organizations
- Ciphers
- Certificate analysis
- Added PUBLISH/SUBSCRIBE methods to SIP
- Implemented STUN cache to enhance matching of STUN-based protocols
- Dissection improvements
- Viber
- AmazonVideo
- SnapChat
- FTP
- QUIC
- OpenVPN support for UDP-based VPNs
- Facebook Messenger mobile
- Various improvements for STUN, Hangout and Duo
- Added new categories: CUSTOM_CATEGORY_ANTIMALWARE, NDPI_PROTOCOL_CATEGORY_MUSIC, NDPI_PROTOCOL_CATEGORY_VIDEO, NDPI_PROTOCOL_CATEGORY_SHOPPING, NDPI_PROTOCOL_CATEGORY_PRODUCTIVITY and NDPI_PROTOCOL_CATEGORY_FILE_SHARING
- Added NDPI_PROTOCOL_DANGEROUS classification
Fixes
- Fixed the dissection of certain invalid DNS responses
- Fixed Spotify dissection
- Fixed false positives with FTP and FTP_DATA
- Fix to discard STUN over TCP flows
- Fixed MySQL dissector
- Fix category detection due to missing initialization
- Fix DNS rsp_addr missing in some tiny responses
- Various hardening fixes
2.8 Stable
New Supported Protocols and Services
- Added Modbus over TCP dissector
Improvements
- Wireshark Lua plugin compatibility with Wireshark 3
- Improved MDNS dissection
- Improved HTTP response code handling
- Full dissection of HTTP responses
Fixes
- Fixed false positive mining detection
- Fixed invalid TCP DNS dissection
- Releasing buffers upon
reallocfailures - ndpiReader: Prevents references after free
- Endianness fixes
- Fixed IPv6 HTTP traffic dissection
- Fixed H.323 detection
Other
- Disabled ookla statistics which need to be improved
- Support for custom protocol files of arbitrary length
- Update radius.c to RFC2865
2.6 Stable
New Supported Protocols and Services
- New Bitcoin, Ethereum, ZCash, Monero dissectors all identified as Mining
- New Signal.org dissector
- New Nest Log Sink dissector
- New UPnP dissector
- Added support for SMBv1 traffic, split from SMBv23
Improvements
- Improved Skype detection, merged Skype call in/out into Skype Call
- Improved heuristics for Skype, Teredo, Netbios
- Improved SpeedTest (Ookla) detection
- Improved WhatsApp detection
- Improved WeChat detection
- Improved Facebook Messenger detection
- Improved Messenger/Hangout detection
- Improved SSL detection, prevent false positives
- Improved guess for UDP protocols
- Improved STUN detection
- Better Hyperscan integration
- Added more Ubuntu servers
- Added missing categorization with giveup/guess
- Optimisations for TCP flows that do not start with a SYN packet (early giveup)
Fixes
- Fixed eDonkey false positives
- Fixed Dropbox dissector
- Fixed Spotify dissector
- Fixed custom protocol loading
- Fixed missing Application Data packet for TLS
- Fixed buffer overflows
- Fixed custom categories match by IP
- Fixed category field not accounted in ndpi_get_proto_category
- Fixed null pointer dereference in ndpi_detection_process_packet
- Fixed compilation on Mac
Other
- Deb and RPM packages: ndpi with shared libraries and binaries, ndpi-dev with headers and static libraries
- Protocols now have an optional subprotocol: Spotify cannot have subprotocols, DNS can (DNS.Spotify)
New API functions:
- ndpi_fill_ip_protocol_category() to handle ICMP flows category
- ndpi_flowv4_flow_hash() and ndpi_flowv6_flow_hash() to support the Community ID Flow Hashing (https://github.com/corelight/community-id-spec)
- ndpi_protocol2id() to print the protocol as ID
- ndpi_get_custom_category_match() to search host in custom categories
- Changed ndpi_detection_giveup() API: guess is now part of the call
- Added DPDK support to ndpiReader
- Removed Musical.ly protocol (service no longer used)
- Custom categories have now priority over protocol related categories
- Improved clang support
2.4 Stable
New Supported Protocols and Services
- Showmax.com
- Musical.ly
- RapidVideo
- VidTO streaming service
- Apache JServ Protocol
- Facebook Messenger
- FacebookZero protocol
Improvements
- Improved YouTube support
- Improved Netflix support
- Updated Google Hangout detection
- Updated Twitter address range
- Updated Viber ports, subnet and domain
- Updated AmazonVideo detection
- Updated list of FaceBook sites
- Initial Skype in/out support
- Improved Tor detection
- Improved hyperscan support and category definition
- Custom categories loading, extended ndpiReader (
-c <file>) for loading name-based categories
Fixes
- Fixes for Instagram flows classified as Facebook
- Fixed Spotify detection
- Fixed minimum packet payload length for SSDP
- Fixed length check in MSN, x-steam-sid, Tor certificate name
- Increase client's maximum payload length for SSH
- Fixed end-of-line bounds handling
- Fixed substring matching
- Fix for handling IP address based custom categories
- Repaired wrong timestamp calculation
- Fixed memory leak
- Optimized memory usage
Other/Changes
- New API calls:
ndpi_set_detection_preferences()ndpi_load_hostname_category()ndpi_enable_loaded_categories()ndpi_fill_protocol_category()ndpi_process_extra_packet()
- Skype CallIn/CallOut are now set as Skype.SkypeCallOut Skype.SkypeCallIn
- Added support for SMTPS on port 587
- Changed RTP from VoIP to Media category
- Added site unavailable category
- Added custom categories CUSTOM_CATEGORY_MINING, CUSTOM_CATEGORY_MALWARE, CUSTOM_CATEGORY_ADVERTISEMENT, CUSTOM_CATEGORY_BANNED_SITE
- Implemented hash-based categories
- Converted some not popular protocols to NDPI_PROTOCOL_GENERIC with category detection
2.2 Stable
2.2. stable nDPI release
