Release 1.10.1

Fix various issues found by code analysis tools, a few compiler warnings removed, undefined bit shift behavior fixed, parameter memory leaks plugged and documentation updates.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dnscap/dnscap-1.10.1.tar.gz
sha256: f919e6a8494e35b4a3428999fe4665ecb35c2ae1452d0c35da823f608c77a84f

Packages are available at: https://dev.dns-oarc.net/packages/

Fixes:

• dump_dns: Remove usage of strcpy() and use snprintf() instead of sprintf()
• bpft:
  • Use text_ptr->len to store length of generated text
  • Use memcpy() instead of strcat()
  • Remove unneeded realloc() and strcpy()
• plugins/cryptopan: Fix strict-aliasing warnings
• network: Rework part of dl_pkt() to remove usage of strcpy() and use snprintf() instead of sprintf()
• plugins/anonaes128: Use a6 as dest when copying v4 addresses for readability and code analysis
• plugins/cryptopan: Run first pass separate to eliminate a 32bit shift by 32 (undefined behavior)
• plugins/cryptopant: Fix memory leak of keyfile if -k is specified more then once

Documentation:

• Update README.md with correction to building from git and note about PCAP on OpenBSD
• Fix #190: Update link to libbind source

074923c Funding
5d2e84c libbind
8ee9f2a Travis-CI
6babd09 Fixes
bb2d1c7 README, compile warnings
0d9cd9c LGTM, Travis-CI

Release 1.10.0

This release adds a new plugin type "filter" and 5 new plugins that can do anonymization, deanonymization and masking of the IP addresses.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dnscap/dnscap-1.10.0.tar.gz
sha256: 00f4b04a179e2a7c81a426c72ab228efe408868c1caf0c6cb0aa7906055222f4

Packages are available at: https://dev.dns-oarc.net/packages/

New features:

• Check plugins for pluginname_type() which returns enum plugin_type, if missing the plugin is counted as an "output" plugin
• New plugin type "filter" which calls pluginname_filter() prior of outputting any data or calling of "output" plugins, if the new function returns non-zero then the packet is filtered out (dropped)
• New extension DNSCAP_EXT_SET_IADDR that gives access to a function for setting the from and to IP addresses both in the extracted data and the wire

New plugins:

• anonaes128: Anonymize IP addresses using AES128
• anonmask: Pseudo-anonymize IP addresses by masking them
• cryptopan: Anonymize IP addresses using an extension to Crypto-PAn (College of Computing, Georgia Tech) made by David Stott (Lucent)
• cryptopant: Anonymize IP addresses using cryptopANT, a different implementation of Crypto-PAn made by the ANT project at USC/ISI
• ipcrypt: Anonymize IP addresses using ipcrypt create by Jean-Philippe Aumasson @veorq

Bugfixes:

• Fix changing royparse and txtout with other plugins (thanks to Duane Wessels @wessels and Paul Hoffman @paulehoffman)
• Free pointers to allocated strings in text_free() (thanks to Michał Kępień @kempniu)
• Fix IP checksum calculation

Other changes:

• -B and -E can be used without -w (thanks to Duane Wessels @wessels)
• Use pcap_findalldevs() instead of pcap_lookupdev() (thanks to Michał Kępień @kempniu)
• Document and add -? option to all plugins
• Fix clang scan-build bugs and LGTM alerts
• Use gmtime_r() instead of gmtime()
• Update pcap-thread to v4.0.0

67d8e2c Fix
fb0ed02 Plugin documentation
a2c9a6c cryptopant
39db1ca Deanonymize, IPv6 test
afc7107 Crypto-PAn, cryptopANT
f1912cc OpenSSL, anonaes128
f2bab62 ipcrypt, anonmask
158b1e7 anonmask help
60ece58 anonmask
8f1b138 Plugin types, filter plugin, set iaddr extension, anonymization by masking
b7d7991 IP checksum
641a23a Free pointers to allocated strings in text_free()
4d313bf pcap_findalldevs()
091e0ca Use pcap_findalldevs() instead of pcap_lookupdev()
6a7b25e Clean up use of feature test macros on Linux
cbba14c Configure, uninitialized
f228c9c Code formatting
3fd738c man-page
770168a Test
714e4f5 Fix -B so that it works when reading offline pcap files.
8675bea Test
911fec9 Implementing test9 as a test of -B and -E command line args.
a7cc72d -B and -E can work fine without -w .
04c4928 Made the same changes to txtout as were in 165a786
165a786 Workaround for stdio mystery causing duplicate royparse output.

Release 1.9.0

This release adds a new option to change how the Berkeley Packet Filter is generated to include the host restrictions for all selections, previously this restriction would only apply to specific parts.

Additional tweaks to the RSSM plugin has been made to conform to the RSSAC002v3 specification. One noticeable change is that the plugin now requires the DNS to be parsed before counted, any error in the parsing will result in the message being left out of the statistics.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dnscap/dnscap-1.9.0.tar.gz
sha256: 1c3793a65948e90be65730c915d808cfdc894ec3b61ee9a266c080eb1fac5382

Packages are available at: https://dev.dns-oarc.net/packages/

Changes:

• Fix spacing in BPF filter to look better
• Fix #146: Add bpf_hosts_apply_all, apply any host restriction to all
• plugin/rssm:
  • Remove quoting of start-period and correctly handle empty hashes
  • Issue #152, Issue #91: Parse DNS before processing RSSM counters
• plugin/rssm/dnscap-rssm-rssac002: Use YAML::Dump() for output

47d892b Issue #152: RSSM YAML output
d4f1466 Issue #152, Issue #91: Parse DNS before processing RSSM counters
68fc1ff BPF, bpf_hosts_apply_all

Release 1.8.0

This release updates the TCP stream code in order to be able to look at more then just the first query, for handling already ongoing TCP connections without having seen SYN/ACK and for reassembly of the TCP stream prior of parsing it for DNS with an additional layer of parsing (see reassemble_tcp_bfbparsedns).

Updates to the Root Server Scaling Measurement (RSSM) plugin have also been made to bring it up to date with RSSAC002v3 specification, be able to output the YAML format described and an additional script to merge YAML files if the interval is less then the RSSAC002v3 24 hour period. See "Updates to the RSSM plugin" below and plugins/rssm/README.md.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dnscap/dnscap-1.8.0.tar.gz
sha256: 5440fecc1a3f98948207e23549afe0c102b45eda5cde2a2b80ed446d88457558

Packages are available at: https://dev.dns-oarc.net/packages/

New extended options:

• parse_ongoing_tcp: Start tracking TCP connections even if SYN/ACK has not been seen
• allow_reset_tcpstate: Allow external reset of TCP state
• reassemble_tcp: Use to enable TCP stream reassembly
• reassemble_tcp_faultreset: Number of faults before reseting TCP state when reassembly is enabled
• reassemble_tcp_bfbparsedns: Enable an experimental additional layer of reassemble that uses libbind to parse the payload before accepting it. If the DNS is invalid it will move 2 bytes within the payload and treat it as a new payload, taking the DNS length again and restart the process. Requires libbind and reassemble_tcp.

New extension functions for plugins:

• DNSCAP_EXT_TCPSTATE_GETCURR: Function to get a pointer for the current TCP state
• DNSCAP_EXT_TCPSTATE_RESET: Function to reset a TCP state

New features:

• Parse additional DNS queries in TCP connections
• -g and the txtout plugin will reset TCP state (if allowed) on failure to parse DNS

Bugfixes:

• Fix -g output, separate error message with a space
• Fix TCP packets wrongfully flagged as DNS when using layers.
• Fix TCP debug output when using layers, ia_str() is not safe to call twice in the same printf because of local buffer.
• Fix exported extension functions, need to be file local

New tests for:

• Multiple DNS queries in one TCP connection
• Query over TCP without SYN
• Queries over TCP with first query missing length
• Queries over TCP with middle payloads missing
• Add test with TCP stream that missing multiple packets in the middle

Updates to the RSSM plugin (plugins/rssm):

• Add info about saving counts and sources
• Fix memory leak on fopen() errors
• Update to RSSAC002v3 specification
• New options:
  • -D to disable forking on close
  • -Y: Use RSSAC002v3 YAML format when writing counters, the file will contain multiple YAML documents, one for each RSSAC002v3 metric
    Used with; -S adds custom metric dnscap-rssm-sources and -A adds dnscap-rssm-aggregated-sources
  • -n: Set the service name to use in RSSAC002v3 YAML
  • -S: Write source IPs into counters file with the prefix source
  • -A: Write aggregated IPv6(/64) sources into counters file with the prefix aggregated-source
  • -a: Write aggregated IPv6(/64) sources to <name>.<timesec>.<timeusec>
• Add dnscap-rssm-rssac002 Perl script for merging RSSAC002v3 YAML files
• Add README.md for the plugin man-page for dnscap-rssm-rssac002
• Add test for YAML output and merging of YAML files

c7058c8 Use file local functions for all extensions
66b352d RSSM RSSAC002v3 YAML Tool
b09efc2 plugins/rssm RSSAC002v3
709aba6 Fix #89: Add additional reassembly layers that parses the payload byte for byte for valid DNS
04fa013 Fix CID 1463944 (again)
b1cf623 RSSM saving data and forking
fb23305 Fix CID 1463944
0fca1a8 Issue #89: TCP stream reassemble
bb6428c CID 1463814: Check ns_initparse() for errors
a57066f Fix #88: TCP handling

Release 1.7.1

The library used for parsing DNS (libbind) is unable to parse DNS messages when there is padding at the end (the UDP/TCP payload is larger then the DNS message). This has been fixed by trying to find the actual DNS message size, walking all labels and RR data, and then retry parsing.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dnscap/dnscap-1.7.1.tar.gz
sha256: a6839a0f5024947f764d1087244daabb7296447123b717c68d2574b673affb5f

Packages are available at: https://dev.dns-oarc.net/packages/

Other changes and bug-fixes:

• Fix size when there is a VLAN to match output of use_layers yes/no
• Add test of VLAN matching
• Fix hashtbl.c building in rssm
• Add test with padded DNS message

49e5400 Fix #127: If ns_initparse() returns EMSGSIZE, try and get actual size and reparse
99bda0b Fix #98: VLAN

Release 1.7.0

This release adds IP fragmentation handling by using layers in pcap-thread which also adds a new flag to output and modules. DNSCAP_OUTPUT_ISLAYER indicates that pkt_copy is equal to payload since the layers of the traffic have already been parsed. IP fragments are reassembled with the pcap_thread_ext_frag extension that is included in pcap-thread.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dnscap/dnscap-1.7.0.tar.gz
sha256: 422fa923746387dd2fa38aecbee217476d03ad43f8a8845a074b347d179c6d98

Packages are available at: https://dev.dns-oarc.net/packages/

New extended (-o) options:

• use_layers: Use pcap-thread layers to handle the traffic
• defrag_ipv4: Enabled IPv4 de-fragmentation
• defrag_ipv6: Enabled IPv6 de-fragmentation
• max_ipv4_fragments: Set maximum fragmented IPv4 packets to track
• max_ipv4_fragments_per_packet: Set the maximum IPv4 fragments per tracked packet
• max_ipv6_fragments: Set maximum fragmented IPv6 packets to track
• max_ipv6_fragments_per_packet: Set the maximum IPv6 fragments per tracked packet

Currently -w does not work with use_layers and the plugins pcapdump and royparse will discard output with the flag DNSCAP_OUTPUT_ISLAYER because they need access to the original packet.

The rzkeychange plugin now encodes certain flag bits in the data that it reports for RFC8145 key tag signaling. The flags of interest are: DO, CD, and RD. These are encoded in an bit-mask as a hexadecimal value before the _ta component of the query name.

Other changes and bug-fixes:

• Fix #115: document -g output, see OUTPUT FORMATS diagnostic in dnscap(1) man-page
• Add test to match output from non-layers runs with those using layers
• Add test with fragmented DNS queries
• Fix #120: CBOR/CDS compiles again, update tinycbor to v0.4.2
• Fix ip->ip_len byte order
• Fix parsing of IP packets with padding or missing parts of payload

0347f74 Add AUTHORS section in man-page
ef1b68c Fix CID 1463073
8a79f89 Layers
a404d08 Update pcap-thread to v3.1.0, add test for padding fixes
08402f1 Fix byte order bug. ip->ip_len must be evaluated with ntohs().
d6d2340 CBOR/CDS and formatting
85ec2d8 Fix #87: IP fragmentation reassembly
22bfd4a Documentation
c35f19f Adding flag bits to rzkeychange RFC8145 key tag signaling data. This may be useful to find "false" key tag signals from sources that don't actually perform DNSSEC validation.

Release 1.6.0

New additions to the plugins:

• rzkeychange can now collect RFC8145 key tag signaling. Signals are saved during the collection interval, and then sent to the specified -k <zone>, one at a time, at the end of the interval. Only root zone signals are collected. Added by Duane Wessels (@wessels).
• royparse is a new plugin to splits a PCAP into two streams, queries in PCAP format and responses in ASCII format. Created by Roy Arends (@RoyArends).
• txtout new option -s for short output, only print QTYPE and QNAME for IN records. Added by Paul Hoffman (@paulehoffman)
• The extension interface has been extended with DNSCAP_EXT_IA_STR to export the ia_str() function.

Bugfixes and other changes:

• Remove duplicated hashtbl code
• rssm: fix bug where count in table was taken out as uint16_t but was a uint64_t
• Handle return values from hashtbl functions
• txtout: removed unused -f options
• Change ia_str() to use buffers with correct sizes, thanks to @RoyArends for spotting this!

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dnscap/dnscap-1.6.0.tar.gz
sha256: 13fa9871e47559b61be8ad2936aa3e353631a09f1623e9f40a5143b1ef42efaa

Packages are available at: https://dev.dns-oarc.net/packages/

Commits:
3f78a31 Add copy/author text
1bd914d Fix CID 1462343, 1462344, 1462345
f9bb955 Fix fprintf() format for message size
abedf84 Fix #105: inet_ntop buffers
bfdcd0d Addresses the suggestions from Jerry.
dda0996 royparse :)
4f6520a royparse plugin finished
f1aa4f2 Fix #103: Remove opt_f
32355b7 Rearrange code to keep the change smaller and fix indentation
d6612c1 Added -s to txtout for short output
9d8d1ef Check return of snprintf()
55f5aba Format code
9f19ec3 Fixed memory leak in rzkeychange_keytagsignal()
58b8784 Fix memory leaks and better return value checks in rzkeychange_submit_counts()
b06659f Add server and node to keytag signal query name
705a866 Always free response packets in rzkeychange plugin.
e802843 Implement RFC8145 key tag signal collection in rzkeychange plugin
5fbf6d0 Added extension for ia_str() so it can be used by rzkeychange plugin.
3be8b8f Split dnscap.c into more files
e431d14 Fix #92: hashtbl

Release 1.5.1

Compatibility fixes for FreeBSD 11.1+ which is now packing struct ip and for OpenBSD.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dnscap/dnscap-1.5.1.tar.gz
sha256: d218b707a0bbb158bbf88306e5d53b914394d971f4f9736032afde7b119e7da6

Packages are available at: https://dev.dns-oarc.net/packages/

Commits:
17e3c92 FreeBSD is packing struct ip, need to memcpy()
f8add66 Code formatting
38cd585 Add documentation about libbind
d1dd55b Fix #82: Update dependencies for OpenBSD

Release 1.5.0

Added support for writing gzipped PCAP if the -W suffix ends with .gz and made -X work without -x. New interface for plugins to tell them what extensions are available and a new plugin rzkeychange.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dnscap/dnscap-1.5.0.tar.gz
sha256: 6dd3359a73b4f13846b045493262fabb88a1e4c49ffd2b66e43a2f3b623af651

Packages are available at: https://dev.dns-oarc.net/packages/

Plugin extensions:

• Call plugin_extension(ext, arg) to tell plugin what extensions exists
• Add extension for checking responder (is_responder())

The rzkeychange plugin was developed by Duane Wessels 2016 in support of the root zone ZSK size increase. It is also being used in support of the 2017 root KSK rollover and collects the following measurements:

• total number of responses sent
• number of responses with TC bit set
• number of responses over TCP
• number of DNSKEY responses
• number of ICMP_UNREACH_NEEDFRAG messages received
• number of ICMP_TIMXCEED_INTRANS messages received
• number of ICMP_TIMXCEED_REASS messages received

Other fixes (author Duane Wessels):

• 232cbd0: Correct comment description for meaning of IPPROTO_AH
• 181eaa4: Add #include <sys/time.h> for struct timeval on NetBSD

Commits:

1d894e2 Make -x and -X work correctly together and update man-page
34bc54c Make the -X option work without requiring a -x option.
f43222e Fix CID 1440488, 1440489, 1440490
aa54395 Update pcap-thread to v2.1.3
81174ce Prepare SPEC for OSB/COPR
21d7468 New plugin rzkeychange and plugin extensions
38491a3 Config header is generated by autotools
419a8ab Small tweaks and fixes for gzip support
1967abc updated for earlier BSD versions
f135c90 added auto gzip if the -W suffix ends with .gz

Commits during development of rzkeychange (author Duane Wessels):

• 620828d: Add rzkeychange -z option to specify resolver IP addresses
• 1f77987: Add -p and -t options to rzkeychange plugin to configure an alternate port and TCP. Useful for ssh tunnels.
• 2a571f1: Split ICMP time exceeded counter into two counters for time exceeded due to TTL and another due to fragmentation
• e4ee2d3: The rzkeychange data collection plugin uses DNSCAP_EXT_IS_RESPONDER extension to know if an IP address is a "responder" or not, because when dnscap is instructed to collect ICMP with -I, it processes all ICMP packets, not just those limited to responders (or initiators).
• cee16b8: Add ICMP Time Exceeded to counters
• ad8a227: Counting source IPs has performance impacts. #ifdef'd out for now add ICMP "frag needed" counts
• c25e72b: Implemented DNS queries with ldns. First there will be some test queries to ensure the zone is reachable and configured to receive data. Then a query naming the fields, followed by the periodic queries delivering counts.
• fd23be7: Make report zone, server, node command line argumements mandatory
• 137789b: Adding rzkeychange plugin files

Release 1.4.1

Fixed an issue that when compiled with libpcap that had a specific feature enabled it would result in a runtime error which could not be worked around.

Also fixed various compatibility issues and updated dependency documentation for CentOS.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dnscap/dnscap-1.4.1.tar.gz
sha256: c65342c198caeabfa09d97126e07cb32e90cba98daddb622507dd606d018a024

Commits:

785d4c4 Fix compiler warnings
2d4df8d Fix #65: Update pcap-thread to v2.1.2
26d3fbc Fix #64: Add missing dependency
55e6741 Update pcap-thread to v2.1.1, fix issue with libpcap timestamp type
c6fdb7a Fix typo and remove unused variables