Dear community,
April is nearing it's end and it's time to release a bunch of bugfixes. Please find below the list of changes. Thanks to all contributors for the issues reported and pull requests!
The new version is already available on GitHub, PyPI, the deb+rpm repositories and DockerHub.
Installation documentation: https://intelmq.readthedocs.io/en/maintenance/user/installation.html Upgrade documentation: https://intelmq.readthedocs.io/en/maintenance/user/upgrade.html
### Core - `intelmq.lib.harmonization`: - `TLP` type: accept value "yellow" for TLP level AMBER.
### Bots #### Collectors - `intelmq.bots.collectors.shadowserver.collector_reports_api`: - Handle timeouts by logging the error and continuing to next report (PR#1852 by Marius Karotkis and Sebastian Wagner, fixes #1823).
#### Parsers - `intelmq.bots.parsers.shadowserver.config`: - Parse and harmonize field `end_time` as date in Feeds "Drone-Brute-Force" and "Amplification-DDoS-Victim" (PR#1833 by Mikk Margus Möll). - Add conversion function `convert_date_utc` which assumes UTC and sanitizes the data to datetime (by Sebastian Wagner, fixes #1848). - `intelmq.bots.parsers.shadowserver.parser_json`: - Use the overwrite parameter for optionally overwriting the "feed.name" field (by Sebastian Wagner). - `intelmq.bots.parsers.microsoft.parser_ctip`: - Handle fields `timestamp`, `timestamp_utc`, `source_ip`, `source_port`, `destination_ip`, `destination_port`, `computer_name`, `bot_id`, `asn`, `geo` in `Payload` of CTIP Azure format (PR#1841, PR#1851 and PR#1879 by Sebastian Wagner). - `intelmq.bots.parsers.shodan.parser`: - Added support for unique keys and verified vulns (PR#1835 by Mikk Margus Möll). - `intelmq.bots.parsers.cymru.parser_cap_program`: - Fix parsing in whitespace edge case in comments (PR#1870 by Alex Kaplan, fixes #1862).
#### Experts - `intelmq.bots.experts.modify`: - Add a new rule to the example configuration to change the type of malicious-code events to `c2server` if the malware name indicates c2 (PR#1854 by Sebastian Wagner). - `intelmq.bots.experts.gethostbyname.expert`: - Fix handling of parameter `gaierrors_to_ignore` with value `None` (PR#1890 by Sebastian Wagner, fixes #1886).
#### Outputs - `intelmq.bots.outputs.elasticsearch`: Fix log message on required elasticsearch library message (by Sebastian Wagner).
### Documentation - `dev/data-harmonization`: Fix taxonomy name "information gathering" should be "information-gathering" (by Sebastian Wagner).
### Tests - `intelmq.tests.bots.parsers.microsoft.test_parser_ctip_azure`: - Add test case for TLP level "YELLOW".
### Known issues - ParserBot: erroneous raw line recovery in error handling (#1850).