Dear community,
It's again long overdue for a new release and here it is finally. Since August we collected quite a few bugfixes - Thanks to all contributors!
IntelMQ Installation documentation: https://github.com/certtools/intelmq/blob/2.2.2/docs/INSTALL.md IntelMQ Upgrade documentation: https://github.com/certtools/intelmq/blob/2.2.2/docs/UPGRADING.md
*News for IntelMQ 2.2.2*
### Bots #### Cymru Whois Lookup The cache key calculation has been fixed. It previously led to duplicate keys for different IP addresses and therefore wrong results in rare cases. The cache key calculation is intentionally not backwards-compatible. Therefore, this bot may take longer processing events than usual after applying this update. More details can be found in [issue #1592](https://github.com/certtools/intelmq/issues/1592).
### Harmonization #### Shadowserver Feed/Parser The feed "Blacklisted-IP" has been renamed by ShadowServer to "Blocklist". In IntelMQ, the old name can still be used in IntelMQ until version 3.0.
*Changes for IntelMQ 2.2.2*
### Core - `intelmq.lib.upgrades`: - Add upgrade function for renamed Shadowserver feed name "Blacklisted-IP"/"Blocklist".
### Bots #### Parsers - `intelmq.bots.parsers.shadowserver`: - Rename "Blacklisted-IP" feed to "Blocklist", old name is still valid until IntelMQ version 3.0 (PR#1588 by Thomas Hungenberg). - Added support for the feeds `Accessible Radmin` and `CAIDA IP Spoofer` (PR#1600 by sinus-x). - `intelmq.bots.parsers.anubisnetworks.parser`: Fix parsing error where `dst.ip` was not equal to `comm.http.host`. - `intelmq/bots/parsers/danger_rulez/parser`: correctly skip malformed rows by defining variables before referencing (PR#1601 by Tomas Bellus). - `intelmq.bots.parsers.misp.parser: Fix MISP Event URL (#1619, PR#1618 by Nedfire23). - `intelmq.bots.parsers.microsoft.parser_ctip`: - Add support for `DestinationIpInfo.*` and `Signatures.Sha256` fields, used by the `ctip-c2` feed (PR#1623 by Mikk Margus Möll). - Use `extra.payload.text` for the feed's field `Payload` if the content cannot be decoded (PR#1610 by Giedrius Ramas).
#### Experts - `intelmq.bots.experts.cymru_whois`: - Fix cache key calculation which previously led to duplicate keys and therefore wrong results in rare cases. The cache key calculation is intentionally not backwards-compatible (#1592, PR#1606). - The bot now caches and logs (as level INFO) empty responses from Cymru (PR#1606).
### Documentation - README: - Add Core Infrastructure Initiative Best Practices Badge. - Bots: - Generic CSV Parser: Add note on escaping backslashes (#1579). - Remove section of non-existing "Copy Extra" Bot. - Explain taxonomy expert. - Add documentation on n6 parser. - Gethostbyname expert: Add documentation how errors are treated. - Feeds: - Fixed bot modules of Calidog CertStream feed. - Add information on Microsoft CTIP C2 feed.
### Packaging - In Debian packages, `intelmqctl check` and `intelmqctl upgrade-config` are executed in the postinst step (#1551, PR#1624 by Birger Schacht).
### Tests - `intelmq.tests.lib.test_pipeline`: Skip `TestAmqp.test_acknowledge` on Travis with Python 3.8. - `intelmq.tests.bots.outputs.elasticsearch.test_output`: Refresh index `intelmq` manually to fix random test failures (#1593, PR#1595 by Zach Stone).
### Tools - `intelmqctl check`: - For disabled bots which do not have any pipeline connections, do not raise an error, but only warning. - Fix check on source/destination queues for bots as well the orphaned queues.
### Contrib - Bash completion scripts: Check both `/opt/intelmq/` as well as LSB-paths (`/etc/intelmq/` and `/var/log/intelmq/`) for loading bot information (#1561, PR#1628 by Birger Schacht).
### Known issues - Bots started with IntelMQ-Manager stop when the webserver is restarted. (#952). - Corrupt dump files when interrupted during writing (#870).