Dear IntelMQ community, users, developers, and Incident Response teams!

Please see the list of all changes below.

Documentation: https://doc.intelmq.org/
Source code: https://github.com/certtools/intelmq

Thanks to all contributors to this release, in alphabetical order:

• DigitalTrustCenter
• Edvard Rejthar (CSIRT.CZ)
• elsif2 (Shadowserver Foundation)
• Kamil Mankowski (CERT.at)
• Mikk Margus Möll (CERT.ee)
• Sebastian Wagner (Institute for Common Good Technology, Intevation & BSI)

The full list of changes:

Core
====
- `intelmq.lib.utils.drop_privileges`: When IntelMQ is called as `root` and dropping the privileges to user `intelmq`, also set the non-primary groups associated with the `intelmq` user. Makes the behaviour of running intelmqctl as `root` closer to the behaviour of `sudo -u intelmq ...` (PR#2507 by Mikk Margus Möll).
- `intelmq.lib.utils.unzip`: Ignore directories themselves when extracting data to prevent the extraction of empty data for a directory entries (PR#2512 by Kamil Mankowski).

Bots
====

Collectors
----------

- `intelmq.bots.collectors.shadowserver.collector_reports_api.py`:
  - Added support for the types parameter to be either a string or a list (PR#2495 by elsif2).
  - Refactored to utilize the type field returned by the API to match the requested types instead of a sub-string match on the filename.
  - Fixed timezone issue for collecting reports (PR#2506 by elsif2).
  - Fixed behaviour if parameter `reports` value is empty string, behave the same way as not set, not like no report (PR#2523 by Sebastian Wagner).
- `intelmq.bots.collectors.shodan.collector_stream` (PR#2492 by Mikk Margus Möll):
  - Add `alert` parameter to Shodan stream collector to allow fetching streams by configured alert ID
- `intelmq.bots.collectors.mail._lib`: Remove deprecated parameter `attach_unzip` from default parameters (PR#2511 by Sebastian Wagner).

Parsers
-------

- `intelmq.bots.parsers.shadowserver._config`:
  - Fetch schema before first run (PR#2482 by elsif2, fixes #2480).
- `intelmq.bots.parsers.dataplane.parser`: Use `  |  ` as field delimiter, fix parsing of AS names including `|` (PR#2488 by DigitalTrustCenter).
- all parsers: add `copy_collector_provided_fields` parameter allowing copying additional fields from the report, e.g. `extra.file_name`. (PR#2513 by Kamil Mankowski).

Experts
-------

- `intelmq.bots.experts.sieve.expert`:
  - For `:contains`, `=~` and `!~`, convert the value to string before matching avoiding an exception. If the value is a dict, convert the value to JSON (PR#2500 by Sebastian Wagner).
  - Add support for variables in Sieve scripts (PR#2514 by Mikk Margus Möll, fixes #2486).
- `intelmq.bots.experts.filter.expert`:
  - Treat value `false` for parameter `filter_regex` as false (PR#2499 by Sebastian Wagner).

Outputs
-------

- `intelmq.bots.outputs.misp.output_feed`: Handle failures if saved current event wasn't saved or is incorrect (PR by Kamil Mankowski).
- `intelmq.bots.outputs.smtp_batch.output`: Documentation on multiple recipients added (PR#2501 by Edvard Rejthar).

Documentation
=============

- Bots: Clarify some section of Mail collectors and the Generic CSV Parser (PR#2510 by Sebastian Wagner).

-- 
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578