Dear all
Apparently, around Monday Shadowserver changed their Feed Parsing
Schema, that is used by your Shadowserver Parser Bot.
These are the changes:
+------------------------+-------------------------+
| Previous | Now |
+------------------------+-------------------------+
| extra.severity | severity |
| extra.device_vendor | product.vendor |
| extra.device_model | product.name |
| extra.device_version | product.version |
| extra.vulnerability_id | product.vulnerabilities |
+------------------------+-------------------------+
You may need to update your workflows accordingly.
If you do not update the schema automatically (parameter 'auto_update'
by Shadowserver Parser Bot), you can start the update of the Parsing
Schema manually:
intelmq.bots.parsers.shadowserver.parser --update-schema
Best regards
Sebastian
--
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578
Dear IntelMQ users
The latest IntelMQ release, version 3.5.0, introduced a new field named
'severity' with its possible values critical, high, medium, low, info,
and undefined.
If you have incoming data that places the severity information in the
'extra.severity' field, but you want it in the 'severity' field, you can
use a workaround using a Modify expert.
For example, this is necessary for all data coming from Shadowserver.
Howto:
1. Add a Modify Expert, e.g., named 'severity-modify-expert', to your
dataflow and put it somewhere after the parsers.
2. Set the configuration path to e.g.
/var/lib/intelmq/bots/modify/severity.conf
3. Put the content of the attached file into this file.
4. Start the bot and reload it right before the new
'severity-modify-expert' is introduced. Thisis doneautomatically if
you use the IntelMQ Manager.
The attached configuration for the Modify expert contains two rules:
1. If the 'extra.severity' field exists (is not empty), but the
'severity' field does not (is empty), use the value of the latter
for the first.
2. Otherwise, if both fields do not exist (are empty), set the default
value 'undefined'
Of course, you can also add these two rules to an existing Modify expert
to keep your configuration small.
To test if your newly updated configuration works, you can use the
commands below.
Please let us know if this is helpful to you or if you have any further
questions.
Best regards
Sebastian
The test commands:
intelmqctl run severity-expert process --dryrun --show-sent --msg '{}'
-> this results in the fallback {"severity": "undefined"}
intelmqctl run severity-expert process --dryrun --show-sent --msg
'{"extra.severity": "high"}'
-> this results in {"extra.severity": "high", "severity": "high"}
intelmqctl run severity-expert process --dryrun --show-sent --msg
'{"extra.severity": "high", "severity": "low"}'
-> this results in no change: {"extra.severity": "high", "severity": "low"}
--
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578
Dear IntelMQ community
The new IntelMQ release 3.5.0 brings support for a few new data fields
that many of you have requested:
* severity
* product.full_name, product.name, product.vendor, product.version,
product.vulnerabilities (IEP009: IntelMQ Data Format: Describe
product and mark vulnerabilities
https://github.com/certtools/ieps/tree/main/009)
* constituency (IEP008: IntelMQ Data Format: Constituency field
https://github.com/certtools/ieps/tree/main/008)
If you use a database to store events, update the schema of theevents
table with the SQL statements shown below.
If you use deb- or rpm-packages, you have two new packages available:
intelmq-contrib and intelmq-autostart. The first contains stufffrom the
repositories' contrib/ folder: some eventdb scripts, an example
extension package, the feeds config generator, malware name mapping
scripts, and systemd scripts.The autostart package contains systemd
units to start IntelMQ at boot time and then run it periodically.
The full list of changes is very long, belowyou can read the full list
of changes.
Thanks to all who contributed to this version, whether through user
feedback or code.
This version contains code contributions from Edvard Rejthar, Kamil
Mańkowski, Karl-Johan Karlsson, Lukas Heindl, Mikk Margus Möll, Nakul
Rajpal, Sebastian Wagner, Timde Boer.
The 3.5.0 version is available at GitHub, PyPI andin the package
repositories:
https://github.com/certtools/intelmq/releases/tag/3.5.0https://pypi.org/project/intelmq/https://docs.intelmq.org/latest/admin/installation/linux-packages/
The complete change log:
## Requirements
Python `>=3.9` is now required, which is available on all platforms
supported by IntelMQ.
## Data Format
To save new fields from IntelMQ Data Format in existing PostgreSQL
instances, the following schema update is necessary:
```sql
CREATE TYPE severity_enum AS ENUM (
'critical',
'high',
'medium',
'low',
'info',
'undefined'
);
ALTER TABLE events ADD "product.full_name" text;
ALTER TABLE events ADD "product.name" text;
ALTER TABLE events ADD "product.vendor" text;
ALTER TABLE events ADD "product.version" text;
ALTER TABLE events ADD "product.vulnerabilities" text;
ALTER TABLE events ADD severity varchar(10);
ALTER TABLE events ADD "constituency" text;
UPDATE events SET severity = (extra ->> 'severity')::severity_enum;
```
Optionally remove the severity field from the extra fields in existing
entries:
```sql
UPDATE events SET extra = extra - 'severity';
```
To switch to the more efficient data type `jsonb` instead of `json`, use
the following SQL statement. Data is preserved. JSONB also has more
query and data manipulation features than plain JSON.
```sql
ALTER TABLE events
ALTER COLUMN "extra" SET DATA TYPE jsonb;
```
## Configuration
- New parameter `stop_retry_limit` (PR#2598 by Lukas Heindl).
## Core
- Drop support for Python 3.8 (fixes #2616, PR#2617 by Sebastian Wagner).
- `intelmq.lib.splitreports`: Handle bot parameter `chunk_size` values
empty string, due to missing parameter typing checks (PR#2604 by
Sebastian Wagner).
- `intelmq.lib.mixins.sql` Add Support for MySQL (PR#2625 by Karl-Johan
Karlsson).
- New parameter `stop_retry_limit` to gracefully handle stopping bots
which take longer to shutdown(PR#2598 by Lukas Heindl, fixes #2595).
- `intelmq.lib.datatypes`: Remove unneeded Dict39 alias (PR#2639 by
Nakul Rajpal, fixes #2635)
- `intelmq.lib.mixins.http`: Only set HTTP header 'Authorization' if
username or password are set and are not both empty stringas they are by
default in the Manager (fixes #2590, PR#2634 by Sebastian Wagner).
- `intelmq.lib.message.Message.from_dict`: Do not modify the dict
parameter by adding the `__type` field and raise an error when type is
not determinable (PR#2545 by Sebastian Wagner).
- `intelmq.lib.mixins.cache.CacheMixin` was extended to support
temporarystoring messages in a cache queue
(PR#2509 by Kamil Mankowski).
## Data Format
- Implementing [IEP009](https://github.com/certtools/ieps/tree/main/009)
introducingfields to identify products and vulnerabilities:
`product.full_name`, `product.name`, `product.vendor`,
`product.version`, `product.vulnerabilities`. To store in existing
PostgreSQL instances, a schema update is necessary, see firstsection.
- added `severity` field to help with triaging received events (PR#2575
by Kamil Mańkowski). To allow saving the field in PostgreSQLdatabase in
existing installations, the schema update shown in the first section is
necessary.
- Implementing [IEP008](https://github.com/certtools/ieps/tree/main/008)
introducingthe `constituency` field for easier identification in
multi-constituency setups. (PR#2573 by Kamil Mańkowski). To use in
current PostgreSQL installations, a schema update is necessary, see
firstsection.
## Bots
## Collectors
- `intelmq.bots.collectors.mail.collector_mail_attach`: Decrypt GPG
attachments (PR#2623 by Edvard Rejthar).
- `intelmq.bots.collectors.mail.collector_mail_attach`: Allow empty
attachments (PR#2647 by Edvard Rejthar).
- `intelmq.bots.collectors.shodan.collector_alert`: Added a new
collector to query the Shodan Alert API (PR#2618 by Sebastian Wagner and
Malawi CERT).
- Remove `intelmq.bots.collectors.blueliv` as it uses an unmaintained
library, does not work any moreandbreaks other CI tests (fixes #2593,
PR#2632 by Sebastian Wagner).
## Parsers
- `intelmq.bots.parsers.cymru.parser_cap_program`: Add mapping for TOR
and ipv6-icmpprotocol (PR#2621 by Mikk Margus Möll).
- Remove `intelmq.bots.parsers.blueliv` as it is obsolete with the
removed collector (PR#2632 by Sebastian Wagner).
- `intelmq.bots.parser.json.parser`:
- Support data containing lists of JSON Events (PR#2545 by Tim de Boer).
- Add default `classification.type` with value `undetermined` if
input data has nowclassification itself (PR#2545 by Sebastian Wagner).
## Experts
- `intelmq.bots.experts.asn_lookup.expert`:
- Print URLs to stdout only in verbose mode (PR#2591 by Sebastian
Wagner).
- Check for database file existence and writability (fixes #2566).
- Use database path matching to installation type (PR#2606 by
Sebastian Wagner).
- `intelmq.bots.experts.fake.expert`:
- Use database path matching to installation type (PR#2606 by
Sebastian Wagner).
- Add new mode `random_single_value` (PR#2601 by Sebastian Wagner).
- `intelmq.bots.experts.sieve.expert`: Test for textX dependency in
self-check (PR#2605 by Sebastian Wagner).
- `intelmq.bots.experts.trusted_introducer_lookup.expert`: Change to new
TI database URL (fixes #2620, PR#2633 by Sebastian Wagner).
## Outputs
- `intelmq.bots.outputs.smtp_batch.output`:
- Add newparameter `additional_grouping_keys` for an enhanced email
batching feature.
- Add newparameter `templating` for additional template variables.
- Add new parameter `allowed_fieldnames` for csvfield specification.
- Add new parameter `fieldnames_translation` for naming csvheaders
(PR#2610 by Lukas Heindl, fixes #2586).
- `intelmq.bots.outputs.sql.output`: Add Support for MySQL (PR#2625 by
Karl-Johan Karlsson).
## Documentation
- Fix and refresh links to mailing lists (PR#2609 by Kamil Mańkowski)
- `Aggregate Bot`: Add illustration graphics (PR#2612 by Sebastian Wagner).
## Packaging
- Replace `/opt/intelmq` example paths in bots with variable
`VAR_STATE_PATH` for correct paths in LSB-path setups likewith packages
(PR#2587 by Sebastian Wagner).
- New deb-package `intelmq-contrib` with all `contrib/` scripts and
documentation (PR#2614 by Sebastian Wagner).
- New deb-package `intelmq-autostart` containing systemd services and
timers to start all enabled IntelMQ bots at boot and periodically
(PR#2638 by Sebastian Wagner).
## Tests
- `intelmq.tests.lib.test_pipeline.TestAmqp.test_acknowledge`: Skip on
all Python versions when running on CI (PR#2602 by Sebastian Wagner).
- `.github/workflows/codespell.yml`, `debian-package.yml`,
`regexploit.yml`: Upgrade to `ubuntu-latest` runners (PR#2602 by
Sebastian Wagner).
- `intelmq.test.test_conf`: With changed behaviourin ruamel.yaml on line
wrapping since version 0.18.13, only test the parsabiltyof
`runtime.yaml` (PR#2619 by Sebastian Wagner).
- `intelmq.test.BotTestCase.test_static_bot_check_method`: Remove
debugging stub raising for all non-empty checks (PR#2622 by Sebastian
Wagner).
## Tools
- `intelmq.bin.intelmq_psql_initdb`: Use `JSONB` type by default,
Postgres supports it since version 9 (PR#2597 by Sebastian Wagner).
- `intelmq.bin.rewrite_config_files`: Removed obsolete JSON
configuration file rewriter (PR#2613 by Sebastian Wagner).
- `intelmq/lib/bot_debugger.py`: Fix overwriting the runtime logging
level by command line parameter (PR#2603 by Sebastian Wagner, fixes #2563).
- `intelmq.bin.intelmqctl`: Fix bot log level filtering (PR#2607 by
Sebastian Wagner, fixes #2596).
## Contrib
- Bash Completion: Adapt to YAML-style runtime configuration (PR#2642 by
Sebastian Wagner, fixes #2094).
- Remove `prettyprint` script, use `jq` instead (PR#2551 by Sebastian
Wagner).
## Known issues
Thisis shortlist of the most important known issues. The fulllist can be
retrievedfrom
[GitHub](https://github.com/certtools/intelmq/labels/bug?page=2&q=is%3Aopen+….
- stomp.py 8.2.0+ breaks the version check in stomp bots (#2600).
- Traceback when calling intelmqdump without write access to the log
file (#2529).
- pyyaml PendingDeprecationWarning: you should no longer specify
'unsafe' -> test failure (#2489).
- `intelmq.parsers.html_table` may not process invalid URLs in patched
Python version due to changes in `urllib` (#2382).
- Breaking changes in 'rt' 3.0 library (#2367).
- Type error with SQL output bot's `prepare_values` returning list
instead of tuple(#2255).
- `intelmq_psql_initdb` does not work for SQLite (#2202).
- intelmqsetup: should install a default state file (#2175).
- Misp Expert - Crash if misp event already exist(#2170).
- Spamhaus CERT parser uses wrongfield (#2165).
- Custom headers ignored in HTTPCollectorBot (#2150).
- intelmqctl log: parsing syslog does not work (#2097).
- Bots started with IntelMQ-API/Manager stop when the webserveris
restarted (#952).
- Corrupt dump files when interrupted during writing (#870).
--
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578
Dear IntelMQ operators
We have two news for you regarding the deb packages for Debian and Ubuntu:
- A new package 'intelmq-contrib'
- Drop of Ubuntu 20.04 packages
For the standard repository, they will be in effect with the next
IntelMQ release 4.1.0 in August.
The changes are already effective for the unstable repository
(https://docs.intelmq.org/latest/dev/bot-development/?h=unstable#testing-pre…).
These changes were backed by CSIRT.LI. Thank you!
More details:
New package 'intelmq-contrib'
-----------------------------
- EventDB tools:
-
https://github.com/certtools/intelmq/tree/develop/contrib/eventdb#readme
- A script to apply the Malware Name Mapping to an existing database
- A script to apply Domain Suffixes to an existing database
- A PostgreSQL trigger keeping track of the oldest "time.source"
- A script to export EventDB data to JSON, to use it in IntelMQ again
- Example extension package template
- https://docs.intelmq.org/latest/dev/extensions-packages/
- Feeds Config Generator
-
https://github.com/certtools/intelmq/tree/develop/contrib/feeds-config-gene…
- Malware Name Mapping Downloader
-
https://github.com/certtools/intelmq/tree/develop/contrib/malware_name_mapp…
- Script to update the locally downloaded mapping
- prettyprint script
- systemd tools
- logcheck rules:
- moved from the main package to the contrib package
-
https://github.com/certtools/intelmq/tree/develop/contrib/logcheck#readme
- A ruleset with patterns of (non-)error IntelMQ log lines for
alerting purposes
Drop of Ubuntu 20.04 packages
-----------------------------
As the package builds on Ubuntu 20.04 didn't work, and given that
standard support for this version already ended, the next IntelMQ
version will not be packaged for 20.04.
The target operating systems for the packages are then:
- Debian 11 Bullseye
- Debian 12 Bookworm
- Ubuntu 22.04 Jammy
- Ubuntu 24.04 Noble
If you have questions, concerns or other feedback about this, please get
in touch with us.
Best regards
--
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578
Hi,
I have installed intelmq and intelmq-manager o 2 platform:
almalinux 9 on bare metal using pypi installation
debian12 using dpkg installation
In both instances I am getting the same error:
Login failed with unknown reason. Please report this bug.
I am not sure where I went wrong.
This is the snippet from /var/log/httpd/httpd_log
72.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET
/intelmq-manager/plugins/dataTables/dataTables.bootstrap.js HTTP/1.1" 200
9614 "http://delta.bc.edu/intelmq-m
anager/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36"
172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET
/intelmq-manager/js/dynvar.js HTTP/1.1" 200 82 "-" "Mozilla/5.0 (Macintosh;
Intel Mac OS X 10_15_7) AppleWebK
it/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36"
172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET
/intelmq-manager/js/var.js HTTP/1.1" 404 196 "-" "Mozilla/5.0 (Macintosh;
Intel Mac OS X 10_15_7) AppleWebKit
/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36"
172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET
/intelmq-manager/plugins/metisMenu/metisMenu.js HTTP/1.1" 200 2268 "-"
"Mozilla/5.0 (Macintosh; Intel Mac OS
X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0
Safari/537.36"
172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET
/intelmq-manager/js/sb-admin-2.js HTTP/1.1" 200 1808 "-" "Mozilla/5.0
(Macintosh; Intel Mac OS X 10_15_7) App
leWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36"
172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET
/intelmq-manager/js/static.js HTTP/1.1" 200 19876 "-" "Mozilla/5.0
(Macintosh; Intel Mac OS X 10_15_7) AppleW
ebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36"
172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET
/intelmq-manager/js/intelmq-manager.js HTTP/1.1" 200 847 "-" "Mozilla/5.0
(Macintosh; Intel Mac OS X 10_15_7)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36"
172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET
/intelmq-manager/plugins/bootstrap/bootstrap.min.js HTTP/1.1" 200 39680 "-"
"Mozilla/5.0 (Macintosh; Intel Ma
c OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0
Safari/537.36"
172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET
/intelmq-manager/plugins/dataTables/dataTables.bootstrap.js HTTP/1.1" 200
9614 "-" "Mozilla/5.0 (Macintosh; I
ntel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/
134.0.0.0 Safari/537.36"
172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET
/intelmq-manager/plugins/dataTables/jquery.dataTables.js HTTP/1.1" 200
445793 "-" "Mozilla/5.0 (Macintosh; In
tel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/
134.0.0.0 Safari/537.36"
172.17.10.149 - - [24/Mar/2025:08:54:27 -0400] "POST /intelmq/v1/api/login
HTTP/1.1" 503 299 "http://delta.bc.edu/intelmq-manager/" "Mozilla/5.0
(Macintosh; Inte
l Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0
Safari/537.36"
--
Guy Smallwood
System Security Engineer
St. Clements
Security +
smallwog(a)bc.edu
Hello everyone!
As we all know, Friday evenings are a great time to release a new
software version, and there it is, the log awaited feature release 3.4.0! :)
Version 3.4.0 of IntelMQ brings changes to 16 bots and two new bots. It
contains changes of four contributors: Kamil Mankowski, Sebastian
Wagner, Radek Vyhnal and Frank Westers. A big thanks goes to CSIRT.LI
for making this release possible!
Please refer to https://docs.intelmq.org/latest/admin/upgrade/ for
upgrade instructions
The release is available in the GitHub repository, on PyPI and in the
deb-package repositories.
Read the full NEWS and changelog here:
https://github.com/certtools/intelmq/blob/develop/NEWS.md#340-feature-relea…https://github.com/certtools/intelmq/blob/develop/CHANGELOG.md#340-feature-…
The most important changes potentially requiring administration attention:
- Requirements: Python 3.8 or newer is required.
- /CIF 3 API Output/ is deprecated
- /Twitter Collector/ is removed (was dysfunctional)
- The /Twitter Parser/ is renamed to /IoC Extractor Parser/
(/intelmq.bots.parsers.ioc_extractor/).
- Packages are now also available for Ubuntu 24.04. To upgrade an Ubuntu
22.04 installation to 24.04 please refer to the Ubuntu documentation:
https://documentation.ubuntu.com/server/how-to/software/upgrade-your-releas…
Please refer to the NEWS file linked above for more details on these
changes.
We encourage you to share your feedback with us, also positive news
about seamless upgrades :)
We don't hope you experience any abnormal behavior, but if you do,
please report it to us via GitHub or e-mail.
best regards
Sebastian
for the IntelMQ maintainer group: Aaron, Kamil, Sebastian
--
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578
Dear IntelMQ users & developers,
To follow up with changes to IntelMQ Data Format proposed over a year
ago (have I heard "long To-Do list"?), I have opened following PRs:
https://github.com/certtools/intelmq/pull/2575 - severity field
https://github.com/certtools/intelmq/pull/2573 - constituency field (IEP008)
https://github.com/certtools/intelmq/pull/2574 - product.* fields (IEP009)
I'd like to ask you to look if the proposed version still answers yours
needs, and eventually support merging them ;) I hope we could agree on
merging soon and see them in next minor release.
--
Best regards
// Kamil Mańkowski <mankowski(a)cert.at> - T: +43 676 898 298 7204
// CERT Austria - https://www.cert.at/
// CERT.at GmbH, FB-Nr. 561772k, HG Wien
Dear IntelMQ community
With Ubuntu 20.04 as our oldest supported target platform using Python
3.8 and all other platforms using newer Python versions, it is time to
drop IntelMQ's official support for Python 3.7 in the subsequent releases[0]
The Security support for 3.7 has ended in June 2023.
What does that mean in practice?
We no longer run the test suites on 3.7 and IntelMQ requires Python 3.8
at installation time.
If you're running IntelMQ on an older system, we strongly recommend
upgrading your environment to ensure it remains secure and up-to-date.
Should you require assistance with the upgrade process, please don't
hesitate to contact us for support.[1]
At the other end of the spectrum, we added 3.12 and 3.13 to our test
suite, which already revealed the issue with the cif3 output bot as you
read in my other e-mail today.
[0]: https://github.com/certtools/intelmq/pull/2541
[1]: https://docs.intelmq.org/latest/help/#assistance
--
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578
Dear IntelMQ users
We are forced to deprecate and likely later remove the CIF3 Output
bot[0], which was originally contributed in 2022 by REN-ISAC.
Background:
The bot depends on the cifsdk[1] library < 4.0. Even the latest release
4.0 was published in 2019; and both versions, 3 and 4, are not
compatible with Python >= 3.12.
Further, the cifsdk library[1] hasn't received any activity since 5
years and has since stopped receiving maintenance.
Even a v5 CIF library[3] by the same author was stopped 4 years ago.
Therefore, the output bot needs an overhaul.
If you are using this bot and/or willing to contribute to IntelMQ,
please get in touch.
best regards
Sebastian
[0]: https://docs.intelmq.org/latest/user/bots/#cifv3-api
[1]: https://github.com/csirtgadgets/cifsdk-v4-py
[2]: https://github.com/certtools/intelmq/issues/2543
[3]: https://github.com/csirtgadgets/cif-v5
Do you need assistance? Have a look at
https://docs.intelmq.org/latest/help/#assistance
--
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578
Dear IntelMQ community, users, developers, and Incident Response teams!
We are excited to announce the release of IntelMQ version 3.3.1, which
includes important bug fixes.
The updated version is available on PyPI, in the git repository, and the
deb/rpm repositories.
Please see the list of all changes below.
Documentation: https://doc.intelmq.org/
Source code: https://github.com/certtools/intelmq
Thanks to all contributors to this release, in alphabetical order:
* DigitalTrustCenter
* Edvard Rejthar (CSIRT.CZ)
* elsif2 (Shadowserver Foundation)
* Kamil Mankowski (CERT.at)
* Mikk Margus Möll (CERT.ee)
* Sebastian Wagner (Institute for Common Good Technology, Intevation &
BSI)
The full list of changes:
Core
====
- `intelmq.lib.utils.drop_privileges`: When IntelMQ is called as `root`
and dropping the privileges to user `intelmq`, also set the non-primary
groups associated with the `intelmq` user. Makes the behaviour of
running intelmqctl as `root` closer to the behaviour of `sudo -u intelmq
...` (PR#2507 by Mikk Margus Möll).
- `intelmq.lib.utils.unzip`: Ignore directories themselves when
extracting data to prevent the extraction of empty data for a directory
entries (PR#2512 by Kamil Mankowski).
Bots
====
Collectors
----------
- `intelmq.bots.collectors.shadowserver.collector_reports_api.py`:
- Added support for the types parameter to be either a string or a
list (PR#2495 by elsif2).
- Refactored to utilize the type field returned by the API to match
the requested types instead of a sub-string match on the filename.
- Fixed timezone issue for collecting reports (PR#2506 by elsif2).
- Fixed behaviour if parameter `reports` value is empty string,
behave the same way as not set, not like no report (PR#2523 by Sebastian
Wagner).
- `intelmq.bots.collectors.shodan.collector_stream` (PR#2492 by Mikk
Margus Möll):
- Add `alert` parameter to Shodan stream collector to allow fetching
streams by configured alert ID
- `intelmq.bots.collectors.mail._lib`: Remove deprecated parameter
`attach_unzip` from default parameters (PR#2511 by Sebastian Wagner).
Parsers
-------
- `intelmq.bots.parsers.shadowserver._config`:
- Fetch schema before first run (PR#2482 by elsif2, fixes #2480).
- `intelmq.bots.parsers.dataplane.parser`: Use ` | ` as field
delimiter, fix parsing of AS names including `|` (PR#2488 by
DigitalTrustCenter).
- all parsers: add `copy_collector_provided_fields` parameter allowing
copying additional fields from the report, e.g. `extra.file_name`.
(PR#2513 by Kamil Mankowski).
Experts
-------
- `intelmq.bots.experts.sieve.expert`:
- For `:contains`, `=~` and `!~`, convert the value to string before
matching avoiding an exception. If the value is a dict, convert the
value to JSON (PR#2500 by Sebastian Wagner).
- Add support for variables in Sieve scripts (PR#2514 by Mikk Margus
Möll, fixes #2486).
- `intelmq.bots.experts.filter.expert`:
- Treat value `false` for parameter `filter_regex` as false (PR#2499
by Sebastian Wagner).
Outputs
-------
- `intelmq.bots.outputs.misp.output_feed`: Handle failures if saved
current event wasn't saved or is incorrect (PR by Kamil Mankowski).
- `intelmq.bots.outputs.smtp_batch.output`: Documentation on multiple
recipients added (PR#2501 by Edvard Rejthar).
Documentation
=============
- Bots: Clarify some section of Mail collectors and the Generic CSV
Parser (PR#2510 by Sebastian Wagner).
--
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578