- IntelMQ-users

Packages: New contrib subpackage, drop Ubuntu 20.04
by Sebix 04 Aug '25

04 Aug '25

Dear IntelMQ operators We have two news for you regarding the deb packages for Debian and Ubuntu: - A new package 'intelmq-contrib' - Drop of Ubuntu 20.04 packages For the standard repository, they will be in effect with the next IntelMQ release 4.1.0 in August. The changes are already effective for the unstable repository (https://docs.intelmq.org/latest/dev/bot-development/?h=unstable#testing-pre…). These changes were backed by CSIRT.LI. Thank you! More details: New package 'intelmq-contrib' ----------------------------- - EventDB tools: - https://github.com/certtools/intelmq/tree/develop/contrib/eventdb#readme - A script to apply the Malware Name Mapping to an existing database - A script to apply Domain Suffixes to an existing database - A PostgreSQL trigger keeping track of the oldest "time.source" - A script to export EventDB data to JSON, to use it in IntelMQ again - Example extension package template - https://docs.intelmq.org/latest/dev/extensions-packages/ - Feeds Config Generator - https://github.com/certtools/intelmq/tree/develop/contrib/feeds-config-gene… - Malware Name Mapping Downloader - https://github.com/certtools/intelmq/tree/develop/contrib/malware_name_mapp… - Script to update the locally downloaded mapping - prettyprint script - systemd tools - logcheck rules: - moved from the main package to the contrib package - https://github.com/certtools/intelmq/tree/develop/contrib/logcheck#readme - A ruleset with patterns of (non-)error IntelMQ log lines for alerting purposes Drop of Ubuntu 20.04 packages ----------------------------- As the package builds on Ubuntu 20.04 didn't work, and given that standard support for this version already ended, the next IntelMQ version will not be packaged for 20.04. The target operating systems for the packages are then: - Debian 11 Bullseye - Debian 12 Bookworm - Ubuntu 22.04 Jammy - Ubuntu 24.04 Noble If you have questions, concerns or other feedback about this, please get in touch with us. Best regards -- Institute for Common Good Technology gemeinnütziger Kulturverein - nonprofit cultural society https://commongoodtechnology.org/ ZVR 1510673578

1 0

Login failed with unknown reason. Please report this bug.
by Guy Smallwood 24 Mar '25

24 Mar '25

Hi, I have installed intelmq and intelmq-manager o 2 platform: almalinux 9 on bare metal using pypi installation debian12 using dpkg installation In both instances I am getting the same error: Login failed with unknown reason. Please report this bug. I am not sure where I went wrong. This is the snippet from /var/log/httpd/httpd_log 72.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET /intelmq-manager/plugins/dataTables/dataTables.bootstrap.js HTTP/1.1" 200 9614 "http://delta.bc.edu/intelmq-m anager/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36" 172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET /intelmq-manager/js/dynvar.js HTTP/1.1" 200 82 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebK it/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36" 172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET /intelmq-manager/js/var.js HTTP/1.1" 404 196 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit /537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36" 172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET /intelmq-manager/plugins/metisMenu/metisMenu.js HTTP/1.1" 200 2268 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36" 172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET /intelmq-manager/js/sb-admin-2.js HTTP/1.1" 200 1808 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) App leWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36" 172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET /intelmq-manager/js/static.js HTTP/1.1" 200 19876 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleW ebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36" 172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET /intelmq-manager/js/intelmq-manager.js HTTP/1.1" 200 847 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36" 172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET /intelmq-manager/plugins/bootstrap/bootstrap.min.js HTTP/1.1" 200 39680 "-" "Mozilla/5.0 (Macintosh; Intel Ma c OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36" 172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET /intelmq-manager/plugins/dataTables/dataTables.bootstrap.js HTTP/1.1" 200 9614 "-" "Mozilla/5.0 (Macintosh; I ntel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ 134.0.0.0 Safari/537.36" 172.17.10.149 - - [24/Mar/2025:08:54:15 -0400] "GET /intelmq-manager/plugins/dataTables/jquery.dataTables.js HTTP/1.1" 200 445793 "-" "Mozilla/5.0 (Macintosh; In tel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ 134.0.0.0 Safari/537.36" 172.17.10.149 - - [24/Mar/2025:08:54:27 -0400] "POST /intelmq/v1/api/login HTTP/1.1" 503 299 "http://delta.bc.edu/intelmq-manager/" "Mozilla/5.0 (Macintosh; Inte l Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36" -- Guy Smallwood System Security Engineer St. Clements Security + smallwog(a)bc.edu

2 1

IntelMQ 3.4.0 with new bots and support for Ubuntu 24.04
by Sebix 14 Mar '25

14 Mar '25

Hello everyone! As we all know, Friday evenings are a great time to release a new software version, and there it is, the log awaited feature release 3.4.0! :) Version 3.4.0 of IntelMQ brings changes to 16 bots and two new bots. It contains changes of four contributors: Kamil Mankowski, Sebastian Wagner, Radek Vyhnal and Frank Westers. A big thanks goes to CSIRT.LI for making this release possible! Please refer to https://docs.intelmq.org/latest/admin/upgrade/ for upgrade instructions The release is available in the GitHub repository, on PyPI and in the deb-package repositories. Read the full NEWS and changelog here: https://github.com/certtools/intelmq/blob/develop/NEWS.md#340-feature-relea… https://github.com/certtools/intelmq/blob/develop/CHANGELOG.md#340-feature-… The most important changes potentially requiring administration attention: - Requirements: Python 3.8 or newer is required. - /CIF 3 API Output/ is deprecated - /Twitter Collector/ is removed (was dysfunctional) - The /Twitter Parser/ is renamed to /IoC Extractor Parser/ (/intelmq.bots.parsers.ioc_extractor/). - Packages are now also available for Ubuntu 24.04. To upgrade an Ubuntu 22.04 installation to 24.04 please refer to the Ubuntu documentation: https://documentation.ubuntu.com/server/how-to/software/upgrade-your-releas… Please refer to the NEWS file linked above for more details on these changes. We encourage you to share your feedback with us, also positive news about seamless upgrades :) We don't hope you experience any abnormal behavior, but if you do, please report it to us via GitHub or e-mail. best regards Sebastian for the IntelMQ maintainer group: Aaron, Kamil, Sebastian -- Institute for Common Good Technology gemeinnütziger Kulturverein - nonprofit cultural society https://commongoodtechnology.org/ ZVR 1510673578

1 0

Implementing proposed severity, constituency and product.* fields
by Kamil Mankowski 03 Mar '25

03 Mar '25

Dear IntelMQ users & developers, To follow up with changes to IntelMQ Data Format proposed over a year ago (have I heard "long To-Do list"?), I have opened following PRs: https://github.com/certtools/intelmq/pull/2575 - severity field https://github.com/certtools/intelmq/pull/2573 - constituency field (IEP008) https://github.com/certtools/intelmq/pull/2574 - product.* fields (IEP009) I'd like to ask you to look if the proposed version still answers yours needs, and eventually support merging them ;) I hope we could agree on merging soon and see them in next minor release. -- Best regards // Kamil Mańkowski <mankowski(a)cert.at> - T: +43 676 898 298 7204 // CERT Austria - https://www.cert.at/ // CERT.at GmbH, FB-Nr. 561772k, HG Wien

1 0

Python versions: Removing 3.7, adding 3.12 and 3.13
by Sebix 01 Dec '24

01 Dec '24

Dear IntelMQ community With Ubuntu 20.04 as our oldest supported target platform using Python 3.8 and all other platforms using newer Python versions, it is time to drop IntelMQ's official support for Python 3.7 in the subsequent releases[0] The Security support for 3.7 has ended in June 2023. What does that mean in practice? We no longer run the test suites on 3.7 and IntelMQ requires Python 3.8 at installation time. If you're running IntelMQ on an older system, we strongly recommend upgrading your environment to ensure it remains secure and up-to-date. Should you require assistance with the upgrade process, please don't hesitate to contact us for support.[1] At the other end of the spectrum, we added 3.12 and 3.13 to our test suite, which already revealed the issue with the cif3 output bot as you read in my other e-mail today. [0]: https://github.com/certtools/intelmq/pull/2541 [1]: https://docs.intelmq.org/latest/help/#assistance -- Institute for Common Good Technology gemeinnütziger Kulturverein - nonprofit cultural society https://commongoodtechnology.org/ ZVR 1510673578

1 0

CIF3 output bot deprecation and removal
by Sebix 01 Dec '24

01 Dec '24

Dear IntelMQ users We are forced to deprecate and likely later remove the CIF3 Output bot[0], which was originally contributed in 2022 by REN-ISAC. Background: The bot depends on the cifsdk[1] library < 4.0. Even the latest release 4.0 was published in 2019; and both versions, 3 and 4, are not compatible with Python >= 3.12. Further, the cifsdk library[1] hasn't received any activity since 5 years and has since stopped receiving maintenance. Even a v5 CIF library[3] by the same author was stopped 4 years ago. Therefore, the output bot needs an overhaul. If you are using this bot and/or willing to contribute to IntelMQ, please get in touch. best regards Sebastian [0]: https://docs.intelmq.org/latest/user/bots/#cifv3-api [1]: https://github.com/csirtgadgets/cifsdk-v4-py [2]: https://github.com/certtools/intelmq/issues/2543 [3]: https://github.com/csirtgadgets/cif-v5 Do you need assistance? Have a look at https://docs.intelmq.org/latest/help/#assistance -- Institute for Common Good Technology gemeinnütziger Kulturverein - nonprofit cultural society https://commongoodtechnology.org/ ZVR 1510673578

1 0

IntelMQ 3.3.1 bugfix release
by Sebix 03 Sep '24

03 Sep '24

Dear IntelMQ community, users, developers, and Incident Response teams! We are excited to announce the release of IntelMQ version 3.3.1, which includes important bug fixes. The updated version is available on PyPI, in the git repository, and the deb/rpm repositories. Please see the list of all changes below. Documentation: https://doc.intelmq.org/ Source code: https://github.com/certtools/intelmq Thanks to all contributors to this release, in alphabetical order: * DigitalTrustCenter * Edvard Rejthar (CSIRT.CZ) * elsif2 (Shadowserver Foundation) * Kamil Mankowski (CERT.at) * Mikk Margus Möll (CERT.ee) * Sebastian Wagner (Institute for Common Good Technology, Intevation & BSI) The full list of changes: Core ==== - `intelmq.lib.utils.drop_privileges`: When IntelMQ is called as `root` and dropping the privileges to user `intelmq`, also set the non-primary groups associated with the `intelmq` user. Makes the behaviour of running intelmqctl as `root` closer to the behaviour of `sudo -u intelmq ...` (PR#2507 by Mikk Margus Möll). - `intelmq.lib.utils.unzip`: Ignore directories themselves when extracting data to prevent the extraction of empty data for a directory entries (PR#2512 by Kamil Mankowski). Bots ==== Collectors ---------- - `intelmq.bots.collectors.shadowserver.collector_reports_api.py`: - Added support for the types parameter to be either a string or a list (PR#2495 by elsif2). - Refactored to utilize the type field returned by the API to match the requested types instead of a sub-string match on the filename. - Fixed timezone issue for collecting reports (PR#2506 by elsif2). - Fixed behaviour if parameter `reports` value is empty string, behave the same way as not set, not like no report (PR#2523 by Sebastian Wagner). - `intelmq.bots.collectors.shodan.collector_stream` (PR#2492 by Mikk Margus Möll): - Add `alert` parameter to Shodan stream collector to allow fetching streams by configured alert ID - `intelmq.bots.collectors.mail._lib`: Remove deprecated parameter `attach_unzip` from default parameters (PR#2511 by Sebastian Wagner). Parsers ------- - `intelmq.bots.parsers.shadowserver._config`: - Fetch schema before first run (PR#2482 by elsif2, fixes #2480). - `intelmq.bots.parsers.dataplane.parser`: Use ` | ` as field delimiter, fix parsing of AS names including `|` (PR#2488 by DigitalTrustCenter). - all parsers: add `copy_collector_provided_fields` parameter allowing copying additional fields from the report, e.g. `extra.file_name`. (PR#2513 by Kamil Mankowski). Experts ------- - `intelmq.bots.experts.sieve.expert`: - For `:contains`, `=~` and `!~`, convert the value to string before matching avoiding an exception. If the value is a dict, convert the value to JSON (PR#2500 by Sebastian Wagner). - Add support for variables in Sieve scripts (PR#2514 by Mikk Margus Möll, fixes #2486). - `intelmq.bots.experts.filter.expert`: - Treat value `false` for parameter `filter_regex` as false (PR#2499 by Sebastian Wagner). Outputs ------- - `intelmq.bots.outputs.misp.output_feed`: Handle failures if saved current event wasn't saved or is incorrect (PR by Kamil Mankowski). - `intelmq.bots.outputs.smtp_batch.output`: Documentation on multiple recipients added (PR#2501 by Edvard Rejthar). Documentation ============= - Bots: Clarify some section of Mail collectors and the Generic CSV Parser (PR#2510 by Sebastian Wagner). -- Institute for Common Good Technology gemeinnütziger Kulturverein - nonprofit cultural society https://commongoodtechnology.org/ ZVR 1510673578

1 0

Valkey replacing Redis in Linux distros
by Sebix 08 Jul '24

08 Jul '24

Dear IntelMQ users With Redis' license model change[0], the fork 'Valkey'[1] has emerged as the drop-in replacement following the spirit of FOSS the closest. It is expected that Linux distros will replace redis with valkey. FreeBSD already provides valkey[2], but hasn't dropped redis yet. openSUSE has already dropped redis[3] in favor of valkey in their rolling-release/development product. And Debian is considering the change too[4]. Before this change comes into effect in the distros' stable releases, one or two years will probably still pass, so this is an early heads-up :) You may also consider using other redis-replacements[5], but none of them are yet tested thoroughly with IntelMQ best regards Sebastian [0]: https://redis.io/blog/redis-adopts-dual-source-available-licensing/ [1]: https://github.com/valkey-io/valkey [2]: https://www.freshports.org/databases/valkey/ [3]: https://build.opensuse.org/request/show/1182574 [4]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068342 [5]: https://github.com/certtools/intelmq/issues/2222 -- Institute for Common Good Technology gemeinnütziger Kulturverein - nonprofit cultural society https://commongoodtechnology.org/ ZVR 1510673578

1 0

mail collector extra fields
by Thomas Hungenberg 08 Jul '24

08 Jul '24

Hello, according to <https://github.com/certtools/intelmq/blob/develop/docs/user/bots.md> events collected using a "Generic Mail URL Fetcher" should include this information: feed.url extra.email_date extra.email_subject extra.email_from extra.email_message_id extra.file_name In our database, the events DO include feed.url but DO NOT include any of the extra fields. Events collected using a "Generic Mail Attachment Fetcher" are missing the extra fields as well. I wonder if this is a bug or caused by some configuration issue with our setup. - Thomas

3 2

intelmq 3.3.0 release
by L. Aaron Kaplan 01 Mar '24

01 Mar '24

Dear intelmq users, developers and IR automation people, we are very happy to announce (finally announce!) IntelMQ 3.3.0 See: https://github.com/certtools/intelmq Important changes =================== The most relevant changes are: 1. Documentation ------------------ great new documentation (mkdocs) at https://docs.intelmq.org/ (Thanks a lot to Filip/ @gethvi! Amazing work) Documentation has been updated and restructured into User, Administrator and Developer Guide. The documentation is easier to navigate, slicker and just way more readable now. We know that the user-experience with a framework such as IntelMQ is very important. Good documentation helps a lot here. Please do check it out and give us feedback! 2. dynamic shadowserver reports and -parser ------------------------------------------- Lots of thanks to Shadowserver / @elsif2 for his continuous and dedicated contributions to make IntelMQ much better with shadowserver feeds. Motivation for this: Shadowserver adds new scans on a nearly weekly basis. IntelMQ's release cycle and the need for a stable release could not keep up with this high intensity of shadowserver parser changes. We therefore (thanks to @eslif2) move the shadowserver reports collector and parser to a new, dynamic system. It can: • fetch the shadowserver schema from shadowserver (https://interchange.shadowserver.org/intelmq/v1/schema) • dynamically collect new reports (see also https://docs.intelmq.org/latest/user/bots/?h=shadow#shadowserver-reports-api) • parse the new reports You can find all about the new shadowserver reports in IntelMQ here: https://docs.intelmq.org/latest/user/bots/#shadowserver-reports-api and https://docs.intelmq.org/latest/user/bots/#shadowserver 3. More ------------ And of course, lots of changes in the background and the core. Big thanks to Kamil and Sebix! You can find the full details here: https://github.com/certtools/intelmq/blob/develop/NEWS.md (short version) and here: https://github.com/certtools/intelmq/blob/develop/CHANGELOG.md 4. Future of IntelMQ and release cycles ---------------------------------------- We know that the last one, two years, IntelMQ releases were a bit sporadic. We plan to change this and make it more deterministic. We therefore commit to 2 releases per year + two release candidates (intermediate, BETA) releases: Release 1: in quarter 1: March. Release 2: release-candidate in Q2 (end of June). Release 3: Sept Release 4: release-candidate release in Q4 (Dec). Also, in order to secure the future sustainability of IntelMQ, we will join the "ossbase" Open Source Security Software alliance. The idea here is to join a bigger foundation (similarly structured to the Apache Foundation) which will be created together with CIRCL.lu. Thanks Alexandre for continuing to push this idea. Being a part of ossbase will allow us to support IntelMQ in the future years while keeping the project independent, alive, kicking and a great community project. And that's what relevant. Stay tuned for updates here. 5. Thank you -------------- And finally and again: a big THANK YOU for all the committers, contributors, bug reporters, etc.!!! Without this community, we would not exist. You are mentioned here: https://github.com/certtools/intelmq/blob/develop/AUTHORS https://github.com/certtools/intelmq/blob/develop/CHANGELOG.md https://github.com/certtools/intelmq/issues

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

IntelMQ-users