IntelMQ-dev

intelmq-dev@lists.cert.at

239 discussions

Shadowserver: proposed mapping change for blocklist.source
by elsif 18 Jul '25

18 Jul '25

Hello, I would like to propose a change to the mapping of the 'source' field in the blocklist report which is currently mapped to 'extra.source' in IntelMQ to 'extra.source_name'. This change is necessary to avoid conflicts with the elasticsearch mapping of the 'extra.source' field where it is commonly an object instead of a string. Reports details can be found here: https://www.shadowserver.org/what-we-do/network-reporting/blocklist-report/ Regards, Jason

2 2

Problem installing intelmq-manager 3.3.0-1 to Ubuntu 24 LTS
by Mika Silander 04 Apr '25

04 Apr '25

Hi, I'm trying to set up intelmq-manager to a fresh Ubuntu 24 LTS but I keep bumping into this: E: Failed to fetch http://some-mirror-host-name-here/pub/opensuse/repositories/home%3A/sebix%3… File has unexpected size (3960404 != 3960834). Mirror sync in progress? [IP: some-mirror-hosts-IP-here 80] Hashes of expected file: - SHA256:5269aabe03adb09d03df161294dd41fe1798f3e9b4fefbb88647e13672093cbe - SHA1:2c028c8bf346d0591747d00cad5501b6e18a7e79 [weak] - MD5Sum:fbcee8e105b081b5c5edf635f5c15626 [weak] - Filesize:3960834 [weak] It's quite seldom I've encountered a mismatch in file size checks so is the package somehow broken or have I missed something? Br, Mika The information in this email may be confidential and is intended solely for the use of the individual or entity to whom it is intended. If you are not the intended recipient of this message, please delete the message and notify the sender immediately. For information on how we process personal data and our contact information, please see CSC's website: Privacy<https://csc.fi/en/privacy> T?m?n s?hk?postin tiedot voivat olla luottamuksellisia ja ne on tarkoitettu yksinomaan sen henkil?n tai yhteis?n k?ytt??n, jolle ne on osoitettu. Jos et ole viestiss? tarkoitettu vastaanottaja, tuhoa viesti ja ilmoita asiasta v?litt?m?sti viestin l?hett?j?lle. Tietoja henkil?tietojen ja yhteystietojen k?sittelyst? l?yd?t CSC:n verkkosivuilta: Tietosuoja<https://csc.fi/tietosuoja>

1 0

A question on Shadowserver's Sinkhole HTTP Referer Events feed
by Mika Silander 02 Apr '25

02 Apr '25

Hi, We received one sinkhole http referer event that got flagged as not belonging to our constituency. At a closer look, it turned out that intelmq's destination.ip field (see event4_sinkhole_http_referer in https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/inte…<https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/inte…>) contains the IP the malware tried to access (?) and then the IP of the machine possibly infected by the malware ended up in extra.http_referer_ip. Everything fine and coherent with the feed's documentation, no complaints. To deduce to whom in our constituency an event belongs, we have a bot that first looks at the source.ip field and if this does not match, then we try the destination.ip field. This heuristic works fine with all feeds but in the above case it fails. Is there a chance the report field http_referer_ip gets mapped one day in intelmq.json to intelmq's source.ip field (like in the majority of other feeds) instead of extra.http_referer_ip? Just asking since I'd like to keep things simpler and avoid work-arounds. Br, Mika The information in this email may be confidential and is intended solely for the use of the individual or entity to whom it is intended. If you are not the intended recipient of this message, please delete the message and notify the sender immediately. For information on how we process personal data and our contact information, please see CSC's website: Privacy<https://csc.fi/en/privacy> T?m?n s?hk?postin tiedot voivat olla luottamuksellisia ja ne on tarkoitettu yksinomaan sen henkil?n tai yhteis?n k?ytt??n, jolle ne on osoitettu. Jos et ole viestiss? tarkoitettu vastaanottaja, tuhoa viesti ja ilmoita asiasta v?litt?m?sti viestin l?hett?j?lle. Tietoja henkil?tietojen ja yhteystietojen k?sittelyst? l?yd?t CSC:n verkkosivuilta: Tietosuoja<https://csc.fi/tietosuoja>

5 5

Re: A question on Shadowserver's Sinkhole HTTP Referer Events feed
by Mika Silander 31 Mar '25

31 Mar '25

Hi Sebastian, Thanks for a quick response. It appears the IPv6 "sibling" of the feed, i.e. the event6_sinkhole_http_referer also needs the same small change of field names (extra.http_referer_ip -> source.ip). I haven't made a thorough check, so I'm not sure if these are the only ones. Br, Mika ________________________________ From: Sebastian Wagner Sent: Monday, March 31, 2025 11.41 To: intelmq-dev(a)lists.cert.at Cc: Mika Silander; elsif Subject: Re: [IntelMQ-dev] A question on Shadowserver's Sinkhole HTTP Referer Events feed Hi Mika I agree with you. As per IntelMQ's guidelines/documentation (https://docs.intelmq.org/latest/user/event/#meaning-of-source-and-destinati…) the data we care about is always in the source part, and the IP address in source.ip. Jason, can you please adapt the mapping? best regards Sebastian On 3/31/25 10:08 AM, Mika Silander via IntelMQ-dev wrote: Hi, We received one sinkhole http referer event that got flagged as not belonging to our constituency. At a closer look, it turned out that intelmq's destination.ip field (see event4_sinkhole_http_referer in https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/inte…<https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/inte…>) contains the IP the malware tried to access (?) and then the IP of the machine possibly infected by the malware ended up in extra.http_referer_ip. Everything fine and coherent with the feed's documentation, no complaints. To deduce to whom in our constituency an event belongs, we have a bot that first looks at the source.ip field and if this does not match, then we try the destination.ip field. This heuristic works fine with all feeds but in the above case it fails. Is there a chance the report field http_referer_ip gets mapped one day in intelmq.json to intelmq's source.ip field (like in the majority of other feeds) instead of extra.http_referer_ip? Just asking since I'd like to keep things simpler and avoid work-arounds. Br, Mika The information in this email may be confidential and is intended solely for the use of the individual or entity to whom it is intended. If you are not the intended recipient of this message, please delete the message and notify the sender immediately. For information on how we process personal data and our contact information, please see CSC's website: Privacy<https://csc.fi/en/privacy> Tämän sähköpostin tiedot voivat olla luottamuksellisia ja ne on tarkoitettu yksinomaan sen henkilön tai yhteisön käyttöön, jolle ne on osoitettu. Jos et ole viestissä tarkoitettu vastaanottaja, tuhoa viesti ja ilmoita asiasta välittömästi viestin lähettäjälle. Tietoja henkilötietojen ja yhteystietojen käsittelystä löydät CSC:n verkkosivuilta: Tietosuoja<https://csc.fi/tietosuoja> _______________________________________________ IntelMQ-dev mailing list -- intelmq-dev(a)lists.cert.at<mailto:intelmq-dev@lists.cert.at> To unsubscribe send an email to intelmq-dev-leave(a)lists.cert.at<mailto:intelmq-dev-leave@lists.cert.at> -- Institute for Common Good Technology gemeinnütziger Kulturverein - nonprofit cultural society https://commongoodtechnology.org/ ZVR 1510673578

1 0

IntelMQ 3.4.0 with new bots and support for Ubuntu 24.04
by Sebix 14 Mar '25

14 Mar '25

Hello everyone! As we all know, Friday evenings are a great time to release a new software version, and there it is, the log awaited feature release 3.4.0! :) Version 3.4.0 of IntelMQ brings changes to 16 bots and two new bots. It contains changes of four contributors: Kamil Mankowski, Sebastian Wagner, Radek Vyhnal and Frank Westers. A big thanks goes to CSIRT.LI for making this release possible! Please refer to https://docs.intelmq.org/latest/admin/upgrade/ for upgrade instructions The release is available in the GitHub repository, on PyPI and in the deb-package repositories. Read the full NEWS and changelog here: https://github.com/certtools/intelmq/blob/develop/NEWS.md#340-feature-relea… https://github.com/certtools/intelmq/blob/develop/CHANGELOG.md#340-feature-… The most important changes potentially requiring administration attention: - Requirements: Python 3.8 or newer is required. - /CIF 3 API Output/ is deprecated - /Twitter Collector/ is removed (was dysfunctional) - The /Twitter Parser/ is renamed to /IoC Extractor Parser/ (/intelmq.bots.parsers.ioc_extractor/). - Packages are now also available for Ubuntu 24.04. To upgrade an Ubuntu 22.04 installation to 24.04 please refer to the Ubuntu documentation: https://documentation.ubuntu.com/server/how-to/software/upgrade-your-releas… Please refer to the NEWS file linked above for more details on these changes. We encourage you to share your feedback with us, also positive news about seamless upgrades :) We don't hope you experience any abnormal behavior, but if you do, please report it to us via GitHub or e-mail. best regards Sebastian for the IntelMQ maintainer group: Aaron, Kamil, Sebastian -- Institute for Common Good Technology gemeinnütziger Kulturverein - nonprofit cultural society https://commongoodtechnology.org/ ZVR 1510673578

1 0

Implementing proposed severity, constituency and product.* fields
by Kamil Mankowski 03 Mar '25

03 Mar '25

Dear IntelMQ users & developers, To follow up with changes to IntelMQ Data Format proposed over a year ago (have I heard "long To-Do list"?), I have opened following PRs: https://github.com/certtools/intelmq/pull/2575 - severity field https://github.com/certtools/intelmq/pull/2573 - constituency field (IEP008) https://github.com/certtools/intelmq/pull/2574 - product.* fields (IEP009) I'd like to ask you to look if the proposed version still answers yours needs, and eventually support merging them ;) I hope we could agree on merging soon and see them in next minor release. -- Best regards // Kamil Mańkowski <mankowski(a)cert.at> - T: +43 676 898 298 7204 // CERT Austria - https://www.cert.at/ // CERT.at GmbH, FB-Nr. 561772k, HG Wien

1 0

Shadowserver Accessible-MS-RPC-Endpoint-Mapper renamed to Accessible-MS-RPC-Service
by Mika Silander 22 Jan '25

22 Jan '25

Hi, It looks to me there's been a small rename of a feed in the Shadowserver offering: Accessible-MS-RPC-Endpoint-Mapper is now Accessible-MS-RPC-Service and the following changes are needed to https://github.com/The-Shadowserver-Foundation/report_schema/reports.json : - "name" : "Accessible-MS-RPC-Endpoint-Mapper", - "url" : "https://www.shadowserver.org/what-we-do/network-reporting/ms-rpc-endpoint-m…" + "name" : "Accessible-MS-RPC-Service", + "url" : "https://www.shadowserver.org/what-we-do/network-reporting/accessible-ms-rpc…" and intelmq.json : - "feed_name" : "Accessible-MS-RPC-Endpoint-Mapper", + "feed_name" : "Accessible-MS-RPC-Service", "file_name" : "scan_msrpc", "optional_fields" : [ [ @@ -22247,7 +22247,7 @@ "convert_int" ] ], - "url" : "https://www.shadowserver.org/what-we-do/network-reporting/ms-rpc-endpoint-m…" + "url" : "https://www.shadowserver.org/what-we-do/network-reporting/accessible-ms-rpc…" The field names within appear unchanged. Br, Mika

2 2

IEP010: Shadowserver schema update (2025-01-14)
by elsif 18 Jan '25

18 Jan '25

Hello, The following schema changes are open for discussion and scheduled to be committed on 2025-01-28. Change summary * Added scan_ip_tunnel and scan6_ip_tunnel reports. * Revised the feed_name and url for the scan_msrpc report. Full details can be found at https://github.com/elsif2/ieps/tree/main/010 Regards, Jason

4 5

Test message after update
by Kamil Mankowski 16 Jan '25

16 Jan '25

Hi all, I'm just checking the list after me migrated it yesterday to Mailman 3 (yes, we're in the XXI century now). We're not finished with the migration of all our lists yet, so some old links may not work temporary (the redirection should come soon). If you want to manage your subscription, please visit https://lists.cert.at/mailman3/postorius/lists/ -- Best regards // Kamil Mańkowski <mankowski(a)cert.at> - T: +43 676 898 298 7204 // CERT Austria - https://www.cert.at/ // CERT.at GmbH, FB-Nr. 561772k, HG Wien

1 0

Shadowserver parser schema: Harmonization of feed names
by Thomas Hungenberg 07 Jan '25

07 Jan '25

Hi all, the feed names currently do not have a consistent format. Some feed names have an "IPv6-" prefix but no indicator for IPv4, e.g. Accessible-FTP IPv6-Accessible-FTP while others have an " IPv4" / " IPv6" suffix: Device-Identification IPv4 Device-Identification IPv6 I'd like to see this harmonized. My preference would be to always use an IPv4 / IPv6 suffix like with Device-Identification. Or maybe better with a dash like Device-Identification-IPv4 Device-Identification-IPv6 to prevent configuration issues resulting from the blank. What do you think? Regards Thomas

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

IntelMQ-dev