Hello,
I would like to propose a change to the mapping of the 'source' field in
the blocklist report which is currently mapped to 'extra.source' in
IntelMQ to 'extra.source_name'.
This change is necessary to avoid conflicts with the elasticsearch
mapping of the 'extra.source' field where it is commonly an object
instead of a string.
Reports details can be found here:
https://www.shadowserver.org/what-we-do/network-reporting/blocklist-report/
Regards,
Jason
Hi,
I'm trying to set up intelmq-manager to a fresh Ubuntu 24 LTS but I keep bumping into this:
E: Failed to fetch http://some-mirror-host-name-here/pub/opensuse/repositories/home%3A/sebix%3… File has unexpected size (3960404 != 3960834). Mirror sync in progress? [IP: some-mirror-hosts-IP-here 80]
Hashes of expected file:
- SHA256:5269aabe03adb09d03df161294dd41fe1798f3e9b4fefbb88647e13672093cbe
- SHA1:2c028c8bf346d0591747d00cad5501b6e18a7e79 [weak]
- MD5Sum:fbcee8e105b081b5c5edf635f5c15626 [weak]
- Filesize:3960834 [weak]
It's quite seldom I've encountered a mismatch in file size checks so is the package somehow broken or have I missed something?
Br, Mika
The information in this email may be confidential and is intended solely for the use of the individual or entity to whom it is intended. If you are not the intended recipient of this message, please delete the message and notify the sender immediately. For information on how we process personal data and our contact information, please see CSC's website: Privacy<https://csc.fi/en/privacy>
T?m?n s?hk?postin tiedot voivat olla luottamuksellisia ja ne on tarkoitettu yksinomaan sen henkil?n tai yhteis?n k?ytt??n, jolle ne on osoitettu. Jos et ole viestiss? tarkoitettu vastaanottaja, tuhoa viesti ja ilmoita asiasta v?litt?m?sti viestin l?hett?j?lle. Tietoja henkil?tietojen ja yhteystietojen k?sittelyst? l?yd?t CSC:n verkkosivuilta: Tietosuoja<https://csc.fi/tietosuoja>
Hi,
We received one sinkhole http referer event that got flagged as not belonging to our constituency. At a closer look, it turned out that intelmq's destination.ip field (see event4_sinkhole_http_referer in https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/inte…<https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/inte…>)
contains the IP the malware tried to access (?) and then the IP of the machine possibly infected by the malware ended up in extra.http_referer_ip. Everything fine and coherent with the feed's documentation, no complaints.
To deduce to whom in our constituency an event belongs, we have a bot that first looks at the source.ip field and if this does not match, then we try the destination.ip field. This heuristic works fine with all feeds but in the above case it fails.
Is there a chance the report field http_referer_ip gets mapped one day in intelmq.json to intelmq's source.ip field (like in the majority of other feeds) instead of extra.http_referer_ip? Just asking since I'd like to keep things simpler and avoid work-arounds.
Br, Mika
The information in this email may be confidential and is intended solely for the use of the individual or entity to whom it is intended. If you are not the intended recipient of this message, please delete the message and notify the sender immediately. For information on how we process personal data and our contact information, please see CSC's website: Privacy<https://csc.fi/en/privacy>
T?m?n s?hk?postin tiedot voivat olla luottamuksellisia ja ne on tarkoitettu yksinomaan sen henkil?n tai yhteis?n k?ytt??n, jolle ne on osoitettu. Jos et ole viestiss? tarkoitettu vastaanottaja, tuhoa viesti ja ilmoita asiasta v?litt?m?sti viestin l?hett?j?lle. Tietoja henkil?tietojen ja yhteystietojen k?sittelyst? l?yd?t CSC:n verkkosivuilta: Tietosuoja<https://csc.fi/tietosuoja>
Hi Sebastian,
Thanks for a quick response. It appears the IPv6 "sibling" of the feed, i.e. the event6_sinkhole_http_referer also needs the same small change of field names (extra.http_referer_ip -> source.ip). I haven't made a thorough check, so I'm not sure if these are the only ones.
Br, Mika
________________________________
From: Sebastian Wagner
Sent: Monday, March 31, 2025 11.41
To: intelmq-dev(a)lists.cert.at
Cc: Mika Silander; elsif
Subject: Re: [IntelMQ-dev] A question on Shadowserver's Sinkhole HTTP Referer Events feed
Hi Mika
I agree with you. As per IntelMQ's guidelines/documentation (https://docs.intelmq.org/latest/user/event/#meaning-of-source-and-destinati…) the data we care about is always in the source part, and the IP address in source.ip.
Jason, can you please adapt the mapping?
best regards
Sebastian
On 3/31/25 10:08 AM, Mika Silander via IntelMQ-dev wrote:
Hi,
We received one sinkhole http referer event that got flagged as not belonging to our constituency. At a closer look, it turned out that intelmq's destination.ip field (see event4_sinkhole_http_referer in https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/inte…<https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/inte…>)
contains the IP the malware tried to access (?) and then the IP of the machine possibly infected by the malware ended up in extra.http_referer_ip. Everything fine and coherent with the feed's documentation, no complaints.
To deduce to whom in our constituency an event belongs, we have a bot that first looks at the source.ip field and if this does not match, then we try the destination.ip field. This heuristic works fine with all feeds but in the above case it fails.
Is there a chance the report field http_referer_ip gets mapped one day in intelmq.json to intelmq's source.ip field (like in the majority of other feeds) instead of extra.http_referer_ip? Just asking since I'd like to keep things simpler and avoid work-arounds.
Br, Mika
The information in this email may be confidential and is intended solely for the use of the individual or entity to whom it is intended. If you are not the intended recipient of this message, please delete the message and notify the sender immediately. For information on how we process personal data and our contact information, please see CSC's website: Privacy<https://csc.fi/en/privacy>
Tämän sähköpostin tiedot voivat olla luottamuksellisia ja ne on tarkoitettu yksinomaan sen henkilön tai yhteisön käyttöön, jolle ne on osoitettu. Jos et ole viestissä tarkoitettu vastaanottaja, tuhoa viesti ja ilmoita asiasta välittömästi viestin lähettäjälle. Tietoja henkilötietojen ja yhteystietojen käsittelystä löydät CSC:n verkkosivuilta: Tietosuoja<https://csc.fi/tietosuoja>
_______________________________________________
IntelMQ-dev mailing list -- intelmq-dev(a)lists.cert.at<mailto:intelmq-dev@lists.cert.at>
To unsubscribe send an email to intelmq-dev-leave(a)lists.cert.at<mailto:intelmq-dev-leave@lists.cert.at>
--
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578
Hello everyone!
As we all know, Friday evenings are a great time to release a new
software version, and there it is, the log awaited feature release 3.4.0! :)
Version 3.4.0 of IntelMQ brings changes to 16 bots and two new bots. It
contains changes of four contributors: Kamil Mankowski, Sebastian
Wagner, Radek Vyhnal and Frank Westers. A big thanks goes to CSIRT.LI
for making this release possible!
Please refer to https://docs.intelmq.org/latest/admin/upgrade/ for
upgrade instructions
The release is available in the GitHub repository, on PyPI and in the
deb-package repositories.
Read the full NEWS and changelog here:
https://github.com/certtools/intelmq/blob/develop/NEWS.md#340-feature-relea…https://github.com/certtools/intelmq/blob/develop/CHANGELOG.md#340-feature-…
The most important changes potentially requiring administration attention:
- Requirements: Python 3.8 or newer is required.
- /CIF 3 API Output/ is deprecated
- /Twitter Collector/ is removed (was dysfunctional)
- The /Twitter Parser/ is renamed to /IoC Extractor Parser/
(/intelmq.bots.parsers.ioc_extractor/).
- Packages are now also available for Ubuntu 24.04. To upgrade an Ubuntu
22.04 installation to 24.04 please refer to the Ubuntu documentation:
https://documentation.ubuntu.com/server/how-to/software/upgrade-your-releas…
Please refer to the NEWS file linked above for more details on these
changes.
We encourage you to share your feedback with us, also positive news
about seamless upgrades :)
We don't hope you experience any abnormal behavior, but if you do,
please report it to us via GitHub or e-mail.
best regards
Sebastian
for the IntelMQ maintainer group: Aaron, Kamil, Sebastian
--
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578
Dear IntelMQ users & developers,
To follow up with changes to IntelMQ Data Format proposed over a year
ago (have I heard "long To-Do list"?), I have opened following PRs:
https://github.com/certtools/intelmq/pull/2575 - severity field
https://github.com/certtools/intelmq/pull/2573 - constituency field (IEP008)
https://github.com/certtools/intelmq/pull/2574 - product.* fields (IEP009)
I'd like to ask you to look if the proposed version still answers yours
needs, and eventually support merging them ;) I hope we could agree on
merging soon and see them in next minor release.
--
Best regards
// Kamil Mańkowski <mankowski(a)cert.at> - T: +43 676 898 298 7204
// CERT Austria - https://www.cert.at/
// CERT.at GmbH, FB-Nr. 561772k, HG Wien
Hello,
The following schema changes are open for discussion and scheduled to be
committed on 2025-01-28.
Change summary
* Added scan_ip_tunnel and scan6_ip_tunnel reports.
* Revised the feed_name and url for the scan_msrpc report.
Full details can be found at https://github.com/elsif2/ieps/tree/main/010
Regards,
Jason