- IntelMQ-users

Ending packages support for Ubuntu 18.04
by Sebix 25 May '22

25 May '22

Dear users, I intend to give up the deb-packages support for Ubuntu 18.04, i.e. the repository located at http://download.opensuse.org/repositories/home:/sebix:/intelmq/xUbuntu_18.0… Ubuntu Bionic behaves a bit different compared to it's newer colleagues and needs workarounds, not only in the packages, but also in the automated test chain[0]. I don't think it's worth the effort. If you need that repository, please speak up *now*. Otherwise, I will drop it end of June. cheers Sebastian [0]: https://github.com/certtools/intelmq-vagrant/

1 0

Update on my involvement in IntelMQ
by Sebix 18 May '22

18 May '22

Dear IntelMQ community, Last autumn I left my previous employer CERT.at. Now I want to give you a short update regarding my future involvement with IntelMQ. First, let me state I'm operating in different roles here. There is voluntary non-profit work ("sebix"[0]) and my work as part my employment. I recently joined Intevation[1] part-time. This is a Free Software company located in Germany. They are already engaged in IntelMQ since 2016 (1½ years after IntelMQ's beginning) and are e.g. contracted by CERT-BUND[2] for IntelMQ-related work. As part of that contract I'm able to contribute a small part of my work time to the community support and maintenance of IntelMQ. If you are interested in contracting with us, please get in touch. In addition, I have founded a non-profit association[0] for non-commercial initiatives and cooperation. If you are interested in collaborating with us, please get in contact with me. IntelMQ is still in need of mid- and long-term contribution and financing. One approach is led by Aaron, who works to form an umbrella organisational support structure for IntelMQ. Intevation and myself aim to complement this effort. cheers Sebastian [0]: https://sebix.at/ <https://intevation.de/> [1]: https://intevation.de/ [2]: https://www.cert-bund.de/

1 0

Installation problems intelmq and manager
by Harbich, Christian 25 Mar '22

25 Mar '22

Hi Guys, I followed the installation guide and run into a problem while using the intelmq-api to get a login token as mentioned here: https://intelmq.readthedocs.io/en/maintenance/user/intelmq-api.html I executed curl --location --request POST http://localhost/intelmq/v1/api/login/\ --header "Content-Type: application/x-www-form-urlencoded"\ --data-urlencode "username=$username"\ --data-urlencode "password=$password" And got the answer <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not Found</h1> <p>The requested URL was not found on this server.</p> <hr> <address>Apache/2.4.29 (Ubuntu) Server at localhost Port 80</address> </body></html> I'm running on Ubuntu 18.04 and used the native packages which I downloaded manually and installed them using dpkg. I also noticed that my apache2 only got a conf file for intelmq-api.conf ... shouldnt there be one for the intelmq itself when the curl statement on top is called? PS: I'm new to intelmq and also not an Ubuntu specialist so please be gentle :D Cheers Chris Aufsichtsratsvorsitzender: Norbert Rotter Geschäftsführung: Heino Feige, Michael Krüger Sitz der Gesellschaft: Halle/Saale Registergericht: Amtsgericht Stendal | Handelsregister-Nr. HRB 208414 UST-ID-Nr. DE 158253683 Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Empfänger sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail oder des Inhalts dieser Mail sind nicht gestattet. Diese Kommunikation per E-Mail ist nicht gegen den Zugriff durch Dritte geschützt. Die GISA GmbH haftet ausdrücklich nicht für den Inhalt und die Vollständigkeit von E-Mails und den gegebenenfalls daraus entstehenden Schaden. Sollte trotz der bestehenden Viren-Schutzprogramme durch diese E-Mail ein Virus in Ihr System gelangen, so haftet die GISA GmbH - soweit gesetzlich zulässig - nicht für die hieraus entstehenden Schäden.

3 11

Fwd: Fwd: Invitation to an IntelMQ in depth training on February 23th, 14:30h CET
by Sebix 09 Feb '22

09 Feb '22

-------- Forwarded Message -------- Subject: [IntelMQ-dev] Fwd: Invitation to an IntelMQ in depth training on February 23th, 14:30h CET Date: Thu, 3 Feb 2022 16:12:26 +0100 From: Otmar Lendl via IntelMQ-dev <intelmq-dev(a)lists.cert.at> Reply-To: Otmar Lendl <lendl(a)cert.at> To: ihap(a)lists.trusted-introducer.org <ihap(a)lists.trusted-introducer.org>, intelmq-dev(a)lists.cert.at

1 0

Taxii collector bot
by Joaquin Cabrera 18 Jan '22

18 Jan '22

Dear community, I'm looking for a collector bot to retrieve data from a taxii server, but i couldn't find any. We are tying to use intelMQ as our main tool to collect all security feeds and one of them is a taxii feed. Does anyone have this kind of scenario? Best regards, Joaquín Cabrera CERTuy - AGESIC Torre Ejecutiva Anexo Liniers 1280 piso 1 Tel: (+598) 2901 2929 Int. 8509 (11.000) Montevideo – URUGUAY www.agesic.gub.uy

4 5

[IntelMQ] Question about harmonization.conf
by Jonathan SCOUPREMAN 17 Dec '21

17 Dec '21

Greetings IntelMQ community, I'm looking for information about the harmonization.conf file located in /etc. In this file I see two sections: «event», and «report». Could you please tell me the difference? Unfortunately, I was not able to find hints in the current documentation. Also, as I see «report», does that mean IntelMQ has a reporting capability ? As far as I know IntelMQ doesn't provide a reporting capability, hence my doubt. ?? Thanks, Jonathan -- Jonathan SCOUPREMAN | jscoupreman(a)excellium-services.lu<mailto:jscoupreman@excellium-services.lu> | PGP Key ID: 0xAD971C07<http://pgp.circl.lu/pks/lookup?op=vindex&fingerprint=on&search=0x6802c48ead…> CERT-XLM | cert(a)excellium-services.com<mailto:cert@excellium-services.com> | PGP Key ID: 0xD74E5AC0<http://pgp.circl.lu/pks/lookup?op=vindex&fingerprint=on&search=0x67B311E5D7…> CERT-XLM Incident Handler @ excellium-services.com<https://excellium-services.com/> Excellium Services S.A. | 5 rue Goell L-5326 Contern Mobile: +352 691 982 790 Emergency: +352 262 039 64 708 | emergency(a)excellium-services.com<mailto:emergency@excellium-services.com> | PGP Key ID: 0x42662EFE<https://excellium-services.com/assets/EMERGENCY_PKEY.asc> This email is confidential and may contain legally privileged information. If you are not the intended recipient, you should not copy, distribute, disclose or use the information it contains, please e-mail the sender immediately and delete this message from your system. Note: e-mails are susceptible to corruption, interception and unauthorised amendment; we do not accept liability for any such changes, or for their consequences. You should be aware that we may monitor your e-mails and their content. Excellium Services SA.

3 3

Invitation to an IntelMQ Demo/Workshop on Dec. 9th, 13h CET
by Sebastian Waldbauer 25 Nov '21

25 Nov '21

Hey IntelMQ-Users :wave: I'd like to invite you to CERT.at's IntelMQ presentation. The presentation will be about 3 topics: - A short summary of what IntelMQ is & how it works under the hood - See it in action how CERT.at uses it - IntelMQ Toolset & Ecosystem Date/Time: 2021-12-09 13:00 - 14:30 UTC+1 (CET) Location: Zoom: https://us06web.zoom.us/j/84397883968?pwd=dm4xSFdlTUZUdmV2dVlFeGk3ZlRZdz09 Meeting-ID: 843 9788 3968 Code: 031551 Presenter: Sebastian Waldbauer No registration is required :) Kind regards, Sebastian -- // Sebastian Waldbauer <waldbauer(a)cert.at> - T: +43 1 5056416 7202 // CERT.at GmbH - https://www.cert.at/ // Firmenbuchnummer 561772k, LG Wien

1 0

IntelMQ 3.0.2 release fixing performance issues
by Sebastian Wagner 10 Sep '21

10 Sep '21

Dear community, Last week I said in the announcement for 3.0.1, that we didn't plan any further bugfix release before 3.1.0. I'm now proving myself wrong. Narzhan reported general performance issues and we investigated the problem on our own instances. We found two problems which different effects. It could be the case that you didn't even notice any performance drop, as both bugs affected only certain bots or situations. - When any collector creates a new report, it initializes an object of type Report. Since five years there was a bug, that this always loaded the harmonization.conf, instead of loading it just once at the start (which is - correctly - the case for new events). This was never a problem, as the Python's built-in json module was fast enough, but ruamel.yaml is slower. If a collector didn't massively create reports, that bug was not obvious, especially under testing conditions. And the bug was only relevant, if stream data is processed (for example with the Shodan stream collector). - Two bots, the STOMP collector and the API collector, don't have a proper process method, because they are using threading. A thread retrieves data asynchronously and sends the data. Normally, the process-method is running in an endless-loop, with sleeping rate_limit seconds between. If rate_limit is 0 or not defined, this would result in high load, therefore these bots defined the __collector_empty_process attribute which should have prohibited the endless loop. But due to class-restructuring, the attribute which would have been to be set was _Bot__collector_empty_process, and the flag was not effective. That did not only cause a loop, but as bot statistics are written to redis after every process-call, that caused continuous connections to redis. We've now renamed the flag to _collector_empty_process and fixed the two bots. The flag __is_multithreadable was affected analogous, but that should not have cause performance issues in particular. Additionally, the bot statistics can now be disabled by setting |statistics_host| to null (previously this caused an exception).| | The new version is on GitHub and PyPI, very soon in the deb/rpm repositories. The Docker image will follow on Monday or earlier. You can read the full changelog here: https://github.com/certtools/intelmq/releases/tag/3.0.2 have a nice weekend Sebastian -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 676 898 298 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

1 0

IEP04: The choice of the UUID-format
by Sebastian Wagner 07 Sep '21

07 Sep '21

Dear allies, The discussion around the IEP04 proposal, adding meta-information to IntelMQ messages, has stalled over the last months - first because of the time-intensive IntelMQ 3.0 release preparations and then because of the vacation season. Here is the current proposal: https://github.com/certtools/ieps/tree/main/004#readme Aaron, Sebastian Waldbauer and myself worked on it over the summer and also identified two open issues to be discussed: 1. The exact format of the meta-information and how to name and structure the fields. AIL made the first move and now uses a format similar to the previously proposed Variant "A". The IEP04 document contains the current proposal which is in line with the AIL format: https://github.com/certtools/ieps/tree/main/004#user-content-variant-ail If there are no other proposals, this will most probably the way to go. 2. The format of the UUID format which we want to uniquely identify IntelMQ events. We don't necessarily need to use the UUIDv4 format which represents pure randomness, but also other options which include the time and are even /time-sortable/. Sebastian Waldbauer analysed a couple of options and summarised his results in this document: https://github.com/certtools/ieps/blob/main/004/UUID.md Please let us know your opinion on the different UUID options. cheers Sebastian -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 676 898 298 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

2 1

IntelMQ & API & Manager release 3.0.1 and upgrade note
by Sebastian Wagner 02 Sep '21

02 Sep '21

Dear community, Over the past two months, IntelMQ contributors had no summer pause, but did the final finish for IntelMQ 3.0. A special thanks goes to Mikk Margus Möll (CERT.ee) who has put tremendous efforts in the IntelMQ Manager tackling structural and usability issues, mainly in the JavaScript-components! The deb/rpm repositories did not receive the 3.0.0 release at beginning of July to get more experience with the major changes before doing automatic upgrades, but now they deliver the brand-new 3.0.1 version. Please note, that the automatic upgrade procedures may still not be fully smooth. Just now, we have noticed, that the packages contain a small flaw, which harms the upgrade experience: The packages ship a default configuration (the file is now called `runtime.yaml`), but only if the file does not exist before - for new installations. But now in this special case, we renamed the configuration from `runtime.conf` to `runtime.yaml` and therefore, the new - default shipped - configuration takes precedence. I hope the following commands and hints will be of help to you. # remove the runtime configuration shipped by the package (can be called /etc/intelmq/runtime.*) and rename your original one to /etc/intelmq/runtime.yaml # the previously used runtime.conf can be used as drop-in to runtime.yaml (YAML is backwards-compatible with JSON) sudo -u intelmq intelmqctl upgrade-config -f -u v300_pipeline_file_removal sudo -u intelmq intelmqctl upgrade-config -f -u v300_defaults_file_removal sudo -u intelmq intelmqctl upgrade-config -f -u v301_deprecations The last three steps are important to merge the defaults and pipeline configuration into the new combined configuration file Please do not hesitate to ask. The deb-packages are also already available for the newly released Debian 11 Bullseye. We are not planning a bugfix release until the 3.1.0 release, so that one will be the next version to be released. Here's a short summary of what happened during the summer: - various fixes related to the IEP001 implementation (IEP001 was the change configuration format and merge of files, rewrite oft the internal parameter-handling) - removal of the malwaredomains feed and parser, because it does not exist anymore - Various fixes in the Shadowserver Parser and support for new reports: Vulnerable SMTP Server, Microsoft Sinkhole Events Report & Microsoft Sinkhole HTTP Events Report, Honeypot HTTP Scan - SMTP Output: Added Content-Disposition-header to the attachment, fixing the display in MS Outlook clients (as reported and dicussed on the Mailinglist). - Heavy refactoring of IntelMQ-Manager's JavaScript parts to fix errors and usability issues. If you are interested in developing on IntelMQ and you don't know where to start, have a look at the dev guide an the issues labeled "good first issue": https://github.com/certtools/intelmq/issues?q=is%3Aissue+is%3Aopen+label%3A… We are especially welcoming contributions to the documentation! You can read the full changelogs here: - https://github.com/certtools/intelmq/releases/tag/3.0.1 - https://github.com/certtools/intelmq-api/releases/tag/3.0.1 - https://github.com/certtools/intelmq-manager/releases/tag/3.0.1 https://cert.at/en/blog/2021/9/intelmq-301-release https://twitter.com/CERT_at/status/1433475188381806594 btw: There's new contact management portal called "tuency" for administrating abuse contacts available, which can be used in conjunction with IntelMQ. Read more about its features here: https://cert.at/en/blog/2021/9/tuency-constituency-portal-for-iocs-and-certs https://gitlab.com/intevation/tuency/tuency -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 676 898 298 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

IntelMQ-users