[IntelMQ-dev] IntelMQ 3.2 announcement

[L. Aaron Kaplan]

IntelMQ 3.2 release announcement
==================================

[TLP:WHITE, please pass this on, if appropriate]



Dear everyone,


We are excited to announce that today we can release 3.2.0 of the open source
IntelMQ (https://github.com/certtools/intelmq) incident handling automation
framework.


Special thanks go to Kamil (CERT.at), Sebix, Intevation, Filip Pokorny (Gethvi)
and Jason from Shadowserver (you all know who you are).  Also I would like to
thank the 93 contributors [1] who have been helping in many ways over the last
year(s) with IntelMQ.  Core members of IntelMQ are committed to continuing to
develop, support and enhance IntelMQ in the long run.  Especially since IntelMQ
is a quite well adopted extract transform load (ETL) tool for fetching incident
report feeds, processing them and sending them to the right place. It is widely
used in Europe and in the whole world, in CERTs, SOCs and MSSPs.  And best of
it, it is open source and freely available.


Again, we can not thank all the contributors enough.



So, what's in release 3.2?
===========================



Highlights:

* The main change is that IntelMQ's code can now be used as a 
  python module / library. That means: you can easily embed the hard 
  work that IntelMQ parser writers did into your application. 
  It's as simple as instantiating an IntelMQ bot in python and 
  giving it data. Thanks go to Sebix.
  See also  https://github.com/certtools/ieps/tree/main/007

* the IntelMQ API moved from hug to fastapi [4]. The API 
  interface stayed the same. Thanks a lot to Kamil.

* We have an updated intelmq developer version docker image.


Apart from that, a few other highlights:


* Shadowserver parser improvements. 
* New shadowserver data feeds are supported
* Changes time_format parameter to use new TimeFormat class
* new CIF3 output bot
* intelmq.bots.outputs.smtp_batch.output: Added a bot to gathering 
  the events and sending them by e-mails at a stroke as CSV files 
  (PR#2253 by Edvard Rejthar)



As always, you can find all the nitty gritty details in the CHANGELOG [2] file
and in the NEWS file [3] 



What's next?
==============

* BSI funded a complete re-write of webinput-csv
  (https://github.com/Intevation/intelmq-webinput-csv), watch for an
  announcement soon.

* Improved and re-done documentation via mkdocs (Thanks to Filip)

* The upcoming next version 3.3 will have an update for the dynamic config loading
  of the shadowserver parser [4]. This will always give you the most current
  shadowserver feeds and parsers without having to wait for an official intelmq
  release.  If you depend a lot on shadowserver feeds, it's ok to wait for the
  next release, which will be following very shortly.  (And a big hooray to
  shadowserver for their help, we know that you need this feature quickly)
  We plan to release 3.3 in approx 1 week.


Should I update?
==================

Yes!  



**Note well**: we still have some issues with proper Debian packages for
fastapi (a dependency of the new IntelMQ API server). 

This means re-packaging fastapi with the help of Debian developers or using the
old hug API for now (no difference from a user perspective though).

The deb-package for the new fastapi-based intelmq-api is currently only
available on Debian 11.

We are also sorting out an issue with the package build of intelmq on Debian
12: https://github.com/certtools/intelmq/issues/2384#issuecomment-1637799252
and issues on Ubuntu related to fastapi-Versions.

Packages of the previous compatible hug-based intelmq-api remain the package
repositories of affected distributions.  Once these issues are sorted out, the
new packages will become available for installation and upgrade in the
respective repositories.  For installations via pip or development setups,
nothing needs to be kept in mind.


How to update?
===============

* Upgrading: https://intelmq.readthedocs.io/en/3.2.0/user/upgrade.html
* Fresh installations: https://intelmq.readthedocs.io/en/3.2.0/user/installation.html

(Note that the docker installation might lag behind a bit)


Need help?
===========
You can contact us on the mailing lists:
 * intelmq-dev for developers [5]
 * intelmq-users for users [6]

See also: https://intelmq.readthedocs.io/en/3.2.0/user/support.html




Thank you and thanks to all the contributors!
Aaron Kaplan, Sebix, Kami, Filip
(for the whole IntelMQ team)



[1] https://github.com/certtools/intelmq/graphs/contributors
[2] https://github.com/certtools/intelmq/blob/3.2.0/CHANGELOG.md
[3] https://github.com/certtools/intelmq/blob/3.2.0/NEWS.md
[4] https://github.com/certtools/intelmq/pull/2372
[5] https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
[6] https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users