[IntelMQ-dev] IntelMQ & API & Manager Release 3.0

Thu Aug 5 17:17:04 CEST 2021

Hi,

We recently published a blog post about the release, summarizing a few
of the major changes:
https://cert.at/en/blog/2021/8/intelmq-30-domain-based-workflow-ieps

Sebastian

On 7/5/21 5:59 PM, Sebastian Wagner wrote:
>
> Dear community,
>
> The time has come and IntelMQ 3.0 is final! We, as community, made
> loads of changes, smaller and bigger ones, and I really think that
> IntelMQ became more user-friendly, developer-friendly and feature-rich
> at the same time!
>
> There are some major changes in this release, especially the in the
> field of the configuration and Internal Data format (previously:
> "harmonization"). For the configuration-part, the upgrade part should
> be automatic with `intelmqctl upgrade-config` as usual. For the Data
> format, carefully look at your bot configurations (filters, sieve,
> etc.) to update them. Adaptions in systems connected to IntelMQ,
> especially also databases might be necessary as well. The NEWS.md file
> give a summary of what has changed:
> https://github.com/certtools/intelmq/blob/maintenance/NEWS.md#user-content-300-major-release-2021-07-02
>
> We don't recommend to upgrade existing production instance of IntelMQ
> yet. We of course did testing, including the end-to-end tests, and
> have detailed release notes. But for critical systems, a delayed
> upgrade makes sense ;)
> Therefore the stable deb/rpm repositories don't contain the 3.0
> release yet! Even though an upgrade of production systems is not yet
> recommended, extensive usage and testing of the new releases are very
> much welcome and required to get the necessary feedback for the next
> (maintenance) releases.
>
> The releases are available via git, PyPI, Docker and the *unstable*
> deb/rpm repositories.
>
> Installation documentation:
> https://intelmq.readthedocs.io/en/maintenance/user/installation.html
> Upgrade documentation:
> https://intelmq.readthedocs.io/en/maintenance/user/upgrade.html
> IntelMQ API documentation:
> https://intelmq.readthedocs.io/en/maintenance/user/intelmq-api.html
> IntelMQ Manager documentation:
> https://intelmq.readthedocs.io/en/maintenance/user/intelmq-manager.html
>
> NEWS/release notes of IntelMQ (Core):
> https://github.com/certtools/intelmq/blob/maintenance/NEWS.md#user-content-300-major-release-2021-07-02
> Full Changelog of IntelMQ (Core):
> https://github.com/certtools/intelmq/blob/maintenance/CHANGELOG.md#300-2021-07-02
>
> On a high level, these are the major changes compared to version 2.3.x
> (2.3.3 was released 2021-05-31):
>
> In the core and Docker:
>
>   * Configuration rewrite including parameter loading and handling
>     (IEP01), plus the required adoption of the API and Manager, by
>     Birger Schacht (CERT.at).
>   * Classification sync with RSIT, by Sebastian Wagner (CERT.at).
>   * Removal of the BOTS file, by Sebastian Waldbauer (CERT.at).
>   * Creation and maintenance of the Docker images by Sebastian
>     Waldbauer (CERT.at).
>   * Creation of Docker-instructions for development setups by Einar
>     Lanfranco and Jeremias Pretto (CERT-UNLP cert.unlp.edu.ar).
>
> New and majorly enhanced bots:
>
>   * Added |intelmq.bots.collectors.fireeye|: A bot that collects
>     indicators from Fireeye MAS appliances (PR#1745 by Christopher
>     Schappelwein).
>   * |intelmq.bots.collectors.api.collector_api|: Added UNIX socket
>     capability (PR#1987 by Mikk Margus Möll, fixes #1986).
>   * Added |intelmq.bots.parsers.fireeye|: A bot that parses hashes and
>     URLs from Fireeye MAS indicators (PR#1745 by Christopher
>     Schappelwein).
>   * Added |intelmq.bots.experts.http.expert_status|: A bot that
>     fetches the HTTP Status for a given URI and adds it to the message
>     (PR#1789 by Birger Schacht, fixes #1047 partly).
>   * Added |intelmq.bots.experts.http.expert_content|: A bot that
>     fetches an HTTP resource and checks if it contains a specific
>     string (PR#1811 by Birger Schacht).
>   * Added |intelmq.bots.experts.lookyloo.expert|: A bot that sends
>     requests to a lookyloo instance & adds |screenshot_url| to the
>     event (PR#1844 by Sebastian Waldbauer, fixes #1048).
>   * Added |intelmq.bots.experts.rdap.expert|: A bot that checks the
>     RDAP protocol for an abuse contact for a given domain (PR#1881 by
>     Sebastian Waldbauer and Sebastian Wagner).
>   * |intelmq.bots.experts.sieve.expert|: Major refactoring and lot's
>     of new functionality New operators for working with various types
>     (lists, sets, booleans, float, int), generic rule negation and
>     nesting (PR#1895 by Mikk Margus Möll).
>   * Added |intelmq.bots.experts.uwhoisd|: A bot that fetches the whois
>     entry from a uwhois-instance (PR#1918 by Raphaël Vinot).
>   * Added |intelmq.bots.experts.aggregate|: A bot that aggregate
>     events based upon given fields & a timespan. (PR#1959 by Sebastian
>     Waldbauer)
>   * Added |intelmq.bots.experts.tuency|: A bot that queries the
>     IntelMQ API of a tuency instance (PR#1857 by Sebastian Wagner,
>     fixes #1856).
>   * Added |intelmq.bots.outputs.templated_smtp| (PR#1901 by Karl-Johan
>     Karlsson).
>
> On the documentation front, these are the most important changes
>
>   * License and copyright information was added to all the bots (by
>     Birger Schacht).
>   * Added documentation on the EventDB (PR#1955 by Birger Schacht,
>     PR#1985 by Sebastian Wagner).
>   * Added TimescaleDB for time-series documentation (PR#1990 by
>     Sebastian Waldbauer).
>   * n6 interoperability documentation: Adding more graphs and
>     illustrations (PR#1991 by Sebastian Wagner).
>   * Added documentation on abuse-contact look-ups (PR#2021 by
>     Sebastian Waldbauer and Sebastian Wagner).
>
> And not to forget all the smaller changes and additions.
>
> Thanks to (in random order)
>
> Raphaël Vinot (circl.lu)
> Bernhard Reiter (intevation.de)
> Sebastian Wagner (CERT.AT)
> Filip Pokorný (CSIRT.CZ)
> Guillaume GRANJON de LÉPINEY (CERT XLM excellium-services.com)
> Mikk Margus Möll (CERT.ee)
> Alex Kaplan
> Thomas Hungenberg (CERT-BUND.DE)
> Einar Lanfranco (CERT-UNLP cert.unlp.edu.ar)
> Christopher Schappelwein (milCERT, BMLV.gv.at)
> Marcos Gonzalez (CSIRT-RD cncs.gob.do/csirt-rd/)
> Marius Karotkis (NRDCS.LT)
> Sebastian Waldbauer (CERT.AT)
> Jeremias Pretto (CERT-UNLP cert.unlp.edu.ar)
> Karl-Johan Karlsson (Linköping University LIU.SE)
> Birger Schacht (CERT.AT)
>
> ... and all the contributors of previous releases and as well to all
> reporters, supporters, etc!
>
> best regards
> Sebastian
>
> -- 
> // Sebastian Wagner <wagner at cert.at> - T: +43 676 898 298 7201
> // CERT Austria - https://www.cert.at/
> // Eine Initiative der nic.at GmbH - https://www.nic.at/
> // Firmenbuchnummer 172568b, LG Salzburg
>
-- 
// Sebastian Wagner <wagner at cert.at> - T: +43 676 898 298 7201
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - https://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20210805/a3e11381/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20210805/a3e11381/attachment.sig>