[IntelMQ-dev] IntelMQ & API & Manager release 3.0.1 and upgrade note

[Sebastian Wagner]

Dear community,

Over the past two months, IntelMQ contributors had no summer pause, but
did the final finish for IntelMQ 3.0.

A special thanks goes to Mikk Margus Möll (CERT.ee) who has put
tremendous efforts in the IntelMQ Manager tackling structural and
usability issues, mainly in the JavaScript-components!

The deb/rpm repositories did not receive the 3.0.0 release at beginning
of July to get more experience with the major changes before doing
automatic upgrades, but now they deliver the brand-new 3.0.1 version.
Please note, that the automatic upgrade procedures may still not be
fully smooth. Just now, we have noticed, that the packages contain a
small flaw, which harms the upgrade experience: The packages ship a
default configuration (the file is now called `runtime.yaml`), but only
if the file does not exist before - for new installations. But now in
this special case, we renamed the configuration from `runtime.conf` to
`runtime.yaml` and therefore, the new - default shipped - configuration
takes precedence. I hope the following commands and hints will be of
help to you.

# remove the runtime configuration shipped by the package (can be called
/etc/intelmq/runtime.*) and rename your original one to
/etc/intelmq/runtime.yaml
# the previously used runtime.conf can be used as drop-in to
runtime.yaml (YAML is backwards-compatible with JSON)
sudo -u intelmq intelmqctl upgrade-config -f -u v300_pipeline_file_removal
sudo -u intelmq intelmqctl upgrade-config -f -u v300_defaults_file_removal
sudo -u intelmq intelmqctl upgrade-config -f -u v301_deprecations
The last three steps are important to merge the defaults and pipeline
configuration into the new combined configuration file

Please do not hesitate to ask.

The deb-packages are also already available for the newly released
Debian 11 Bullseye.

We are not planning a bugfix release until the 3.1.0 release, so that
one will be the next version to be released.

Here's a short summary of what happened during the summer:
- various fixes related to the IEP001 implementation (IEP001 was the
change configuration format and merge of files, rewrite oft the internal
parameter-handling)
- removal of the malwaredomains feed and parser, because it does not
exist anymore
- Various fixes in the Shadowserver Parser and support for new reports:
Vulnerable SMTP Server, Microsoft Sinkhole Events Report & Microsoft
Sinkhole HTTP Events Report, Honeypot HTTP Scan
- SMTP Output: Added Content-Disposition-header to the attachment,
fixing the display in MS Outlook clients (as reported and dicussed on
the Mailinglist).
- Heavy refactoring of IntelMQ-Manager's JavaScript parts to fix errors
and usability issues.

If you are interested in developing on IntelMQ and you don't know where
to start, have a look at the dev guide an the issues labeled "good first
issue":
https://github.com/certtools/intelmq/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22
We are especially welcoming contributions to the documentation!

You can read the full changelogs here:
- https://github.com/certtools/intelmq/releases/tag/3.0.1
- https://github.com/certtools/intelmq-api/releases/tag/3.0.1
- https://github.com/certtools/intelmq-manager/releases/tag/3.0.1

https://cert.at/en/blog/2021/9/intelmq-301-release
https://twitter.com/CERT_at/status/1433475188381806594

btw:
There's new contact management portal called "tuency" for administrating
abuse contacts available, which can be used in conjunction with IntelMQ.
Read more about its features here:
https://cert.at/en/blog/2021/9/tuency-constituency-portal-for-iocs-and-certs
https://gitlab.com/intevation/tuency/tuency

-- 
// Sebastian Wagner <wagner at cert.at> - T: +43 676 898 298 7201
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - https://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20210902/da9fc3cb/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20210902/da9fc3cb/attachment.sig>