[IntelMQ-dev] IntelMQ 2.2.3 Christmas release

[Sebastian Wagner]

Merry Christmas, dear community :)

More or less last minute I decided to do a bugfix release before the
holidays *really* start, because we already collected some fixed in the
last weeks/months. There a no spectacular changes in this minor release,
but the upcoming 2.3.0 will have some major changes for the IntelMQ
Manager backend / the new IntelMQ API and the deprecation of Python 3.5.

Installation documentation:
https://github.com/certtools/intelmq/blob/2.2.3/docs/INSTALL.md
Upgrade documentation:
https://github.com/certtools/intelmq/blob/2.2.3/docs/UPGRADING.md

The deb/rpm packages will be available in the repositories in the next
few hours.

The NEWS:

### Harmonization
A bug in the taxonomy expert did set the Taxonomy for the type
`scanning` to `information gathering`
whereas for the type `sniffing` and `social-engineering`, the taxonomy
was correctly set to `information-gathering`.
This inconsistency for the taxonomy `information-gathering` is now
fixed, but the data eventually needs to fixed in data output (databases)
as well.

There are still some inconsistencies in the naming of the classification
taxonomies and types,
more fixes will come in version 3.0.0. See [issue
#1409](https://github.com/certtools/intelmq/issues/1409).

### Postgres databases
The following statements optionally update existing data.
Please check if you did use these feed names and eventually adapt them
for your setup!
```SQL
UPDATE events
   SET "classification.taxonomy" = 'information-gathering'
   WHERE "classification.taxonomy" = 'information gathering';
```

The full CHANGELOG:

### Documentation
- Bots/Sieve expert: Add information about parenthesis in if-expressions
(#1681, PR#1687 by Birger Schacht).

### Harmonization
- See NEWS.md for information on a fixed bug in the taxonomy expert.

### Bots
#### Collectors
- `intelmq.bots.rt.collector_rt`: Log the size of the downloaded file in
bytes on debug logging level.

#### Parsers
- `intelmq.bots.parsers.cymru.parser_cap_program`:
  - Add support for protocols 47 (GRE) and 59 (IPv6-NoNxt).
  - Add support for field `additional_asns` in optional information column.
- `intelmq.bots.parsers.microsoft.parser_ctip`:
  - Fix mapping of `DestinationIpInfo.DestinationIpConnectionType` field
(contained a typo).
  - Explicitly ignore field `DestinationIpInfo.DestinationIpv4Int` as
the data is already in another field.
- `intelmq.bots.parsers.generic.parser_csv`:
  - Ignore line having spaces or tabs only or comment having leading
tabs or spaces (PR#1669 by Brajneesh).
  - Data fields containing `-` are now ignored and do not raise an
exception anymore (#1651, PR#74 by Sebastian Waldbauer).

#### Experts
- `intelmq.bots.experts.taxonomy.expert`: Map type `scanner` to
`information-gathering` instead of `information gathering`. See NEWS
file for more information.

### Tests
- Travis: Deactivate tests with optional requirements on Python 3.5, as
the build fails because of abusix/querycontacts version conflicts on
dnspython.

### Known issues
- Bots started with IntelMQ-Manager stop when the webserver is
restarted. (#952).
- Corrupt dump files when interrupted during writing (#870).

-- 
// Sebastian Wagner <wagner at cert.at> - T: +43 1 5056416 7201
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - https://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20201223/e1288f0e/attachment.sig>