[IntelMQ-dev] First steps to make adding custom Bots easier

[Birger Schacht]

Dear *,

one goal of the IntelMQ 3.0 release (which is planned for this summer) 
is to make the handling of configuration files easier and also to lower 
the bar for adding new bots. With two bigger merge requests merged in 
the last few weeks, we are now one step closer to that goal:

The first change affects how bots handle their configuration. IntelMQ 
Bots used to work with a dict called `parameters` that contained all 
their settings and the settings of the parent bots and default values 
and everything was merged together at some point. When adding a bot 
those parameters had to be checked in the bot class itself, they had to 
be added to the `defaults.conf` and to the `BOTS` file. We now changed 
that approach and bots now simply define their parameters as (public) 
class attributes. When the bot is initialized, we iterate through the 
public attributes and check if they are set to a custom value in the 
configuration and if so, we use that value instead. This also works for 
all the attributes the bot inherits from parent classes. In addition, 
the default values for the variables are now set in the bot classes 
directly and not in a separate `defaults.conf` file. While adding all 
the attributes to the bots, we also tried to add typing information- we 
are not sure yet how exactly we will use that information, but at least 
its there and shows what kind of value is expected. Another additional 
information in the bots is the docstring, which we added to every bot class.

The second change made the `BOTS` file obsolete. The `BOTS` file 
contained a list of all the available bots, together with their 
description, their modulepath and their parameters. Sebastian Waldbauer 
tirelessly worked on refactoring the code and added a function to 
generate the list of bots on the fly. Based on that Sebastian Wagner 
polished the PR by cleaning up a big amount of stale code in the bot 
classes, testing the changes and adding documentation.

So it will now be possible to add new bots without having to deal with 
the `BOTS` file or the `defaults.conf`. The latter still exists, mainly 
because the `intelmqctl` script still uses it, but that will be an easy fix.

We also updated the documentation accordingly, as you can see in the 
development version of the Developer Guide [0].

[0] https://intelmq.readthedocs.io/en/latest/dev/guide.html

cheers,
Birger
-- 
// Birger Schacht <schacht at cert.at>
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - https://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0x3A3C547D2D48D997.asc
Type: application/pgp-keys
Size: 5392 bytes
Desc: not available
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20210316/3e7d62a4/attachment.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20210316/3e7d62a4/attachment.sig>