Folks,
We have been thinking about things to improve IntelMQ in order to have it really suitable for production... and now we want to raise the discussion about the following ideas or other related ideas that you might think of:
- Run modes: stream (always running, e.g. shadowserver-parser bot) or scheduled (run once every day at 10:00AM, e.g. shadowserver-chargen-collector bot)
- Systemd: take care of bots which are running permanently. Use systemd to manage bots in stream run mode that cannot stop and also, do a better job with managing which bots processes which are really running, not using the current implementation with pidfiles.
- Crontab: manage bots that need to run once, e.g. at midnight every day. Use crontab to manage bots in scheduled run mode to just run once on a pre-defined scheduled time.
Why these ideas came up?
Mainly because of two things:
1. Run modes are needed due the different purposes of a bot which is collecting a real-time feed and a bot that is collecting shadowserver reports once in a day.
2. Systemd and Crontab are two well-know tools that we all know about it and IntelMQ seems to need them since we have seen some problems with managing bots processes (running or not) and report automatically failures by email, just to give some examples.
Until now, we don't have any of these features implemented, we only have developed some "rubbish" code in order to do some Proof-of-concept but nothing serious. So, please guys, share your thoughts about these ideas or others related in order to define, as a community, a strategy for the next month.
Thank you.
Cheers!