[Intelmq-dev] Proposal (Request For Comments) - IntelMQ with Run Modes & Process Management

[Sebastian Wagner]

Hi,

On 04/21/2017 10:48 AM, Bernhard Reiter wrote:
> thanks to you, Sebastian and Aaron for working out a proposal
I actually have a small other proposal here for comparison:
https://github.com/wagner-certat/intelmq/blob/proposal/docs/proposal.md
It's also an outcome of the same discussion, with some differences and
simplifications. But it tackles less issues.

I try to give some answers where possible. Tomas, please correct me if I
am wrong.
> There are others that should be considered as well: An alternative would be: 
> Each bot gets helping functions that allows it to be woken up on some events 
> and event types include a sleep until an alarm goes off. The time of being 
> alarmed can be set by the bot itself. This way only one run-mode is necessary 
> as a bot can see if it still wants to configure something (like "on_boot") , 
> run or be woken up later at a certain time.
The idea is to keep intelmq simple (which is one of the design goals
from the beginnings of this projects) and use existing and well-known
tools instead of implementing our own bunch of bugs.
> However again I'm unsure if systemd is a good solution.
Personally, I want to keep the PID-based approach and encourage
developers to provide support for supervisord etc. Making systemd a
hard-requirement is not *my* intention.
However, my idea and intention is to not implement the process
management ourselves, but maybe we can't avoid it because of:
> My second concern regarding systemd is more important: systemd is a general 
> process manager, for intelmq we may need a process manager that implements 
> tactics specific to abuse handling and intelmq's design. For example about 
> the question: What shall be done if we get congestion in intelmq? Should 
> starting of scheduled bots runs be queued up or dropped? Which ones are more 
> important than others? My gut feeling is that we will need some sort of flow 
> control and that systemd will be unable to manage that without additional 
> stunts.
Flow control is definitely an issue and a big topic we should discuss in
depth. We (certat) do not need flow control currently but maybe you do?
> "in the background" and "interactive" mode does not go together.
> Overall it is okay to be able to start processes in the foreground to me, may 
> it be for diagnostic purposes or interaction. I guess the first should should 
> be rephrased to say "by default bots are started in the background"
> or "unless a different option is given".
I think that's the meaning, yes. Basically the current behavior of
`intelmqctl start/run`, just renamed.
> The "reload" action is unclear to me, why making things more complicated?
> From my perspective doing more than "restart"
Does "reload" more than "restart"? AFAIU, they are performing the same
checks. The only difference is, that restart stops/starts the running
continuous bots, and reload sends sighup to those.

Sebastian

-- 
// Sebastian Wagner <wagner at cert.at> - T: +43 1 5056416 7201
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - https://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20170421/08905333/attachment.sig>