Hi Mika,
On 28.01.2021, at 09:40, Mika Silander mika.silander@csc.fi wrote:
Hi Aaron,
Thanks for a prompt reply. I feared I had missed something and it seems I did ... I've skimmed through the available bots several times but the wait bot had escaped the radar :-(.
Np worries :) That's what this list is for.
Speaking of documentation: as a new developer having good documentation is crucial. Do you accept contributions and if yes, how? Git pushes?
Sure! IntelMQ was always and will be a community project - and open source. This was the thing me and Tomas Lima back then in the days when the project started agreed upon (based on the lessons of trying out a previous system).
So, yes, please ! By all means send GitHub pull requests for the documentation. The project lives by and for the community. I know CERT.at is doing a lot of the heavy weight lifting right now, but essentially it is a community project. And that's super important for the long term survivability and adaption to user's and developer's needs.
I'm not thinking of any major undertakings but rather occasional contributions if I come across something I feel could be improved, needs clarifying etc etc.
Even if it is typo or grammar corrections or wording ;-) It's welcome! And thank you very much.
Cheers, Aaron.
Cheers, Mika
----- Original Message ----- From: "L. Aaron Kaplan" aaron@lo-res.org To: "Mika Silander" mika.silander@csc.fi Cc: "intelmq-dev" intelmq-dev@lists.cert.at Sent: Thursday, 28 January, 2021 09:33:41 Subject: Re: [IntelMQ-dev] Event rate limiting
Hi Mika,
in principle what you proposed (sleep) is safe - just be aware of potential overflowing (redis) queues. Basically if you are out of RAM -> redis will not like it. Same for disk space in case you configured redis to write on the disk.
Please take a look at https://github.com/certtools/intelmq/blob/develop/intelmq/bots/experts/wait/... It's already there :) just not that well documented I guess. But it's there.
Best, Aaron.
On 28.01.2021, at 07:32, Mika Silander mika.silander@csc.fi wrote:
Hi,
Maybe this is answered somewhere already but I wasn't able to find any hints and I've just recently started developing bots so here's one for the list:
I'd need to limit the rate at which events are processed to avoid swamping a back end system with requests. Big bursts of events should remain in the inqueue of such a rate limiting bot, and then be forwarded at a configurable rate e.g. X events/minute. Afaik, by default, bots attempt to process all the events in their inqueue without delay. How should one go about to create such a bot? Is it safe to put the bot process (=operating system process) to sleep for N seconds from within the bot's process method and then proceed, can we do busy-looping (not nice!), or, some other solution?
Br, Mika
IntelMQ-dev mailing list IntelMQ-dev@lists.cert.at https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev