Am Freitag 21 Oktober 2016 18:31:10 schrieb L. Aaron Kaplan:
an aggregation of events within IntelMQ might be a reasonable thing to do.
I am not sure if an aggregation *within* intelmq makes sense. The classical way would be to do an aggregation from a datastore/DB after intelmq puts it there.
The aggregation for email notifications in abuse handling is special. We are seeing this while building the solution for CERT-Bund.
It is not the need of data collection for analysis, but just sending out one email. So the time-frame is short.
I *highly* recommend to take a serious look at other ETL and aggregation tools and processes and then come back to this discussion. Intelmq was not made for aggregation.
In a data flow sense, the deduplicator already "aggregates". Some abuse handling decisions will depend on seeing several sources report something in the future, for this they will need to wait a bit, maybe just a few minutes like the deduplicator.
The main question is: How many typical intelmq setups will want to have functionality that sends out an email? If many are, than email should be part of the core intelmq experience. And email means aggregated at least for a few minutes, otherwise it is too much overhead.
Best Regards, Bernhard