[Intelmq-dev] Discussion on intelmq output / transformation architecture

[Otmar Lendl]

On 19.04.2016 23:48, L. Aaron Kaplan wrote:
> 
> I would like to propose that we enhance our architecture to include "transformer bots".

I'd call them "output transformations".

> What do you think?
> Would this work? Do you see any serious problem with this approach?

This approach is good, but I see one point that should be taken into
account:

The parser bot usually creates multiple events from one input event.
(e.g. the collector retrieves a larger csv file in a single event, the
parser creates one event per line of the csv file).

Out the output side we *can* have a similar process, just in reverse:
Multiple events can end up in one email that is sent to e.g. ISPs.

Thus: on the output side there is not just the question of the
transformation to cybox/csv/xml/xarf/..., but also the question of
aggregation: Which set of events should be grouped together?

Yes, there will be cases where a simple event by event translation is
useful, but my gut-feeling is that this is the exception.

I don't have a full-blown proposal ready in my mind, so this just food
for though.

otmar
-- 
// Otmar Lendl <lendl at cert.at> - T: +43 1 5056416 711
// CERT Austria - http://www.cert.at/
// Eine Initiative der nic.at GmbH - http://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20160420/70cabc82/attachment.sig>