[IntelMQ-dev] IEP04: IntelMQ Data Format - Meta-Information

[Pavel Kácha]

> From: Sebastian Waldbauer <waldbauer at cert.at>, Date: bře 30, 2021
>
> nowadays security incidents are more important than 10 years ago. As IntelMQ
> can be used as core element for automated security incident handling, we
> would like to provide a way to share information with other intelmq
> instances. This proposal is also an alternative to IEP03 insofar as solving
> the "multiple values" is possible by using UUIDs so "link" related events in
> a backwards-compatible manner.

Hello,

   couple of notes (as Idea author).

   We decided to not go for linking as the main means to allow multiple
IPs/hostnames, as it works only for source:target in 1:1, 1:N, M:1 cases.
1:1 in current state of affairs in IntelMQ, 1:N is for example scan or
bruteforce coming from one machine to many, M:1 is for example DDoS to one
specific target.
   An then there is M:N - for example detectors, which (based on netflow
statistics) detect DDoS, but with no explicit connection information - so
you have information about traffic from M sources, going to N targets. In
world, where you have only 1:1 mapping events and linking, you end up with
cartesian product (which is not what you want :) ), or two linked events -
one with only sources and no targets and second with only targets and no
sources (which is arguably clumsy).

   Second use case - deduplicating in case of distribution circles - is easy
if everyone uses the same format or passes the IDs (whatever they are, just
reasonably unique, UUID is fine).
   However, problem arises with external sources (which is currently the
main source of information in IntelMQ). 
   Consider: organisation A gets event from Shadowserver into IntelMQ, which
recasts it as IntelMQ format and ads arbitrary ID. Organisation B does the
same. Organisation C, which gets them both, with two distinct IDs, is unable
to deterministically decide, whether event is duplicate, or just
coincidence.
   No clear idea of solution here, maybe stable set of "external source"
identificators (for Shadowserver, Shodan, ...) plus stable ID/hash generated
deterministically from important fields... (as you mentioned, some CyCat
application?)

-- Pavel Kácha, CESNET
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20210331/825dc5f4/attachment-0001.sig>