[IntelMQ-dev] regarding IEP03

[L. Aaron Kaplan]

Hi everyone,

I wanted to send back a better write-up of my stance on IEP03 ("multiple values" in the IntelMQ internal data format).
Alas, I was quite busy and I am sprinting to push out some code for our deadline at work.

However, let me summarise it:

I think the IEP03 is very well written, thank you a lot for this! Thinking this through was important and I think Sebastian Waldbauer did a great job.

Reading it, I realised that my initial proposal of having multiple values is really breaking the KISS principle of IntelMQ in a bad way. Worse than I had thought. So, I am thinking of retracting the proposal.

However, .... https://github.com/certtools/ieps/tree/main/003#alternatives  has a good core in it.

If we have multiple values, instead of doing the n x m complexity explosion, we link different events (JSON rows) together via UUIDs this gives us what we need:

  * UUIDs help with deduplication! That's important when linking IntelMQ instances!
  * lower complexity / keep the KISS principle
  * consumers can ignore the UUID-linking if it's not relevant for them (f.ex enrichment processes/bots)
  * we can still represent linked events.

I would like to add one little but important thing for the UUID linking idea: add a "link-type".

Examples for link-types:
  * parent-child event
  * grouping types (all of these events belong to the same report)
etc.

With this triplet information  , we are close to RDF (left-side, type, right-side) and thus we can (future-proof) represent any type of relation.

A list of valid types needs to be documented in the IDF format page of course.


So, I think with that, we can go ahead.

Thanks,
a.

PS: and sorry that my feedback came a bit late, as said - code sprints.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20210422/f182e42a/attachment.sig>