Hi everyone,
I wanted to send back a better write-up of my stance on IEP03 ("multiple values" in the IntelMQ internal data format). Alas, I was quite busy and I am sprinting to push out some code for our deadline at work.
However, let me summarise it:
I think the IEP03 is very well written, thank you a lot for this! Thinking this through was important and I think Sebastian Waldbauer did a great job.
Reading it, I realised that my initial proposal of having multiple values is really breaking the KISS principle of IntelMQ in a bad way. Worse than I had thought. So, I am thinking of retracting the proposal.
However, .... https://github.com/certtools/ieps/tree/main/003#alternatives has a good core in it.
If we have multiple values, instead of doing the n x m complexity explosion, we link different events (JSON rows) together via UUIDs this gives us what we need:
* UUIDs help with deduplication! That's important when linking IntelMQ instances! * lower complexity / keep the KISS principle * consumers can ignore the UUID-linking if it's not relevant for them (f.ex enrichment processes/bots) * we can still represent linked events.
I would like to add one little but important thing for the UUID linking idea: add a "link-type".
Examples for link-types: * parent-child event * grouping types (all of these events belong to the same report) etc.
With this triplet information , we are close to RDF (left-side, type, right-side) and thus we can (future-proof) represent any type of relation.
A list of valid types needs to be documented in the IDF format page of course.
So, I think with that, we can go ahead.
Thanks, a.
PS: and sorry that my feedback came a bit late, as said - code sprints.