Hello IntelMQ users,
Trying to figure out how to use MISP feed output bot, could someone advise. MISP creates new event once per period (per hour, or per day), and that makes MISP doing correlation between these events created previously. And actually that results correlation table grows exponentially. Am I doing something wrong on IntelMQ side or MISP?
At IntelMQ I configure bot to make one event per day (actually containing ~1500 events in resulting json file). At the MISP side I have MISP feed format feed.
Best regards
--
Marius Urkis
Hello Marius,
Am Donnerstag 03 September 2020 11:21:45 schrieb Marius Urkis:
Trying to figure out how to use MISP feed output bot, could someone advise.
seems my MISP foo is not strong enough to advise without doing lots of tests myself. Just to be sure, you are talking about using
https://github.com/certtools/intelmq/blob/develop/intelmq/bots/outputs/misp/... as documented here https://github.com/certtools/intelmq/blob/develop/docs/Bots.md#misp-feed and runs into MISP as "feed", via https://www.circl.lu/doc/misp/managing-feeds/
MISP creates new event once per period (per hour, or per day), and that makes MISP doing correlation between these events created previously. And actually that results correlation table grows exponentially. Am I doing something wrong on IntelMQ side or MISP?
There are a number of options to MISP feeds, some are related to correlation and whether to keep old data in. Personally I'd play with these and ask in a MISP forum how they handle feeds in general. (We've developed the IntelMQ Output MISP API bot and there you can set the fields explicitely which you want to correlate and you have to chose a few significant ones.)
At IntelMQ I configure bot to make one event per day (actually containing ~1500 events in resulting json file). At the MISP side I have MISP feed format feed.
If those are different events, they should not correlate much (in my simple understanding), but again I don't know how MISP handles other incoming "feeds".
Best Regards, Bernhard
Dear all,
Marius is already in contact with Raphaël Vinot, the MISP Feed Output author and MISP developer. Here is a short summary by Raphaël, which I can share here on his behalf:
If you have a lot of similarities across events, you have the following options to avoid crazy amount of correlations:
* Create less events (once a week for example) * Disable correlation at event level * Keep the feed in memory only and not create events out of it in the database. => in that case, you will still be able to see hits against indicators in the events from the feed, but they're in redis only instead of in MySQL so it's not a problem.
best regards Sebastian
On 9/3/20 11:21 AM, Marius Urkis wrote:
Hello IntelMQ users,
Trying to figure out how to use MISP feed output bot, could someone advise. MISP creates new event once per period (per hour, or per day), and that makes MISP doing correlation between these events created previously. And actually that results correlation table grows exponentially. Am I doing something wrong on IntelMQ side or MISP?
At IntelMQ I configure bot to make one event per day (actually containing ~1500 events in resulting json file). At the MISP side I have MISP feed format feed.
Best regards
--
Marius Urkis