[Intelmq-users] Updating datastores

[L. Aaron Kaplan]

> On 14 Sep 2018, at 03:28, Chris Horsley <chris.horsley at csirtfoundry.com> wrote:
> 
> Expiring indicators needs some careful thought in my experience.
> 
> There are some threat intelligence platforms which have a well-integrated way to do this using a relevancy half-life time per feed or indicator. If the half-life you set for a feed is one month, it starts at 100% relevancy, after one month it's 50%, after two months it's 25% etc.
> 
> Over time, indicators get a less relevant score, but are not deleted by default.  Sometimes, you might want to do a search for all indicators over all time (e.g. you're coming up with the complete history for an ASN / registrar / URL pattern). Other times, you might want to only export IP addresses with a time relevancy score over 70% to your network appliance to keep the list small and useful.
> 
> The trick is that different types of indicators from different feeds probably need different expiry windows. There might also be different use cases for the same data where you want to filter based on timeliness / relevancy.

Totally agree that this would be a smart way to do it. Probably even a very generic way (again, that depends on your use case).
But I am not sure if its in scope of IntelMQ since that just fetches data, processes, filters & enriches it and sends it somewhere.
Anyway, I wanted to say that I do understand the wish for expiring (whatever your method might be) IoCs.
But I am not sure if it fits the model and architecture of intelMQ.

Interesting discussion.

Joanna, what are you trying to achieve with expiring IoCs?
(just to understand your use case better).


> 
> Chris
> 
>> On 12 Sep 2018, at 8:15 pm, L. Aaron Kaplan <kaplan at cert.at> wrote:
>> 
>> Signed PGP part
>> 
>>> On 12 Sep 2018, at 10:23, Sebastian Wagner <wagner at cert.at> wrote:
>>> 
>>> Hi,
>>> 
>>> How do IOCs expire?
>>> 
>> 
>> Well I can imagine a scenario where you fetch for example IP addresses via intelMQ
>> from a blacklist and you want to expire them at some point (to be defined by the blacklist and/or the user of intelmq).
>> 
>> So, I do see a use-case here.
>> 
>> 
>>> Sebastian
>>> 
>>> On 12/09/2018 03.22, joanna at scate.tech wrote:
>>>> Hi,
>>>> 
>>>> Is there a way of updating outputs such as databases when IOCs expire?
>>>> Don't want to spend time re-inventing the wheel.
>>>> 
>>>> Thanks.
>>> 
>>> --
>>> // Sebastian Wagner <wagner at cert.at> - T: +43 1 5056416 7201
>>> // CERT Austria - https://www.cert.at/
>>> // Eine Initiative der nic.at GmbH - https://www.nic.at/
>>> // Firmenbuchnummer 172568b, LG Salzburg
>>> 
>>> 
>>> --
>>> Listen-Einstellungen:
>>> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users
>> 
>> 
>> --
>> // L. Aaron Kaplan <kaplan at cert.at> - T: +43 1 5056416 78
>> // CERT Austria - https://www.cert.at/
>> // Eine Initiative der nic.at GmbH - http://www.nic.at/
>> // Firmenbuchnummer 172568b, LG Salzburg
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
> 
> --
> Listen-Einstellungen:
> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users


--
// L. Aaron Kaplan <kaplan at cert.at> - T: +43 1 5056416 78
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - http://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg






-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20180914/4ea4cc32/attachment.sig>