[IntelMQ-dev] classification attributes in IntelMQ Shadowserver parser schema

[L. Aaron Kaplan]

I agree with everything being said, personally. But I would like to hear Thomas' opinion as well since they shovel *a lot* of data through intelmq.



> On 08.02.2024, at 11:21, Kamil Mankowski <mankowski at cert.at> wrote:
> 
> From my perspective, it shouldn't delay the release.
> 
> We should clearly note the change of ShadowServer parser, especially that it can include important data changes and link to the parser's changelog - and let users decide, whether they feel ready to upgrade their instances or not. We can note that there is ongoing discussion about improving identifiers and point to where people can participate.
> 
> In my opinion, the current delay has already more painful consequences for users than a potential profit of delaying it further. In currently published release, the STOMP collector doesn't work with n6, new ShadowServer feeds are not supported, schema of the rest of them is outdated. New release brings also a possibility for easier extending with custom bots, without dealing with modifying IntelMQ manually.
> 
> Best regards
> 
> // Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
> // CERT Austria - https://www.cert.at/
> // CERT.at GmbH, FB-Nr. 561772k, HG Wien
> 
> On 2/8/24 11:11, L. Aaron Kaplan wrote:
>> Hi all,
>> thanks for the lively discussion first of all, because it shows that these things matter :)
>> However, my question currently is:  green light for release or not?
>> What do you feel Thomas? Do you feel we need more time to discover things which would create a lot of havoc at different installations ? Or do you feel that these changes are so minimal that it's just part of the regular attentive update cycle?
>> We can put that into the CHANGELOG and NEWS file of course. Would that be enough?
>> Because I have been delaying the release a bit for the discussion to settle and in order to make sure that there are as few surprises as possible for everyone with the dynamic schema update.
>> Best,
>> Aaron.
>>> On 08.02.2024, at 10:59, Thomas Hungenberg via IntelMQ-dev <intelmq-dev at lists.cert.at> wrote:
>>> 
>>> On 06.02.24 13:42, Kamil Mankowski wrote:
>>>> When it comes to identifiers changes, I would be very conservative.
>>>> They can be used for filtering, and as so - changing them is potentially dangerous. > I second fixes about IPv6, those were more misleading than helping, but for the rest -
>>>> we need to be careful and announce the change.
>>> 
>>> Yes, our IntelMQ setup with mailgen etc. also heavily depends on the known
>>> classification identifiers. That is why I asked not to change them with the
>>> switch to the dynamic schema.
>>> 
>>> However, Shadowserver renamed some "old" feeds from "open-*" to "accessible-*"
>>> some years ago (e.g. "open-telnet" -> "accessible-telnet").
>>> So far, we have not adopted those changes for the classification identifiers
>>> but still use "open-telnet" etc. for "old" feeds.
>>> On the other hand, for newer feeds like "accessible-ftp" we use the
>>> classification identifier "accessible-ftp".
>>> So we have "open-telnet" but "accessible-ftp" which is not consistent.
>>> 
>>> We should probably discuss which services are "open" and which ones are
>>> "accessible" and change the classification identifiers accordingly.
>>> 
>>> Of course, all those changes need to be documented in the CHANGELOG and
>>> we should provide SQL UPDATE statements in NEWS.md like for the changes
>>> in version 3.0.0.
>>> 
>>> 
>>>     - Thomas
>>> 
>>> _______________________________________________
>>> IntelMQ-dev mailing list
>>> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
>>> https://intelmq.readthedocs.io/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20240208/ffee695f/attachment-0001.sig>