[IntelMQ-dev] classification attributes in IntelMQ Shadowserver parser schema

Kamil Mankowski mankowski at cert.at
Tue Feb 6 13:42:58 CET 2024 

I second that. Especially when schema is now independent, let's stay 
with the last proposal, and speak about taxonomy separately. I haven't 
had time yet to deeply compare which we have differently.

When it comes to identifiers changes, I would be very conservative. They 
can be used for filtering, and as so - changing them is potentially 
dangerous. I second fixes about IPv6, those were more misleading than 
helping, but for the rest - we need to be careful and announce the change.

Best regards

// Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
// CERT Austria - https://www.cert.at/
// CERT.at GmbH, FB-Nr. 561772k, HG Wien

On 2/6/24 11:11, Thomas Hungenberg wrote:
> IMHO we should not make further significant changes to the schema now 
> for the upcoming release
> but discuss in detail for which feeds the classification.type should be 
> changed from "vulnerable-system"
> to "potentially-unwanted-accessible" and where the old 
> classification.identifiers open-* should be
> changed to accessible-* in future versions of the schema.
> 
> 
>      - Thomas
> 
> 
> On 05.02.24 16:38, elsif wrote:
>> Should I make further changes to the schema or should the proposed 
>> version be published?
>>
>> On 2/5/24 1:46 AM, Kamil Mankowski via IntelMQ-dev wrote:
>>> Or rather not fully - as @gethvi brought to my attention that most of 
>>> "Accessible" or "Open" feeds should be rather classified as 
>>> "potentially-unwanted-accessible" according to the taxonomy 
>>> (https://github.com/enisaeu/Reference-Security-Incident-Taxonomy-Task-Force/blob/master/working_copy/humanv1.md) - instead of vulnerable-system or other.
>>>
>>> For many cases we have our own classification enforced - I'm 
>>> attaching an extract from my configuration to compare with the 
>>> original schema. It's a YAML list used to generate the final 
>>> configuration later.
>>>
>>> Best regards
>>>
>>> // Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
>>> // CERT Austria - https://www.cert.at/
>>> // CERT.at GmbH, FB-Nr. 561772k, HG Wien
>>>
>>> On 2/5/24 10:15, Kamil Mankowski wrote:
>>>> Hi, thanks for the changes and reviews! They looks good to me too!
>>>>
>>>> Best regards
>>>>
>>>> // Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
>>>> // CERT Austria - https://www.cert.at/
>>>> // CERT.at GmbH, FB-Nr. 561772k, HG Wien
>>>>
>>>> On 2/2/24 11:48, Thomas Hungenberg via IntelMQ-dev wrote:
>>>>> Hi,
>>>>>
>>>>> thanks a lot for your prompt response and sorry for the delay on my 
>>>>> side.
>>>>>
>>>>> The changes look good!
>>>>>
>>>>> However, I have made a few additional changes:
>>>>>
>>>>> 1)
>>>>> Make classification.identifier for honeypot_ics_scan consistent
>>>>> with other honeypot scans:
>>>>> =====================
>>>>>      "event_honeypot_ics_scan" : {
>>>>>         "constant_fields" : {
>>>>> -         "classification.identifier" : "ics",
>>>>> +         "classification.identifier" : "honeypot-ics-scan",
>>>>> =====================
>>>>>
>>>>> This change should be documented here:
>>>>> https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/completed-changes.md
>>>>>
>>>>>
>>>>> 2)
>>>>> Change classification.taxonomy and classification.type from
>>>>>
>>>>>           "classification.taxonomy" : "other",
>>>>>           "classification.type" : "other",
>>>>>
>>>>> to
>>>>>           "classification.taxonomy" : "vulnerable",
>>>>>           "classification.type" : "vulnerable-system",
>>>>>
>>>>> for accessible-bgp and accessible-msmq.
>>>>>
>>>>> Not included in old _config.py, so no need to document.
>>>>>
>>>>>
>>>>> 3)
>>>>> Change classification.taxonomy and classification.type from
>>>>>
>>>>>           "classification.taxonomy" : "other",
>>>>>           "classification.type" : "other",
>>>>>
>>>>> to
>>>>>           "classification.taxonomy" : "vulnerable",
>>>>>           "classification.type" : "vulnerable-system",
>>>>>
>>>>> for open-mysql, open-postgres, open-couchdb, open-epmd.
>>>>>
>>>>> This change should be documented here:
>>>>> https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/completed-changes.md
>>>>>
>>>>>
>>>>> 4)
>>>>> Correct classification.identifier for vulnerable-http:
>>>>> =====================
>>>>>     "scan_http_vulnerable" : {
>>>>>         "constant_fields" : {
>>>>> -         "classification.identifier" : "accessible-http",
>>>>> +         "classification.identifier" : "vulnerable-http",
>>>>>
>>>>>      "scan6_http_vulnerable" : {
>>>>>         "constant_fields" : {
>>>>> -         "classification.identifier" : "accessible-http",
>>>>> +         "classification.identifier" : "vulnerable-http",
>>>>> =====================
>>>>>
>>>>> This change should be documented here:
>>>>> https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/completed-changes.md
>>>>>
>>>>>
>>>>> Please find the updates intelmq.json attached.
>>>>>
>>>>>
>>>>> Kind regards
>>>>> Thomas
>>>>>
>>>>>
>>>>> On 31.01.24 16:42, elsif wrote:
>>>>>> Hello,
>>>>>>
>>>>>> Proposed changes are attached. Please let me know if you agree 
>>>>>> with the changes or have any alterations.
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> On 1/31/24 7:05 AM, Thomas Hungenberg wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Sebastian (sebix) told me it was agreed that with the translation
>>>>>>> from the current parser _config.py (included with IntelMQ 3.2.1)
>>>>>>> to the new schema, no classification.* attributes will be changed.
>>>>>>>
>>>>>>> This is very important as our setup (and most probably others as 
>>>>>>> well)
>>>>>>> heavily depends on known classification identifiers like "open-rdp"
>>>>>>> and classification types from the initial parsing of events up to
>>>>>>> notification_rules and formats/templates for mailgen.
>>>>>>> So with a change of a classification attribute lots of scripts and
>>>>>>> configs would need to be changed as well.
>>>>>>>
>>>>>>> Looking at the current schema, I see the classification identifiers
>>>>>>> are still correct for some feeds for both IPv4 and IPv6 like here:
>>>>>>>
>>>>>>>    "scan_dns" : {
>>>>>>>       "constant_fields" : {
>>>>>>>          "classification.identifier" : "dns-open-resolver",
>>>>>>>
>>>>>>>    "scan6_dns" : {
>>>>>>>       "constant_fields" : {
>>>>>>>          "classification.identifier" : "dns-open-resolver",
>>>>>>>
>>>>>>>
>>>>>>> However, for other feeds the classification identifier has been kept
>>>>>>> correctly for IPv4 like here:
>>>>>>>
>>>>>>>    "scan_rdp" : {
>>>>>>>       "constant_fields" : {
>>>>>>>          "classification.identifier" : "open-rdp",
>>>>>>>
>>>>>>>    "compromised_website" : {
>>>>>>>       "constant_fields" : {
>>>>>>>          "classification.identifier" : "compromised-website",
>>>>>>>
>>>>>>>
>>>>>>> but for IPv6 it has changed to the name of the feed:
>>>>>>>
>>>>>>>    "scan6_rdp" : {
>>>>>>>       "constant_fields" : {
>>>>>>>          "classification.identifier" : "scan6-rdp", <- should be 
>>>>>>> "open-rdp"
>>>>>>>
>>>>>>>    "compromised_website6" : {
>>>>>>>       "constant_fields" : {
>>>>>>>          "classification.identifier" : "compromised-website6", <- 
>>>>>>> should be "compromised-website"
>>>>>>>
>>>>>>>
>>>>>>> The classification.identifier should describe the incident (like 
>>>>>>> "open-rdp")
>>>>>>> and not the source (like "scan6-rdp").
>>>>>>>
>>>>>>> May I ask you to check and adjust all classification identifiers 
>>>>>>> and types
>>>>>>> in the schema so they are consistent with the ones generated by 
>>>>>>> the current
>>>>>>> _config.py?
>>>>>>>
>>>>>>>
>>>>>>> Thanks a lot for all your work on the new schema based parser!
>>>>>>>
>>>>>>>
>>>>>>> Kind regards
>>>>>>> Thomas
>>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> IntelMQ-dev mailing list
>>>>> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
>>>>> https://intelmq.readthedocs.io/
>>>
>>> _______________________________________________
>>> IntelMQ-dev mailing list
>>> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
>>> https://intelmq.readthedocs.io/
>> _______________________________________________
>> IntelMQ-dev mailing list
>> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
>> https://intelmq.readthedocs.io/
> 
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20240206/8d38c8e6/attachment.sig>

More information about the IntelMQ-dev mailing list