[IntelMQ-dev] classification attributes in IntelMQ Shadowserver parser schema

Thomas Hungenberg th at cert-bund.de
Tue Feb 6 11:11:59 CET 2024 

IMHO we should not make further significant changes to the schema now for the upcoming release
but discuss in detail for which feeds the classification.type should be changed from "vulnerable-system"
to "potentially-unwanted-accessible" and where the old classification.identifiers open-* should be
changed to accessible-* in future versions of the schema.


     - Thomas


On 05.02.24 16:38, elsif wrote:
> Should I make further changes to the schema or should the proposed version be published?
> 
> On 2/5/24 1:46 AM, Kamil Mankowski via IntelMQ-dev wrote:
>> Or rather not fully - as @gethvi brought to my attention that most of "Accessible" or "Open" feeds should be rather classified as 
>> "potentially-unwanted-accessible" according to the taxonomy 
>> (https://github.com/enisaeu/Reference-Security-Incident-Taxonomy-Task-Force/blob/master/working_copy/humanv1.md) - instead of vulnerable-system or 
>> other.
>>
>> For many cases we have our own classification enforced - I'm attaching an extract from my configuration to compare with the original schema. It's a 
>> YAML list used to generate the final configuration later.
>>
>> Best regards
>>
>> // Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
>> // CERT Austria - https://www.cert.at/
>> // CERT.at GmbH, FB-Nr. 561772k, HG Wien
>>
>> On 2/5/24 10:15, Kamil Mankowski wrote:
>>> Hi, thanks for the changes and reviews! They looks good to me too!
>>>
>>> Best regards
>>>
>>> // Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
>>> // CERT Austria - https://www.cert.at/
>>> // CERT.at GmbH, FB-Nr. 561772k, HG Wien
>>>
>>> On 2/2/24 11:48, Thomas Hungenberg via IntelMQ-dev wrote:
>>>> Hi,
>>>>
>>>> thanks a lot for your prompt response and sorry for the delay on my side.
>>>>
>>>> The changes look good!
>>>>
>>>> However, I have made a few additional changes:
>>>>
>>>> 1)
>>>> Make classification.identifier for honeypot_ics_scan consistent
>>>> with other honeypot scans:
>>>> =====================
>>>>      "event_honeypot_ics_scan" : {
>>>>         "constant_fields" : {
>>>> -         "classification.identifier" : "ics",
>>>> +         "classification.identifier" : "honeypot-ics-scan",
>>>> =====================
>>>>
>>>> This change should be documented here:
>>>> https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/completed-changes.md
>>>>
>>>>
>>>> 2)
>>>> Change classification.taxonomy and classification.type from
>>>>
>>>>           "classification.taxonomy" : "other",
>>>>           "classification.type" : "other",
>>>>
>>>> to
>>>>           "classification.taxonomy" : "vulnerable",
>>>>           "classification.type" : "vulnerable-system",
>>>>
>>>> for accessible-bgp and accessible-msmq.
>>>>
>>>> Not included in old _config.py, so no need to document.
>>>>
>>>>
>>>> 3)
>>>> Change classification.taxonomy and classification.type from
>>>>
>>>>           "classification.taxonomy" : "other",
>>>>           "classification.type" : "other",
>>>>
>>>> to
>>>>           "classification.taxonomy" : "vulnerable",
>>>>           "classification.type" : "vulnerable-system",
>>>>
>>>> for open-mysql, open-postgres, open-couchdb, open-epmd.
>>>>
>>>> This change should be documented here:
>>>> https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/completed-changes.md
>>>>
>>>>
>>>> 4)
>>>> Correct classification.identifier for vulnerable-http:
>>>> =====================
>>>>     "scan_http_vulnerable" : {
>>>>         "constant_fields" : {
>>>> -         "classification.identifier" : "accessible-http",
>>>> +         "classification.identifier" : "vulnerable-http",
>>>>
>>>>      "scan6_http_vulnerable" : {
>>>>         "constant_fields" : {
>>>> -         "classification.identifier" : "accessible-http",
>>>> +         "classification.identifier" : "vulnerable-http",
>>>> =====================
>>>>
>>>> This change should be documented here:
>>>> https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/completed-changes.md
>>>>
>>>>
>>>> Please find the updates intelmq.json attached.
>>>>
>>>>
>>>> Kind regards
>>>> Thomas
>>>>
>>>>
>>>> On 31.01.24 16:42, elsif wrote:
>>>>> Hello,
>>>>>
>>>>> Proposed changes are attached. Please let me know if you agree with the changes or have any alterations.
>>>>>
>>>>> Regards
>>>>>
>>>>> On 1/31/24 7:05 AM, Thomas Hungenberg wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Sebastian (sebix) told me it was agreed that with the translation
>>>>>> from the current parser _config.py (included with IntelMQ 3.2.1)
>>>>>> to the new schema, no classification.* attributes will be changed.
>>>>>>
>>>>>> This is very important as our setup (and most probably others as well)
>>>>>> heavily depends on known classification identifiers like "open-rdp"
>>>>>> and classification types from the initial parsing of events up to
>>>>>> notification_rules and formats/templates for mailgen.
>>>>>> So with a change of a classification attribute lots of scripts and
>>>>>> configs would need to be changed as well.
>>>>>>
>>>>>> Looking at the current schema, I see the classification identifiers
>>>>>> are still correct for some feeds for both IPv4 and IPv6 like here:
>>>>>>
>>>>>>    "scan_dns" : {
>>>>>>       "constant_fields" : {
>>>>>>          "classification.identifier" : "dns-open-resolver",
>>>>>>
>>>>>>    "scan6_dns" : {
>>>>>>       "constant_fields" : {
>>>>>>          "classification.identifier" : "dns-open-resolver",
>>>>>>
>>>>>>
>>>>>> However, for other feeds the classification identifier has been kept
>>>>>> correctly for IPv4 like here:
>>>>>>
>>>>>>    "scan_rdp" : {
>>>>>>       "constant_fields" : {
>>>>>>          "classification.identifier" : "open-rdp",
>>>>>>
>>>>>>    "compromised_website" : {
>>>>>>       "constant_fields" : {
>>>>>>          "classification.identifier" : "compromised-website",
>>>>>>
>>>>>>
>>>>>> but for IPv6 it has changed to the name of the feed:
>>>>>>
>>>>>>    "scan6_rdp" : {
>>>>>>       "constant_fields" : {
>>>>>>          "classification.identifier" : "scan6-rdp", <- should be "open-rdp"
>>>>>>
>>>>>>    "compromised_website6" : {
>>>>>>       "constant_fields" : {
>>>>>>          "classification.identifier" : "compromised-website6", <- should be "compromised-website"
>>>>>>
>>>>>>
>>>>>> The classification.identifier should describe the incident (like "open-rdp")
>>>>>> and not the source (like "scan6-rdp").
>>>>>>
>>>>>> May I ask you to check and adjust all classification identifiers and types
>>>>>> in the schema so they are consistent with the ones generated by the current
>>>>>> _config.py?
>>>>>>
>>>>>>
>>>>>> Thanks a lot for all your work on the new schema based parser!
>>>>>>
>>>>>>
>>>>>> Kind regards
>>>>>> Thomas
>>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> IntelMQ-dev mailing list
>>>> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
>>>> https://intelmq.readthedocs.io/
>>
>> _______________________________________________
>> IntelMQ-dev mailing list
>> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
>> https://intelmq.readthedocs.io/
> _______________________________________________
> IntelMQ-dev mailing list
> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
> https://intelmq.readthedocs.io/

More information about the IntelMQ-dev mailing list