[IntelMQ-dev] classification attributes in IntelMQ Shadowserver parser schema

[elsif]

Should I make further changes to the schema or should the proposed 
version be published?

On 2/5/24 1:46 AM, Kamil Mankowski via IntelMQ-dev wrote:
> Or rather not fully - as @gethvi brought to my attention that most of 
> "Accessible" or "Open" feeds should be rather classified as 
> "potentially-unwanted-accessible" according to the taxonomy 
> (https://github.com/enisaeu/Reference-Security-Incident-Taxonomy-Task-Force/blob/master/working_copy/humanv1.md) 
> - instead of vulnerable-system or other.
>
> For many cases we have our own classification enforced - I'm attaching 
> an extract from my configuration to compare with the original schema. 
> It's a YAML list used to generate the final configuration later.
>
> Best regards
>
> // Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
> // CERT Austria - https://www.cert.at/
> // CERT.at GmbH, FB-Nr. 561772k, HG Wien
>
> On 2/5/24 10:15, Kamil Mankowski wrote:
>> Hi, thanks for the changes and reviews! They looks good to me too!
>>
>> Best regards
>>
>> // Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
>> // CERT Austria - https://www.cert.at/
>> // CERT.at GmbH, FB-Nr. 561772k, HG Wien
>>
>> On 2/2/24 11:48, Thomas Hungenberg via IntelMQ-dev wrote:
>>> Hi,
>>>
>>> thanks a lot for your prompt response and sorry for the delay on my 
>>> side.
>>>
>>> The changes look good!
>>>
>>> However, I have made a few additional changes:
>>>
>>> 1)
>>> Make classification.identifier for honeypot_ics_scan consistent
>>> with other honeypot scans:
>>> =====================
>>>      "event_honeypot_ics_scan" : {
>>>         "constant_fields" : {
>>> -         "classification.identifier" : "ics",
>>> +         "classification.identifier" : "honeypot-ics-scan",
>>> =====================
>>>
>>> This change should be documented here:
>>> https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/completed-changes.md 
>>>
>>>
>>>
>>> 2)
>>> Change classification.taxonomy and classification.type from
>>>
>>>           "classification.taxonomy" : "other",
>>>           "classification.type" : "other",
>>>
>>> to
>>>           "classification.taxonomy" : "vulnerable",
>>>           "classification.type" : "vulnerable-system",
>>>
>>> for accessible-bgp and accessible-msmq.
>>>
>>> Not included in old _config.py, so no need to document.
>>>
>>>
>>> 3)
>>> Change classification.taxonomy and classification.type from
>>>
>>>           "classification.taxonomy" : "other",
>>>           "classification.type" : "other",
>>>
>>> to
>>>           "classification.taxonomy" : "vulnerable",
>>>           "classification.type" : "vulnerable-system",
>>>
>>> for open-mysql, open-postgres, open-couchdb, open-epmd.
>>>
>>> This change should be documented here:
>>> https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/completed-changes.md 
>>>
>>>
>>>
>>> 4)
>>> Correct classification.identifier for vulnerable-http:
>>> =====================
>>>     "scan_http_vulnerable" : {
>>>         "constant_fields" : {
>>> -         "classification.identifier" : "accessible-http",
>>> +         "classification.identifier" : "vulnerable-http",
>>>
>>>      "scan6_http_vulnerable" : {
>>>         "constant_fields" : {
>>> -         "classification.identifier" : "accessible-http",
>>> +         "classification.identifier" : "vulnerable-http",
>>> =====================
>>>
>>> This change should be documented here:
>>> https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/completed-changes.md 
>>>
>>>
>>>
>>> Please find the updates intelmq.json attached.
>>>
>>>
>>> Kind regards
>>> Thomas
>>>
>>>
>>> On 31.01.24 16:42, elsif wrote:
>>>> Hello,
>>>>
>>>> Proposed changes are attached. Please let me know if you agree with 
>>>> the changes or have any alterations.
>>>>
>>>> Regards
>>>>
>>>> On 1/31/24 7:05 AM, Thomas Hungenberg wrote:
>>>>> Hi,
>>>>>
>>>>> Sebastian (sebix) told me it was agreed that with the translation
>>>>> from the current parser _config.py (included with IntelMQ 3.2.1)
>>>>> to the new schema, no classification.* attributes will be changed.
>>>>>
>>>>> This is very important as our setup (and most probably others as 
>>>>> well)
>>>>> heavily depends on known classification identifiers like "open-rdp"
>>>>> and classification types from the initial parsing of events up to
>>>>> notification_rules and formats/templates for mailgen.
>>>>> So with a change of a classification attribute lots of scripts and
>>>>> configs would need to be changed as well.
>>>>>
>>>>> Looking at the current schema, I see the classification identifiers
>>>>> are still correct for some feeds for both IPv4 and IPv6 like here:
>>>>>
>>>>>    "scan_dns" : {
>>>>>       "constant_fields" : {
>>>>>          "classification.identifier" : "dns-open-resolver",
>>>>>
>>>>>    "scan6_dns" : {
>>>>>       "constant_fields" : {
>>>>>          "classification.identifier" : "dns-open-resolver",
>>>>>
>>>>>
>>>>> However, for other feeds the classification identifier has been kept
>>>>> correctly for IPv4 like here:
>>>>>
>>>>>    "scan_rdp" : {
>>>>>       "constant_fields" : {
>>>>>          "classification.identifier" : "open-rdp",
>>>>>
>>>>>    "compromised_website" : {
>>>>>       "constant_fields" : {
>>>>>          "classification.identifier" : "compromised-website",
>>>>>
>>>>>
>>>>> but for IPv6 it has changed to the name of the feed:
>>>>>
>>>>>    "scan6_rdp" : {
>>>>>       "constant_fields" : {
>>>>>          "classification.identifier" : "scan6-rdp", <- should be 
>>>>> "open-rdp"
>>>>>
>>>>>    "compromised_website6" : {
>>>>>       "constant_fields" : {
>>>>>          "classification.identifier" : "compromised-website6", <- 
>>>>> should be "compromised-website"
>>>>>
>>>>>
>>>>> The classification.identifier should describe the incident (like 
>>>>> "open-rdp")
>>>>> and not the source (like "scan6-rdp").
>>>>>
>>>>> May I ask you to check and adjust all classification identifiers 
>>>>> and types
>>>>> in the schema so they are consistent with the ones generated by 
>>>>> the current
>>>>> _config.py?
>>>>>
>>>>>
>>>>> Thanks a lot for all your work on the new schema based parser!
>>>>>
>>>>>
>>>>> Kind regards
>>>>> Thomas
>>>>>
>>>
>>>
>>> _______________________________________________
>>> IntelMQ-dev mailing list
>>> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
>>> https://intelmq.readthedocs.io/
>
> _______________________________________________
> IntelMQ-dev mailing list
> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
> https://intelmq.readthedocs.io/