Hi,
like for other Shadowserver reports, we are sending email notifications to affected network owners in Germany based on the Vulnerable-HTTP report. Unlike other reports (like Open-Portmapper), the Vulnerable-HTTP report contains information on different kinds of software/systems. So we are sending the notifications with a different template for each software/system.
For some software/systems (like Fortinet or VMWare), the report contains information on different vulnerabilities (CVEs) and each host included in the report can be affected by one or more of them as listed in "tag", e.g.
"cve-2019-5544;cve-2020-3992;cve-2021-21974;vmware" "cve-2020-3992;cve-2021-21974;vmware" "cve-2019-5544;vmware" "cve-2021-21974;vmware" "cve-2021-21972;cve-2023-34048;vcenter-dcerpc-exposed;vmware" "cve-2021-21972;cve-2023-20892;cve-2023-34048;vmware"
So we need to tell the recipients of our notifications to which of the vulnerabilities their host is exactly vulnerable to. We do this by adding a column "cve" in addition to "ip", "port", etc. to the CSV data included with our notifications.
However, including the raw "tag" attribute from the Shadowserver report would confuse the recipients as "tag" often includes additional information not relevant in the context of the specific report like "ssl-poodle", e.g.
"cve-2024-23897;jenkins;ssl;ssl-poodle" "cve-2023-46805;cve-2024-21887;ivanti-connect-secure;ssl;ssl-freak;vpn"
So we added a function "extract_cve_from_tag" to our local Shadowserver parser _config.py (included with IntelMQ 3.2.1) which returns a sorted comma separated list of CVEs included in "tag", e.g.
extract_cve_from_tag("cve-2023-46805;cve-2024-21887;ivanti-connect-secure;ssl;ssl-freak;vpn") -> "cve-2023-46805,cve-2024-21887"
The result is stored in "extra.cve" by the parser
scan_http_vulnerable = { 'optional_fields': [ ('protocol.transport', 'protocol'), ('source.reverse_dns', 'hostname'), ('extra.', 'tag', validate_to_none), ('extra.cve', 'tag', extract_cve_from_tag),
and this way can later easily be included in the CSV data generated for the notifications.
I think this might be useful for others as well and would be glad if it could be added to the parser code and schema for the next release so we would not need to patch it locally after each update of the parser/schema.
I have just created a pull request for this: https://github.com/certtools/intelmq/pull/2457
@elsif: If the PR is accepted, would you like to change the schema for scan_http_vulnerable and scan6_http_vulnerable like this?
[ "extra.", "tag", "validate_to_none" ], + [ + "extra.cve", + "tag", + "extract_cve_from_tag" + ],
Kind regards Thomas