[IntelMQ-dev] Shadowserver parser extract_cve_from_tag

[Kamil Mankowski]

Hey,

We definitely need to handle vulnerabilities in IntelMQ better. I have 
already proposed an IEP to introduce a vulnerability field in the 
harmonization, together with related product information: 
https://github.com/certtools/ieps/pull/10/files What do you think about 
that proposal?

We currently use a little different approach than just copy CVE numbers: 
we have an additional bot to split events and create them in the way one 
CVE = one event, with a CVE as identifier. This is later used to compare 
with other sources, so we can get the information about vulnerability 
from different sources, but send just a one notification.

Best regards

// Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
// CERT Austria - https://www.cert.at/
// CERT.at GmbH, FB-Nr. 561772k, HG Wien

On 2/5/24 10:03, Thomas Hungenberg via IntelMQ-dev wrote:
> Hi,
> 
> like for other Shadowserver reports, we are sending email notifications to
> affected network owners in Germany based on the Vulnerable-HTTP report.
> Unlike other reports (like Open-Portmapper), the Vulnerable-HTTP report
> contains information on different kinds of software/systems.
> So we are sending the notifications with a different template for each
> software/system.
> 
> For some software/systems (like Fortinet or VMWare), the report contains
> information on different vulnerabilities (CVEs) and each host included in
> the report can be affected by one or more of them as listed in "tag", e.g.
> 
> "cve-2019-5544;cve-2020-3992;cve-2021-21974;vmware"
> "cve-2020-3992;cve-2021-21974;vmware"
> "cve-2019-5544;vmware"
> "cve-2021-21974;vmware"
> "cve-2021-21972;cve-2023-34048;vcenter-dcerpc-exposed;vmware"
> "cve-2021-21972;cve-2023-20892;cve-2023-34048;vmware"
> 
> So we need to tell the recipients of our notifications to which of the
> vulnerabilities their host is exactly vulnerable to.
> We do this by adding a column "cve" in addition to "ip", "port", etc.
> to the CSV data included with our notifications.
> 
> However, including the raw "tag" attribute from the Shadowserver report
> would confuse the recipients as "tag" often includes additional information
> not relevant in the context of the specific report like "ssl-poodle", e.g.
> 
> "cve-2024-23897;jenkins;ssl;ssl-poodle"
> "cve-2023-46805;cve-2024-21887;ivanti-connect-secure;ssl;ssl-freak;vpn"
> 
> So we added a function "extract_cve_from_tag" to our local Shadowserver 
> parser
> _config.py (included with IntelMQ 3.2.1) which returns a sorted comma 
> separated
> list of CVEs included in "tag", e.g.
> 
> extract_cve_from_tag("cve-2023-46805;cve-2024-21887;ivanti-connect-secure;ssl;ssl-freak;vpn")
> -> "cve-2023-46805,cve-2024-21887"
> 
> The result is stored in "extra.cve" by the parser
> 
> scan_http_vulnerable = {
>      'optional_fields': [
>          ('protocol.transport', 'protocol'),
>          ('source.reverse_dns', 'hostname'),
>          ('extra.', 'tag', validate_to_none),
>          ('extra.cve', 'tag', extract_cve_from_tag),
> 
> and this way can later easily be included in the CSV data generated for
> the notifications.
> 
> I think this might be useful for others as well and would be glad if it
> could be added to the parser code and schema for the next release
> so we would not need to patch it locally after each update of the 
> parser/schema.
> 
> I have just created a pull request for this:
> https://github.com/certtools/intelmq/pull/2457
> 
> 
> @elsif: If the PR is accepted, would you like to change the schema
> for scan_http_vulnerable and scan6_http_vulnerable like this?
> 
>           [
>              "extra.",
>              "tag",
>              "validate_to_none"
>           ],
> +        [
> +           "extra.cve",
> +           "tag",
> +           "extract_cve_from_tag"
> +        ],
> 
> 
> Kind regards
> Thomas
> _______________________________________________
> IntelMQ-dev mailing list
> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
> https://intelmq.readthedocs.io/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20240205/18951081/attachment.sig>