[IntelMQ-dev] Question on a few feeds' field mappings in Shadowserver parser

[Mika Silander]

Hi,

 We discovered a few Shadowserver feeds with field mappings that look somewhat odd. We use the feed mapping file https://github.com/The-Shadowserver-Foundation/report_schema/blob/main/intelmq.json and our Shadowserver parser bot is configured to update its own copy of this file dynamically. Our intelmq and its Shadowserver parser bot is the one from intelmq git repo branch release-3.3.1.

 In the intelmq.json file mentioned above, the Sandbox URL feed defines the optional input field "user_agent" to be parsed on output to "user_agent" (right?): 

         [
            "user_agent",
            "user_agent",
            "validate_to_none"
         ],

 However, the parser bot appears to output "extra.user_agent" instead.

 The other mapping that seemed odd was in Sinkhole Events HTTP IPv4 & IPv6 (and in Microsoft Sinkhole Events HTTP IPv4):

        [
            "destination.url",
            "http_url",
            "convert_http_host_and_url",
            true
         ],

 I interpret here that the optional input field "http_url" should be mapped by the Shadowserver parser bot to "destination.url" on output, but we seem to get it mapped to "extra.http_url" instead.

 Is this a hickup in intelmq.json, the parser or have I (again) missed something? Anyone else seeing this?

Best regards, Mika