Hi Mika
On 11/11/24 8:06 AM, Mika Silander via IntelMQ-dev wrote:
In the intelmq.json file mentioned above, the Sandbox URL feed defines the optional input field "user_agent" to be parsed on output to "user_agent" (right?):
[ "user_agent", "user_agent", "validate_to_none" ],However, the parser bot appears to output "extra.user_agent" instead.
Yes, because "user_agent" is used as shortcut for "extra.user_agent", because the field "user_agent" does not exist in IntelMQ. This behavior is specific to the Shadowserver-Parser, not a default in IntelMQ.
https://github.com/certtools/intelmq/blob/e86912f6740ea1592f531fbaa9713e1f60...
However, I think "explicit is better than implicit" and the behavior does not bring any advantages, only potential confusion, as in this case.
The other mapping that seemed odd was in Sinkhole Events HTTP IPv4 & IPv6 (and in Microsoft Sinkhole Events HTTP IPv4):
[ "destination.url", "http_url", "convert_http_host_and_url", true ],I interpret here that the optional input field "http_url" should be mapped by the Shadowserver parser bot to "destination.url" on output, but we seem to get it mapped to "extra.http_url" instead.
That's also how I'd interpret it, but don't have any more insights (and data/examples) here.
Best regards
Sebastian