[IntelMQ-dev] Feed handling bug in shadowserver parser bot?

[Mika Silander]

Hi,

 Currently the parameters section for the parser bot in runtime.yaml is just:

  parameters:
    destination_queues:
      _default: [fc-set-event-constant-expert-queue]
    overwrite: true
  run_mode: continuous

Br, Mika


----- Original Message -----
From: "Sebix" <sebix at sebix.at>
To: "Mika Silander" <mika.silander at csc.fi>, "intelmq-dev" <intelmq-dev at lists.cert.at>
Sent: Monday, 5 September, 2022 10:43:56
Subject: Re: [IntelMQ-dev] Feed handling bug in shadowserver parser bot?

Can you please show the configuration (parameters) of your shadowserver
parser?

On 9/5/22 9:42 AM, Mika Silander wrote:
> Hi,
>
>  We've been running intelmq in a production-like setup for some time now and occasionally we see the Shadowserver Parser bot behave in a way that looks odd. We push reports to it via the mail attachment collector bot. At times, the parser bot ends up treating all reports as being of the type Vulnerable ISAKMP. We don't know what triggers this behaviour but so far we've seen this when the parser parses Sandbox URL and Open TFTP reports, both filling the log (debug level) with notifications of "missing keys" or "optional keys not found" because the parser assumes the feed to be Vulnerable ISAKMP. And before this occurs, the parser has been running for a good while.
>
>  We've picked up the the problematic Sandbox URL and Open TFTP reports above and made unit test cases of these - all tests are correctly parsed, the feed name is deduced correctly and the tests are successful, so the parser handles these correctly if these are the first reports parsed. Therefore, our assumption is that this problem occurs only when the parser has been running for a longer time and some state information about the feed type does not get cleared between parsing incoming reports.
>
>  Anyone else experiencing similar problems? This is a tricky one to debug so I decided to ask on the list first.
>
> Br, Mika
>
> P.S: Our setup is intelmq 3.0.2 on an Ubuntu 20.04 LTS  
> _______________________________________________
> IntelMQ-dev mailing list
> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
> https://intelmq.readthedocs.io/

-- 
Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://sebix.at/
ZVR 1510673578