[Intelmq-users] Shadowserver compromised website parser -ERROR

[Tomislav Protega]

Hi,

I ran into this error:
Shadowserver-Compromised-Website-Parser - ERROR - Could not convert
shadowkey: 'http_host', value: '' via conversion function 'validate_fqdn'.
More detailed log is attached.

This happens when "http_host" field in the shadowserver origin report
contains IP instead of domain which is not something unusual.

At the end IntelMQ does produce the output data, but there's no
'source.url' field which should contain merged 'http_host' and 'url'
parameters from the origin report.

Regards,

-- 
Tomislav

-------------- next part --------------
Shadowserver-Compromised-Website-Parser - ERROR - Could not convert shadowkey: 'http_host', value: '' via conversion function 'validate_fqdn'.
Shadowserver-Compromised-Website-Parser - ERROR - Failed to parse line.

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/intelmq/lib/bot.py", line 659, in process
    events = list(filter(bool, value))
  File "/usr/lib/python3/dist-packages/intelmq/bots/parsers/shadowserver/parser.py", line 138, in parse_line
    value = conv_func(raw_value)
  File "/usr/lib/python3/dist-packages/intelmq/bots/parsers/shadowserver/config.py", line 175, in validate_fqdn
    if harmonization.FQDN.is_valid(value, sanitize=True):
  File "/usr/lib/python3/dist-packages/intelmq/lib/harmonization.py", line 381, in is_valid
    value = FQDN().sanitize(value)
  File "/usr/lib/python3/dist-packages/intelmq/lib/harmonization.py", line 404, in sanitize
    value = value.strip('.')
AttributeError: 'NoneType' object has no attribute 'strip'
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20180107/8beb9849/attachment.sig>