<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>I think the URL parsing is fixed by Thomas' PR
<a class="moz-txt-link-freetext" href="https://github.com/certtools/intelmq/pull/1243">https://github.com/certtools/intelmq/pull/1243</a><br>
That was part of the last releases already<br>
</p>
<br>
<div class="moz-cite-prefix">On 2018-01-07 00:20, Tomislav Protega
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:2b273fd5-91dc-3ecf-2b0c-87d6e73e61be@cert.hr">
<pre wrap="">I took a look at the other reports where there is domain under
'http_host', but the main problem is that parser is joining wrong fields
from shadowserver report.
It joins 'hostname' with 'url' parameters which it shouldn't do, because
under hostname is actually dns ptr record (source_reverse.dns).
So it should join 'http_host'(source.fqdn) + 'url' to get the real
source.url.
Regards,
--
Tomislav
On 07.01.2018 00:02, Tomislav Protega wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi,
I ran into this error:
Shadowserver-Compromised-Website-Parser - ERROR - Could not convert
shadowkey: 'http_host', value: '' via conversion function 'validate_fqdn'.
More detailed log is attached.
This happens when "http_host" field in the shadowserver origin report
contains IP instead of domain which is not something unusual.
At the end IntelMQ does produce the output data, but there's no
'source.url' field which should contain merged 'http_host' and 'url'
parameters from the origin report.
Regards,
</pre>
</blockquote>
<pre wrap="">
</pre>
<!--'"--><br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
// Sebastian Wagner <a class="moz-txt-link-rfc2396E" href="mailto:wagner@cert.at"><wagner@cert.at></a> - T: +43 1 5056416 7201
// CERT Austria - <a class="moz-txt-link-freetext" href="https://www.cert.at/">https://www.cert.at/</a>
// Eine Initiative der nic.at GmbH - <a class="moz-txt-link-freetext" href="https://www.nic.at/">https://www.nic.at/</a>
// Firmenbuchnummer 172568b, LG Salzburg</pre>
</body>
</html>