<html><head></head><body><div class="ydp42f012eeyahoo-style-wrap" style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;"><div></div>
<div><br></div><div dir="ltr" data-setdir="false">Hi Sebastian,</div><div dir="ltr" data-setdir="false">Find output below ;</div><div dir="ltr" data-setdir="false"><div><div>sha256sum /usr/lib/python3.5/csv.py</div><div>6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64 /usr/lib/python3.5/csv.py</div></div><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">Below is error I get after I parse email;</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><div><div>2020-03-19 14:02:16,409 - ShadowServer-Parser - ERROR - Failed to parse line.</div><div>Traceback (most recent call last):</div><div> File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 951, in process</div><div> events = list(filter(bool, value))</div><div> File "/usr/local/lib/python3.5/dist-packages/intelmq/bots/parsers/shadowserver/parser.py", line 111, in parse_line</div><div> ' format or misconfiguration.'.format(shadowkey, self.feedname))</div><div><b>ValueError: Required column 'timestamp' not found in feed 'Blacklisted-IP'. Possible change in data format or misconfiguration.</b></div><div>2020-03-19 14:02:16,410 - ShadowServer-Parser - INFO - Dumping message to dump file.</div><div>2020-03-19 14:02:16,514 - ShadowServer-Parser - INFO - Dumping message to dump file.</div><div>2020-03-19 14:02:16,612 - ShadowServer-Parser - INFO - Sent 0 events and found 2 problem(s).</div></div><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">From Dump;</div><div dir="ltr" data-setdir="false"><div><div> "source_queue": "ShadowServer-Parser-queue",</div><div> "traceback": "Traceback (most recent call last):\n File \"/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py\", line 951, in process\n events = list(filter(bool, value))\n File \"/usr/local/lib/python3.5/dist-<b>packages/intelmq/bots/parsers/shadowserver/parser.py\", line 111, in parse_line\n ' format or misconfiguration.'.format(shadowkey, self.feedname))\nValueError: Required column 'timestamp' not found in feed 'Blacklisted-IP'. Possible change in data format or misconfiguration.\n"</b></div><div> }</div></div><br></div><div dir="ltr" data-setdir="false">Thank you,</div><div dir="ltr" data-setdir="false">Patrick</div><div><br></div>
</div><div id="yahoo_quoted_5594011519" class="yahoo_quoted">
<div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">
<div>
On Thursday, March 19, 2020, 01:48:02 PM GMT+3, Sebastian Wagner <wagner@cert.at> wrote:
</div>
<div><br></div>
<div><br></div>
<div><div id="yiv4521926085"><div>
<p>Hi,</p>
<p>On a fresh, virgin and up-to-date xenial installation I get the
same results as Bernhard, which differ from yours. I suggest to
re-install the affected python3 package on your system (or, maybe
just the whole python3-stack):</p>
<pre><code>sudo apt-get install --reinstall libpython3.5-stdlib
And then show the output of the commands again.
Sebsatian
</code></pre>
<div class="yiv4521926085yqt4862352777" id="yiv4521926085yqtfd27587"><div class="yiv4521926085moz-cite-prefix">On 3/19/20 10:48 AM, patric sungura
wrote:<br clear="none">
</div>
<blockquote type="cite">
</blockquote></div></div><div class="yiv4521926085yqt4862352777" id="yiv4521926085yqtfd81927"></div><div><div class="yiv4521926085yqt4862352777" id="yiv4521926085yqtfd78882"><div class="yiv4521926085ydp56032642yahoo-style-wrap" style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;">
<div dir="ltr">Dear team,</div>
<div dir="ltr">Find below inputs as
requested;</div>
<div><br clear="none">
</div>
<div dir="ltr">
<div>
<div><b>##ls -l /usr/lib/python3.5/csv.py</b></div>
<div>-rw-r--r-- 1 root root 16138 Mar 19 11:28
/usr/lib/python3.5/csv.py</div>
<div><br clear="none">
</div>
<div><b>##sha256sum /usr/lib/python3.5/csv.py</b></div>
<div>fbed6ea919f70b4fd29608ab9620785304fc50a4b90cd2dd5d2363a0b81d9e81
/usr/lib/python3.5/csv.py</div>
<div><br clear="none">
</div>
<div dir="ltr"><b>Find line aroud
line#96</b></div>
<div><br clear="none">
</div>
<div> 79 class DictReader:</div>
<div> 80 def __init__(self, f, fieldnames=None,
restkey=None, restval=None,</div>
<div> 81 dialect="excel", *args, **kwds):</div>
<div> 82 self._fieldnames = fieldnames # list of
keys for the dict</div>
<div> 83 self.restkey = restkey # key to
catch long rows</div>
<div> 84 self.restval = restval # default
value for short rows</div>
<div> 85 self.reader = reader(f, dialect, *args,
**kwds)</div>
<div> 86 self.dialect = dialect</div>
<div> 87 self.line_num = 0</div>
<div> 88 </div>
<div> 89 def __iter__(self):</div>
<div> 90 return self</div>
<div> 91 </div>
<div> 92 @property</div>
<div> 93 def fieldnames(self):</div>
<div> 94 if self._fieldnames is None:</div>
<div> 95 try:</div>
<div><b> 96 self._fieldnames =
next(self.reader)</b></div>
<div> 97 except StopIteration:</div>
<div> 98 pass</div>
<div> 99 self.line_num = self.reader.line_num</div>
<div>100 return self._fieldnames</div>
<div>101 </div>
<div>102 @fieldnames.setter</div>
<div>103 def fieldnames(self, value):</div>
<div>104 self._fieldnames = value </div>
</div>
<br clear="none">
</div>
<div dir="ltr">I have two shadow server
Parser and now have recorded different errors;</div>
<div><br clear="none">
</div>
<div dir="ltr"><span><span style="color:rgb(0, 0, 0);font-family:Helvetica Neue, Helvetica, Arial, sans-serif;"><b>ShadowServer-Parser-2 </b></span></span><br clear="none">
</div>
<div dir="ltr">
<div>
<div>2020-03-19 12:31:07,318 - ShadowServer-Parser-2 - ERROR
- Bot has found a problem.</div>
<div>Traceback (most recent call last):</div>
<div> File
"/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py",
line 267, in start</div>
<div> self.process()</div>
<div> File
"/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py",
line 941, in process</div>
<div> for line in self.parse(report):</div>
<div> File
"/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py",
line 884, in parse_csv_dict</div>
<div> self.csv_fieldnames = csv_reader.fieldnames</div>
<div> File "/usr/lib/python3.5/csv.py", line 96, in
fieldnames</div>
<div> self._fieldnames = next(self.reader)</div>
<div><b>NameError: name 'csv' is not defined</b></div>
<div>2020-03-19 12:31:07,319 - ShadowServer-Parser-2 - INFO
- Dumping message to dump file.</div>
<div><br clear="none">
</div>
<div dir="ltr"><span><span style="color:rgb(0, 0, 0);font-family:Helvetica Neue, Helvetica, Arial, sans-serif;"><b>ShadowServer-Parser</b></span></span><br clear="none">
</div>
<div>2020-03-19 12:37:58,573 - ShadowServer-Parser - ERROR -
Bot has found a problem.</div>
<div>Traceback (most recent call last):</div>
<div> File
"/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py",
line 267, in start</div>
<div> self.process()</div>
<div> File
"/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py",
line 941, in process</div>
<div> for line in self.parse(report):</div>
<div> File
"/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py",
line 884, in parse_csv_dict</div>
<div> self.csv_fieldnames = csv_reader.fieldnames</div>
<div> File "/usr/lib/python3.5/csv.py", line 96, in
fieldnames</div>
<div> self._fieldnames = next(self.reader)</div>
<div>_<b>csv.Error: new-line character seen in unquoted
field - do you need to open the file in
universal-newline mode?</b></div>
<div>2020-03-19 12:37:58,574 - ShadowServer-Parser - INFO -
Dumping message to dump file</div>
</div>
<br clear="none">
</div>
<div dir="ltr"><br clear="none">
</div>
<div dir="ltr"><br clear="none">
</div>
<div dir="ltr">Thank you very much,</div>
<div dir="ltr">Patrick</div>
</div>
</div><div class="yiv4521926085yahoo_quoted" id="yiv4521926085yahoo_quoted_5587659601"><div class="yiv4521926085yqt4862352777" id="yiv4521926085yqtfd44840">
</div><div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;"><div class="yiv4521926085yqt4862352777" id="yiv4521926085yqtfd60458">
<div> On Thursday, March 19, 2020, 11:56:49 AM GMT+3, Bernhard
Reiter <a rel="nofollow" shape="rect" class="yiv4521926085moz-txt-link-rfc2396E" ymailto="mailto:bernhard@intevation.de" target="_blank" href="mailto:bernhard@intevation.de"><bernhard@intevation.de></a> wrote: </div>
<div><br clear="none">
</div>
<div><br clear="none">
</div>
</div><div><div class="yiv4521926085yqt4862352777" id="yiv4521926085yqtfd27452">
<div dir="ltr">Am Mittwoch, 18. März 2020, 17:02:13 CET
schrieben Sie:<br clear="none">
> libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9
amd64<br clear="none">
<br clear="none">
This is the standard version, but still the backtrace
looks strange,<br clear="none">
especially the last bit:<br clear="none">
File "/usr/lib/python3.5/csv.py", line 96, in fieldnames<br clear="none">
self._fieldnames = next(self.reader,
dialect=csv.excel_tab)<br clear="none">
NameError: name 'csv' is not defined<br clear="none">
<br clear="none">
The second part is strange as the original sources [1]
don't have the second<br clear="none">
parameter. So we should double check.<br clear="none">
Can you open the file `/usr/lib/python3.5/csv.py` and
check line 96?<br clear="none">
<br clear="none">
Please send the whole function around the lines here.<br clear="none">
(Can you also post your mail in text-only format? This is
something I can and <br clear="none">
many others can read much faster, and thus can answer you
faster. :) Thanks! )<br clear="none">
<br clear="none">
Can you also post the length, permissions and sha256 sum
of the file?<br clear="none">
Here for comparison the data for my file:<br clear="none">
ls -l /usr/lib/python3.5/csv.py<br clear="none">
-rw-r--r-- 1 root root 16128 Oct 8 16:38
/usr/lib/python3.5/csv.py<br clear="none">
sha256sum /usr/lib/python3.5/csv.py<br clear="none">
6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64
/usr/lib/<br clear="none">
python3.5/csv.py<br clear="none">
<br clear="none">
Best Regards,<br clear="none">
Bernhard<br clear="none">
[1] <a rel="nofollow" shape="rect" target="_blank" href="https://github.com/python/cpython/blob/master/Lib/csv.py">https://github.com/python/cpython/blob/master/Lib/csv.py</a>
<div class="yiv4521926085yqt0000735210" id="yiv4521926085yqtfd32456"><br clear="none">
<br clear="none">
-- <br clear="none">
<a rel="nofollow" shape="rect" class="yiv4521926085moz-txt-link-abbreviated" target="_blank" href="http://www.intevation.de/~bernhard">www.intevation.de/~bernhard</a> +49 541 33 508 3-3<br clear="none">
Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück,
HRB 18998<br clear="none">
Geschäftsführer Frank Koormann, Bernhard Reiter, Dr.
Jan-Oliver Wagner</div>
</div>
</div><div class="yiv4521926085yqt0000735210" id="yiv4521926085yqtfd68988"><div class="yiv4521926085yqt4862352777" id="yiv4521926085yqtfd54411">-- <br clear="none">
Listen-Einstellungen:<br clear="none">
<a rel="nofollow" shape="rect" target="_blank" href="https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users">https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users</a></div><br clear="none">
</div>
</div>
</div>
</div>
<br clear="none">
<fieldset class="yiv4521926085mimeAttachmentHeader"></fieldset>
<pre class="yiv4521926085moz-signature">--
// Sebastian Wagner <a rel="nofollow" shape="rect" class="yiv4521926085moz-txt-link-rfc2396E" ymailto="mailto:wagner@cert.at" target="_blank" href="mailto:wagner@cert.at"><wagner@cert.at></a> - T: +43 1 5056416 7201
// CERT Austria - <a rel="nofollow" shape="rect" class="yiv4521926085moz-txt-link-freetext" target="_blank" href="https://www.cert.at/">https://www.cert.at/</a>
// Eine Initiative der nic.at GmbH - <a rel="nofollow" shape="rect" class="yiv4521926085moz-txt-link-freetext" target="_blank" href="https://www.nic.at/">https://www.nic.at/</a>
// Firmenbuchnummer 172568b, LG Salzburg</pre><div class="yiv4521926085yqt4862352777" id="yiv4521926085yqtfd44906">
</div></div></div><div class="yqt4862352777" id="yqtfd93819">-- <br clear="none">Listen-Einstellungen:<br clear="none"> <a shape="rect" href="https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users" target="_blank">https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users</a><br clear="none"></div></div>
</div>
</div></body></html>