[Intelmq-users] MAIL ATTACHMENT FETCHER FAILS TO READ/PARSE CSV FILE.

[patric sungura]

 Dear team,Find below inputs as requested;
##ls -l /usr/lib/python3.5/csv.py-rw-r--r-- 1 root root 16138 Mar 19 11:28 /usr/lib/python3.5/csv.py
##sha256sum /usr/lib/python3.5/csv.pyfbed6ea919f70b4fd29608ab9620785304fc50a4b90cd2dd5d2363a0b81d9e81  /usr/lib/python3.5/csv.py
Find line aroud line#96
 79 class DictReader: 80     def __init__(self, f, fieldnames=None, restkey=None, restval=None, 81                  dialect="excel", *args, **kwds): 82         self._fieldnames = fieldnames   # list of keys for the dict 83         self.restkey = restkey          # key to catch long rows 84         self.restval = restval          # default value for short rows 85         self.reader = reader(f, dialect, *args, **kwds) 86         self.dialect = dialect 87         self.line_num = 0 88  89     def __iter__(self): 90         return self 91  92     @property 93     def fieldnames(self): 94         if self._fieldnames is None: 95             try: 96                 self._fieldnames = next(self.reader) 97             except StopIteration: 98                 pass 99         self.line_num = self.reader.line_num100         return self._fieldnames101 102     @fieldnames.setter103     def fieldnames(self, value):104         self._fieldnames = value 
I have two shadow server Parser and now have recorded different errors;
ShadowServer-Parser-2 
2020-03-19 12:31:07,318 - ShadowServer-Parser-2 - ERROR - Bot has found a problem.Traceback (most recent call last):  File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start    self.process()  File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process    for line in self.parse(report):  File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict    self.csv_fieldnames = csv_reader.fieldnames  File "/usr/lib/python3.5/csv.py", line 96, in fieldnames    self._fieldnames = next(self.reader)NameError: name 'csv' is not defined2020-03-19 12:31:07,319 - ShadowServer-Parser-2 - INFO - Dumping message to dump file.
ShadowServer-Parser
2020-03-19 12:37:58,573 - ShadowServer-Parser - ERROR - Bot has found a problem.Traceback (most recent call last):  File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start    self.process()  File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process    for line in self.parse(report):  File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict    self.csv_fieldnames = csv_reader.fieldnames  File "/usr/lib/python3.5/csv.py", line 96, in fieldnames    self._fieldnames = next(self.reader)_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?2020-03-19 12:37:58,574 - ShadowServer-Parser - INFO - Dumping message to dump file


Thank you very much,Patrick    On Thursday, March 19, 2020, 11:56:49 AM GMT+3, Bernhard Reiter <bernhard at intevation.de> wrote:  
 
 Am Mittwoch, 18. März 2020, 17:02:13 CET schrieben Sie:
>  libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64

This is the standard version, but still the backtrace looks strange,
especially the last bit:
  File "/usr/lib/python3.5/csv.py", line 96, in fieldnames
  self._fieldnames = next(self.reader, dialect=csv.excel_tab)
  NameError: name 'csv' is not defined

The second part is strange as the original sources [1] don't have the second
parameter. So we should double check.
Can you open the file `/usr/lib/python3.5/csv.py` and check line 96?

Please send the whole function around the lines here.
(Can you also post your mail in text-only format? This is something I can and 
many others can read much faster, and thus can answer you faster. :) Thanks! )

Can you also post the length, permissions and sha256 sum of the file?
Here for comparison the data for my file:
ls -l /usr/lib/python3.5/csv.py
-rw-r--r-- 1 root root 16128 Oct  8 16:38 /usr/lib/python3.5/csv.py
sha256sum /usr/lib/python3.5/csv.py
6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64  /usr/lib/
python3.5/csv.py

Best Regards,
Bernhard
[1] https://github.com/python/cpython/blob/master/Lib/csv.py

-- 
www.intevation.de/~bernhard  +49 541 33 508 3-3
Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998
Geschäftsführer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner-- 
Listen-Einstellungen:
 https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users
  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20200319/267da083/attachment.html>