Dear Team,Please assist me on mail attachment fetcher.Intelmq fails to read/parse csv file as per below erros from log and dump. ShadowServer-Parser-2.log
2020-03-17 16:10:42,064 - ShadowServer-Parser-2 - ERROR - Bot has found a problem.Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader, dialect=csv.excel_tab)NameError: name 'csv' is not defined2020-03-17 16:10:42,065 - ShadowServer-Parser-2 - INFO - Dumping message to dump file. ShadowServer-Parser-2.dump
"source_queue": "ShadowServer-Parser-2-queue", "traceback": [ "Traceback (most recent call last):\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start\n self.process()\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process\n for line in self.parse(report):\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict\n self.csv_fieldnames = csv_reader.fieldnames\n", " File "/usr/lib/python3.5/csv.py", line 96, in fieldnames\n self._fieldnames = next(self.reader, dialect=csv.excel_tab)\n", "NameError: name 'csv' is not defined\n" ] }
Hi, On 3/18/20 6:15 AM, patric sungura wrote:
2020-03-17 16:10:42,064 - ShadowServer-Parser-2 - ERROR - Bot has found a problem. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader, dialect=csv.excel_tab) NameError: name 'csv' is not defined
Which operating system are you using? The error happens inside python, so maybe the csv module is packaged extra in your distro (which would mean we should update the docs)
best regards, Sebastian
Dear Sebastian,I use ;No LSB modules are available.Distributor ID: UbuntuDescription: Ubuntu 16.04.6 LTSRelease: 16.04Codename: xenial
And whenever I do intelmqctl check ,below is the output;intelmqctl check Reading configuration files.Checking defaults configuration.Checking runtime configuration.Checking runtime and pipeline configuration.Orphaned queues found: 'TCP-Output-queue', 'feodo-tracker-browse-parser-queue', 'Elasticsearch-Output-queue'. Possible leftover from past reconfigurations without cleanup. Have a look at the FAQ at https://github.com/certtools/intelmq/blob/master/docs/FAQ.mdChecking harmonization configuration.Checking for bots.Traceback (most recent call last): File "/usr/local/bin/intelmqctl", line 11, in <module> sys.exit(main()) File "/usr/local/lib/python3.5/dist-packages/intelmq/bin/intelmqctl.py", line 1755, in main return x.run() File "/usr/local/lib/python3.5/dist-packages/intelmq/bin/intelmqctl.py", line 958, in run retval, results = args.func(**args_dict) File "/usr/local/lib/python3.5/dist-packages/intelmq/bin/intelmqctl.py", line 1447, in check bot_module = importlib.import_module(bot_config['module']) File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 986, in _gcd_import File "<frozen importlib._bootstrap>", line 969, in _find_and_load File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 673, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 665, in exec_module File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed File "/usr/local/lib/python3.5/dist-packages/intelmq/bots/collectors/misp/collector.py", line 19, in <module> from pymisp import PyMISP File "/usr/local/lib/python3.5/dist-packages/pymisp/__init__.py", line 26, in <module> from .abstract import AbstractMISP, MISPEncode, pymisp_json_default, MISPTag, Distribution, ThreatLevel, Analysis # noqa File "/usr/local/lib/python3.5/dist-packages/pymisp/abstract.py", line 113 self.__edited: bool = True # As we create a new object, we assume it is edited ^SyntaxError: invalid syntax
Can anyone assist to sort this out? as I highly need to integrate Intelmq with shadow server Any lead to accomplish this integration will be highly appreciated. kr,Patrick On Wednesday, March 18, 2020, 04:44:15 PM GMT+3, Sebastian Wagner wagner@cert.at wrote:
Hi, On 3/18/20 6:15 AM, patric sungura wrote:
2020-03-17 16:10:42,064 - ShadowServer-Parser-2 - ERROR - Bot has found a problem. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader, dialect=csv.excel_tab) NameError: name 'csv' is not defined Which operating system are you using? The error happens inside python, so maybe the csv module is packaged extra in your distro (which would mean we should update the docs)
best regards, Sebastian
-- // Sebastian Wagner wagner@cert.at - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg -- Listen-Einstellungen: https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users
Hi,
On 3/18/20 3:38 PM, patric sungura wrote:
Dear Sebastian, I use ; No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.6 LTS Release: 16.04 Codename: xenial
Weird. I currently have no explanation for that, as it's an error inside python:
File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader, dialect=csv.excel_tab) NameError: name 'csv' is not defined
The csv module can't access itself. I will try to reconstruct that with a local xenial installation.
And whenever I do *intelmqctl check ,below is the output;* *intelmqctl check*
[...] File "/usr/local/lib/python3.5/dist-packages/pymisp/abstract.py", line 113 self.__edited: bool = True # As we create a new object, we assume it is edited ^ SyntaxError: invalid syntax
Your installed pymisp version is not supported by your python version. Please execute `sudo pip3 install pymisp<=2.4.119.1` to install the latest pymisp version supported by your system.
Sebastian
Dear Patric,
Am Mittwoch 18 März 2020 15:57:14 schrieb Sebastian Wagner:
Description: Ubuntu 16.04.6 LTS
File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader, dialect=csv.excel_tab) NameError: name 'csv' is not defined
as I happen to have a comparable system here, I took a look in the file. The code on line 96 looks different, so can you double check which version you have, e.g. dpkg -S /usr/lib/python3.5/csv.py and then using the result to query it via dpkg -l, for my system:
dpkg -l libpython3.5-stdlib | cat ii libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64
BTW: Which version of intelmq and how did you install it?
Best, Bernhard
Dear Bernhard and Team;
version of intelmq
Also find below output;
# dpkg -S /usr/lib/python3.5/csv.pylibpython3.5-stdlib:amd64: /usr/lib/python3.5/csv.py # dpkg -S /usr/lib/python3.5/csv.pylibpython3.5-stdlib:amd64: /usr/lib/python3.5/csv.py # dpkg -l libpython3.5-stdlib | catDesired=Unknown/Install/Remove/Purge/Hold| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)||/ Name Version Architecture Description+++-=========================-======================-============-===============================================================================ii libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64 Interactive high-level object-oriented language (standard library, version 3.5) This is now OK after I changed the pymisp version.# intelmqctl checkReading configuration files.Checking defaults configuration.Checking runtime configuration.Checking runtime and pipeline configuration.Orphaned queues found: 'Elasticsearch-Output-queue', 'feodo-tracker-browse-parser-queue', 'TCP-Output-queue'. Possible leftover from past reconfigurations without cleanup. Have a look at the FAQ at https://github.com/certtools/intelmq/blob/master/docs/FAQ.mdChecking harmonization configuration.Checking for bots.No state file found. Please call 'intelmqctl upgrade-config'.No issues found.
Find below collector and parser for shadow server;
"ShadowServer-Parser": { "parameters": { "time_format": null, "extra.file_name": "%Y-%m-%d-blacklist-tanzania-geo.csv", "feedname": "Blacklisted-IP", "overwrite": false }, "name": "ShadowServer", "group": "Parser", "module": "intelmq.bots.parsers.shadowserver.parser", "description": "ShadowServer Parser is a bot capable of parsing all shadowserver feeds, depending on configuration files. Parameter 'feedname' is used as identifier to chose the correct mapping.", "enabled": true, "run_mode": "continuous", "groupname": "parsers", "bot_id": "ShadowServer-Parser" }, "ShadowServer-Parser-2": { "parameters": { "time_format": null, "extra.file_name": "%Y-%m-%d-cisco_smart_install-tanzania-geo.csv", "feedname": "Accessible-Cisco-Smart-Install", "overwrite": false }, "name": "ShadowServer", "group": "Parser", "module": "intelmq.bots.parsers.shadowserver.parser", "description": "ShadowServer Parser is a bot capable of parsing all shadowserver feeds, depending on configuration files. Parameter 'feedname' is used as identifier to chose the correct mapping.", "enabled": true, "run_mode": "continuous", "groupname": "parsers", "bot_id": "ShadowServer-Parser-2" Mail-Attachment-Fetcher-Collector": { "parameters": { "extract_files": false, "attach_regex": "[A-Za-z:0-9\\\\\\\\.\\\\\\\\_ \\\\\\\\[\\\\\\\\]", "folder": "INBOX", "mail_host": "mail.xxxx.xxxx.xxxxx", "mail_password": "xxxxxxxxxxxxxx", "mail_ssl": true, "mail_user": "xxxxx.xxxx", "name": "via IMAP", "provider": "SHADOWSERVER", "rate_limit": 300, "subject_regex": "[A-Za-z:0-9\\\\\\\\.\\\\\\\\_ \\\\\\\\[\\\\\\\\]" }, "name": "Mail Attachment Fetcher", "group": "Collector", "module": "intelmq.bots.collectors.mail.collector_mail_attach", "description": "Monitor IMAP mailboxes and retrieve mail attachments", "enabled": true, "run_mode": "continuous", "groupname": "collectors", "bot_id": "Mail-Attachment-Fetcher-Collector" Mail-Attachment-Fetcher-Collector-2": { "parameters": { "extract_files": false, "attach_regex": "[A-Za-z:0-9\\\\\\\\.\\\\\\\\_ \\\\\\\\[\\\\\\\\]", "folder": "INBOX", "mail_host": "mail.xxxx.xxxx.xxx", "mail_password": "xxxxxx", "mail_ssl": true, "mail_user": "xxxxxxx", "name": "via IMAP", "provider": "SHADOWSERVER", "rate_limit": 300, "subject_regex": "[A-Za-z:0-9\\\\\\\\.\\\\\\\\_ \\\\\\\\[\\\\\\\\]" }, "name": "Mail Attachment Fetcher", "group": "Collector", "module": "intelmq.bots.collectors.mail.collector_mail_attach", "description": "Monitor IMAP mailboxes and retrieve mail attachments", "enabled": true, "run_mode": "continuous", "groupname": "collectors", "bot_id": "Mail-Attachment-Fetcher-Collector-2" Am new user of Intelmq,may you please assist to guide me to finish this intelmq to shadow server integration. Kr,Patrick
On Wednesday, March 18, 2020, 06:42:34 PM GMT+3, Bernhard Reiter bernhard@intevation.de wrote:
Dear Patric,
Am Mittwoch 18 März 2020 15:57:14 schrieb Sebastian Wagner:
Description: Ubuntu 16.04.6 LTS
File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader, dialect=csv.excel_tab) NameError: name 'csv' is not defined
as I happen to have a comparable system here, I took a look in the file. The code on line 96 looks different, so can you double check which version you have, e.g. dpkg -S /usr/lib/python3.5/csv.py and then using the result to query it via dpkg -l, for my system: dpkg -l libpython3.5-stdlib | cat ii libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64
BTW: Which version of intelmq and how did you install it?
Best, Bernhard
Am Mittwoch, 18. März 2020, 17:02:13 CET schrieben Sie:
libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64
This is the standard version, but still the backtrace looks strange, especially the last bit: File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader, dialect=csv.excel_tab) NameError: name 'csv' is not defined
The second part is strange as the original sources [1] don't have the second parameter. So we should double check. Can you open the file `/usr/lib/python3.5/csv.py` and check line 96?
Please send the whole function around the lines here. (Can you also post your mail in text-only format? This is something I can and many others can read much faster, and thus can answer you faster. :) Thanks! )
Can you also post the length, permissions and sha256 sum of the file? Here for comparison the data for my file: ls -l /usr/lib/python3.5/csv.py -rw-r--r-- 1 root root 16128 Oct 8 16:38 /usr/lib/python3.5/csv.py sha256sum /usr/lib/python3.5/csv.py 6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64 /usr/lib/ python3.5/csv.py
Best Regards, Bernhard [1] https://github.com/python/cpython/blob/master/Lib/csv.py
Dear team,Find below inputs as requested; ##ls -l /usr/lib/python3.5/csv.py-rw-r--r-- 1 root root 16138 Mar 19 11:28 /usr/lib/python3.5/csv.py ##sha256sum /usr/lib/python3.5/csv.pyfbed6ea919f70b4fd29608ab9620785304fc50a4b90cd2dd5d2363a0b81d9e81 /usr/lib/python3.5/csv.py Find line aroud line#96 79 class DictReader: 80 def __init__(self, f, fieldnames=None, restkey=None, restval=None, 81 dialect="excel", *args, **kwds): 82 self._fieldnames = fieldnames # list of keys for the dict 83 self.restkey = restkey # key to catch long rows 84 self.restval = restval # default value for short rows 85 self.reader = reader(f, dialect, *args, **kwds) 86 self.dialect = dialect 87 self.line_num = 0 88 89 def __iter__(self): 90 return self 91 92 @property 93 def fieldnames(self): 94 if self._fieldnames is None: 95 try: 96 self._fieldnames = next(self.reader) 97 except StopIteration: 98 pass 99 self.line_num = self.reader.line_num100 return self._fieldnames101 102 @fieldnames.setter103 def fieldnames(self, value):104 self._fieldnames = value I have two shadow server Parser and now have recorded different errors; ShadowServer-Parser-2 2020-03-19 12:31:07,318 - ShadowServer-Parser-2 - ERROR - Bot has found a problem.Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader)NameError: name 'csv' is not defined2020-03-19 12:31:07,319 - ShadowServer-Parser-2 - INFO - Dumping message to dump file. ShadowServer-Parser 2020-03-19 12:37:58,573 - ShadowServer-Parser - ERROR - Bot has found a problem.Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader)_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?2020-03-19 12:37:58,574 - ShadowServer-Parser - INFO - Dumping message to dump file
Thank you very much,Patrick On Thursday, March 19, 2020, 11:56:49 AM GMT+3, Bernhard Reiter bernhard@intevation.de wrote:
Am Mittwoch, 18. März 2020, 17:02:13 CET schrieben Sie:
libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64
This is the standard version, but still the backtrace looks strange, especially the last bit: File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader, dialect=csv.excel_tab) NameError: name 'csv' is not defined
The second part is strange as the original sources [1] don't have the second parameter. So we should double check. Can you open the file `/usr/lib/python3.5/csv.py` and check line 96?
Please send the whole function around the lines here. (Can you also post your mail in text-only format? This is something I can and many others can read much faster, and thus can answer you faster. :) Thanks! )
Can you also post the length, permissions and sha256 sum of the file? Here for comparison the data for my file: ls -l /usr/lib/python3.5/csv.py -rw-r--r-- 1 root root 16128 Oct 8 16:38 /usr/lib/python3.5/csv.py sha256sum /usr/lib/python3.5/csv.py 6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64 /usr/lib/ python3.5/csv.py
Best Regards, Bernhard [1] https://github.com/python/cpython/blob/master/Lib/csv.py
Hi,
On a fresh, virgin and up-to-date xenial installation I get the same results as Bernhard, which differ from yours. I suggest to re-install the affected python3 package on your system (or, maybe just the whole python3-stack):
|sudo apt-get install --reinstall libpython3.5-stdlib And then show the output of the commands again. Sebsatian |
On 3/19/20 10:48 AM, patric sungura wrote:
Dear team, Find below inputs as requested;
*##ls -l /usr/lib/python3.5/csv.py* -rw-r--r-- 1 root root 16138 Mar 19 11:28 /usr/lib/python3.5/csv.py
*##sha256sum /usr/lib/python3.5/csv.py* fbed6ea919f70b4fd29608ab9620785304fc50a4b90cd2dd5d2363a0b81d9e81 /usr/lib/python3.5/csv.py
*Find line aroud line#96*
79 class DictReader: 80 def __init__(self, f, fieldnames=None, restkey=None, restval=None, 81 dialect="excel", *args, **kwds): 82 self._fieldnames = fieldnames # list of keys for the dict 83 self.restkey = restkey # key to catch long rows 84 self.restval = restval # default value for short rows 85 self.reader = reader(f, dialect, *args, **kwds) 86 self.dialect = dialect 87 self.line_num = 0 88 89 def __iter__(self): 90 return self 91 92 @property 93 def fieldnames(self): 94 if self._fieldnames is None: 95 try: * 96 self._fieldnames = next(self.reader)* 97 except StopIteration: 98 pass 99 self.line_num = self.reader.line_num 100 return self._fieldnames 101 102 @fieldnames.setter 103 def fieldnames(self, value): 104 self._fieldnames = value
I have two shadow server Parser and now have recorded different errors;
*ShadowServer-Parser-2 * 2020-03-19 12:31:07,318 - ShadowServer-Parser-2 - ERROR - Bot has found a problem. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader) *NameError: name 'csv' is not defined* 2020-03-19 12:31:07,319 - ShadowServer-Parser-2 - INFO - Dumping message to dump file.
*ShadowServer-Parser* 2020-03-19 12:37:58,573 - ShadowServer-Parser - ERROR - Bot has found a problem. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader) _*csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?* 2020-03-19 12:37:58,574 - ShadowServer-Parser - INFO - Dumping message to dump file
Thank you very much, Patrick On Thursday, March 19, 2020, 11:56:49 AM GMT+3, Bernhard Reiter bernhard@intevation.de wrote:
Am Mittwoch, 18. März 2020, 17:02:13 CET schrieben Sie:
libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64
This is the standard version, but still the backtrace looks strange, especially the last bit: File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader, dialect=csv.excel_tab) NameError: name 'csv' is not defined
The second part is strange as the original sources [1] don't have the second parameter. So we should double check. Can you open the file `/usr/lib/python3.5/csv.py` and check line 96?
Please send the whole function around the lines here. (Can you also post your mail in text-only format? This is something I can and many others can read much faster, and thus can answer you faster. :) Thanks! )
Can you also post the length, permissions and sha256 sum of the file? Here for comparison the data for my file: ls -l /usr/lib/python3.5/csv.py -rw-r--r-- 1 root root 16128 Oct 8 16:38 /usr/lib/python3.5/csv.py sha256sum /usr/lib/python3.5/csv.py 6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64 /usr/lib/ python3.5/csv.py
Best Regards, Bernhard [1] https://github.com/python/cpython/blob/master/Lib/csv.py
-- www.intevation.de/~bernhard +49 541 33 508 3-3 Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998 Geschäftsführer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -- Listen-Einstellungen: https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users
Hi Sebastian,Find output below ;sha256sum /usr/lib/python3.5/csv.py6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64 /usr/lib/python3.5/csv.py
Below is error I get after I parse email; 2020-03-19 14:02:16,409 - ShadowServer-Parser - ERROR - Failed to parse line.Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 951, in process events = list(filter(bool, value)) File "/usr/local/lib/python3.5/dist-packages/intelmq/bots/parsers/shadowserver/parser.py", line 111, in parse_line ' format or misconfiguration.'.format(shadowkey, self.feedname))ValueError: Required column 'timestamp' not found in feed 'Blacklisted-IP'. Possible change in data format or misconfiguration.2020-03-19 14:02:16,410 - ShadowServer-Parser - INFO - Dumping message to dump file.2020-03-19 14:02:16,514 - ShadowServer-Parser - INFO - Dumping message to dump file.2020-03-19 14:02:16,612 - ShadowServer-Parser - INFO - Sent 0 events and found 2 problem(s).
From Dump; "source_queue": "ShadowServer-Parser-queue", "traceback": "Traceback (most recent call last):\n File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 951, in process\n events = list(filter(bool, value))\n File "/usr/local/lib/python3.5/dist-packages/intelmq/bots/parsers/shadowserver/parser.py", line 111, in parse_line\n ' format or misconfiguration.'.format(shadowkey, self.feedname))\nValueError: Required column 'timestamp' not found in feed 'Blacklisted-IP'. Possible change in data format or misconfiguration.\n" }
Thank you,Patrick On Thursday, March 19, 2020, 01:48:02 PM GMT+3, Sebastian Wagner wagner@cert.at wrote:
Hi,
On a fresh, virgin and up-to-date xenial installation I get the same results as Bernhard, which differ from yours. I suggest to re-install the affected python3 package on your system (or, maybe just the whole python3-stack): sudo apt-get install --reinstall libpython3.5-stdlib
And then show the output of the commands again.
Sebsatian On 3/19/20 10:48 AM, patric sungura wrote:
Dear team, Find below inputs as requested; ##ls -l /usr/lib/python3.5/csv.py -rw-r--r-- 1 root root 16138 Mar 19 11:28 /usr/lib/python3.5/csv.py ##sha256sum /usr/lib/python3.5/csv.py fbed6ea919f70b4fd29608ab9620785304fc50a4b90cd2dd5d2363a0b81d9e81 /usr/lib/python3.5/csv.py Find line aroud line#96 79 class DictReader: 80 def __init__(self, f, fieldnames=None, restkey=None, restval=None, 81 dialect="excel", *args, **kwds): 82 self._fieldnames = fieldnames # list of keys for the dict 83 self.restkey = restkey # key to catch long rows 84 self.restval = restval # default value for short rows 85 self.reader = reader(f, dialect, *args, **kwds) 86 self.dialect = dialect 87 self.line_num = 0 88 89 def __iter__(self): 90 return self 91 92 @property 93 def fieldnames(self): 94 if self._fieldnames is None: 95 try: 96 self._fieldnames = next(self.reader) 97 except StopIteration: 98 pass 99 self.line_num = self.reader.line_num 100 return self._fieldnames 101 102 @fieldnames.setter 103 def fieldnames(self, value): 104 self._fieldnames = value I have two shadow server Parser and now have recorded different errors; ShadowServer-Parser-2 2020-03-19 12:31:07,318 - ShadowServer-Parser-2 - ERROR - Bot has found a problem. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader) NameError: name 'csv' is not defined 2020-03-19 12:31:07,319 - ShadowServer-Parser-2 - INFO - Dumping message to dump file. ShadowServer-Parser 2020-03-19 12:37:58,573 - ShadowServer-Parser - ERROR - Bot has found a problem. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader) _csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode? 2020-03-19 12:37:58,574 - ShadowServer-Parser - INFO - Dumping message to dump file
Thank you very much, Patrick On Thursday, March 19, 2020, 11:56:49 AM GMT+3, Bernhard Reiter bernhard@intevation.de wrote:
Am Mittwoch, 18. März 2020, 17:02:13 CET schrieben Sie:
libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64
This is the standard version, but still the backtrace looks strange, especially the last bit: File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader, dialect=csv.excel_tab) NameError: name 'csv' is not defined
The second part is strange as the original sources [1] don't have the second parameter. So we should double check. Can you open the file `/usr/lib/python3.5/csv.py` and check line 96?
Please send the whole function around the lines here. (Can you also post your mail in text-only format? This is something I can and many others can read much faster, and thus can answer you faster. :) Thanks! )
Can you also post the length, permissions and sha256 sum of the file? Here for comparison the data for my file: ls -l /usr/lib/python3.5/csv.py -rw-r--r-- 1 root root 16128 Oct 8 16:38 /usr/lib/python3.5/csv.py sha256sum /usr/lib/python3.5/csv.py 6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64 /usr/lib/ python3.5/csv.py
Best Regards, Bernhard [1] https://github.com/python/cpython/blob/master/Lib/csv.py
-- www.intevation.de/~bernhard +49 541 33 508 3-3 Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998 Geschäftsführer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -- Listen-Einstellungen: https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users
-- // Sebastian Wagner wagner@cert.at - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg -- Listen-Einstellungen: https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users
Hi Team,Now I get this error on parser after the BOT read the email;
2020-03-19 16:14:27,444 - ShadowServer-Parser - ERROR - Bot has found a problem.Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader)_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?2020-03-19 16:14:27,445 - ShadowServer-Parser - INFO - Dumping message to dump file.
"traceback": [ "Traceback (most recent call last):\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start\n self.process()\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process\n for line in self.parse(report):\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict\n self.csv_fieldnames = csv_reader.fieldnames\n", " File "/usr/lib/python3.5/csv.py", line 96, in fieldnames\n self._fieldnames = next(self.reader)\n", "_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?\n" ] }
kr,Patrick On Thursday, March 19, 2020, 02:27:56 PM GMT+3, patric sungura patsung87@yahoo.com wrote:
Hi Sebastian,Find output below ;sha256sum /usr/lib/python3.5/csv.py6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64 /usr/lib/python3.5/csv.py
Below is error I get after I parse email; 2020-03-19 14:02:16,409 - ShadowServer-Parser - ERROR - Failed to parse line.Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 951, in process events = list(filter(bool, value)) File "/usr/local/lib/python3.5/dist-packages/intelmq/bots/parsers/shadowserver/parser.py", line 111, in parse_line ' format or misconfiguration.'.format(shadowkey, self.feedname))ValueError: Required column 'timestamp' not found in feed 'Blacklisted-IP'. Possible change in data format or misconfiguration.2020-03-19 14:02:16,410 - ShadowServer-Parser - INFO - Dumping message to dump file.2020-03-19 14:02:16,514 - ShadowServer-Parser - INFO - Dumping message to dump file.2020-03-19 14:02:16,612 - ShadowServer-Parser - INFO - Sent 0 events and found 2 problem(s).
From Dump; "source_queue": "ShadowServer-Parser-queue", "traceback": "Traceback (most recent call last):\n File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 951, in process\n events = list(filter(bool, value))\n File "/usr/local/lib/python3.5/dist-packages/intelmq/bots/parsers/shadowserver/parser.py", line 111, in parse_line\n ' format or misconfiguration.'.format(shadowkey, self.feedname))\nValueError: Required column 'timestamp' not found in feed 'Blacklisted-IP'. Possible change in data format or misconfiguration.\n" }
Thank you,Patrick On Thursday, March 19, 2020, 01:48:02 PM GMT+3, Sebastian Wagner wagner@cert.at wrote:
Hi,
On a fresh, virgin and up-to-date xenial installation I get the same results as Bernhard, which differ from yours. I suggest to re-install the affected python3 package on your system (or, maybe just the whole python3-stack): sudo apt-get install --reinstall libpython3.5-stdlib
And then show the output of the commands again.
Sebsatian On 3/19/20 10:48 AM, patric sungura wrote:
Dear team, Find below inputs as requested; ##ls -l /usr/lib/python3.5/csv.py -rw-r--r-- 1 root root 16138 Mar 19 11:28 /usr/lib/python3.5/csv.py ##sha256sum /usr/lib/python3.5/csv.py fbed6ea919f70b4fd29608ab9620785304fc50a4b90cd2dd5d2363a0b81d9e81 /usr/lib/python3.5/csv.py Find line aroud line#96 79 class DictReader: 80 def __init__(self, f, fieldnames=None, restkey=None, restval=None, 81 dialect="excel", *args, **kwds): 82 self._fieldnames = fieldnames # list of keys for the dict 83 self.restkey = restkey # key to catch long rows 84 self.restval = restval # default value for short rows 85 self.reader = reader(f, dialect, *args, **kwds) 86 self.dialect = dialect 87 self.line_num = 0 88 89 def __iter__(self): 90 return self 91 92 @property 93 def fieldnames(self): 94 if self._fieldnames is None: 95 try: 96 self._fieldnames = next(self.reader) 97 except StopIteration: 98 pass 99 self.line_num = self.reader.line_num 100 return self._fieldnames 101 102 @fieldnames.setter 103 def fieldnames(self, value): 104 self._fieldnames = value I have two shadow server Parser and now have recorded different errors; ShadowServer-Parser-2 2020-03-19 12:31:07,318 - ShadowServer-Parser-2 - ERROR - Bot has found a problem. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader) NameError: name 'csv' is not defined 2020-03-19 12:31:07,319 - ShadowServer-Parser-2 - INFO - Dumping message to dump file. ShadowServer-Parser 2020-03-19 12:37:58,573 - ShadowServer-Parser - ERROR - Bot has found a problem. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader) _csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode? 2020-03-19 12:37:58,574 - ShadowServer-Parser - INFO - Dumping message to dump file
Thank you very much, Patrick On Thursday, March 19, 2020, 11:56:49 AM GMT+3, Bernhard Reiter bernhard@intevation.de wrote:
Am Mittwoch, 18. März 2020, 17:02:13 CET schrieben Sie:
libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64
This is the standard version, but still the backtrace looks strange, especially the last bit: File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader, dialect=csv.excel_tab) NameError: name 'csv' is not defined
The second part is strange as the original sources [1] don't have the second parameter. So we should double check. Can you open the file `/usr/lib/python3.5/csv.py` and check line 96?
Please send the whole function around the lines here. (Can you also post your mail in text-only format? This is something I can and many others can read much faster, and thus can answer you faster. :) Thanks! )
Can you also post the length, permissions and sha256 sum of the file? Here for comparison the data for my file: ls -l /usr/lib/python3.5/csv.py -rw-r--r-- 1 root root 16128 Oct 8 16:38 /usr/lib/python3.5/csv.py sha256sum /usr/lib/python3.5/csv.py 6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64 /usr/lib/ python3.5/csv.py
Best Regards, Bernhard [1] https://github.com/python/cpython/blob/master/Lib/csv.py
-- www.intevation.de/~bernhard +49 541 33 508 3-3 Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998 Geschäftsführer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -- Listen-Einstellungen: https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users
-- // Sebastian Wagner wagner@cert.at - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg -- Listen-Einstellungen: https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users
Hi,
On 3/19/20 12:27 PM, patric sungura wrote:
Find output below ; sha256sum /usr/lib/python3.5/csv.py 6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64 /usr/lib/python3.5/csv.py
Great, so we now fixed the Python problem :)
On 3/19/20 2:29 PM, patric sungura wrote:
2020-03-19 16:14:27,444 - ShadowServer-Parser - ERROR - Bot has found a problem. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader) *_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?*
In the previous mail the error was different - so what changed?
2020-03-19 14:02:16,409 - ShadowServer-Parser - ERROR - Failed to parse line. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 951, in process events = list(filter(bool, value)) File "/usr/local/lib/python3.5/dist-packages/intelmq/bots/parsers/shadowserver/parser.py", line 111, in parse_line ' format or misconfiguration.'.format(shadowkey, self.feedname)) *ValueError: Required column 'timestamp' not found in feed 'Blacklisted-IP'. Possible change in data format or misconfiguration.*
Sebastian
Sebastian,Only python fix.Find below error at different time . Current I receive csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode? @13:14 GMT 2020-03-19 16:14:27,444 - ShadowServer-Parser - ERROR - Bot has found a problem.Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader)_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?2020-03-19 16:14:27,445 - ShadowServer-Parser - INFO - Dumping message to dump file.
"traceback": [ "Traceback (most recent call last):\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start\n self.process()\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process\n for line in self.parse(report):\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict\n self.csv_fieldnames = csv_reader.fieldnames\n", " File "/usr/lib/python3.5/csv.py", line 96, in fieldnames\n self._fieldnames = next(self.reader)\n", "_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?\n" ] }
@11:02 GMT2020-03-19 14:02:16,409 - ShadowServer-Parser - ERROR - Failed to parse line.Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 951, in process events = list(filter(bool, value)) File "/usr/local/lib/python3.5/dist-packages/intelmq/bots/parsers/shadowserver/parser.py", line 111, in parse_line ' format or misconfiguration.'.format(shadowkey, self.feedname))ValueError: Required column 'timestamp' not found in feed 'Blacklisted-IP'. Possible change in data format or misconfiguration.2020-03-19 14:02:16,410 - ShadowServer-Parser - INFO - Dumping message to dump file.2020-03-19 14:02:16,514 - ShadowServer-Parser - INFO - Dumping message to dump file.2020-03-19 14:02:16,612 - ShadowServer-Parser - INFO - Sent 0 events and found 2 problem(s).
From Dump; "source_queue": "ShadowServer-Parser-queue", "traceback": "Traceback (most recent call last):\n File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 951, in process\n events = list(filter(bool, value))\n File "/usr/local/lib/python3.5/dist-packages/intelmq/bots/parsers/shadowserver/parser.py", line 111, in parse_line\n ' format or misconfiguration.'.format(shadowkey, self.feedname))\nValueError: Required column 'timestamp' not found in feed 'Blacklisted-IP'. Possible change in data format or misconfiguration.\n" }Your assistance is highly requested.
Thank you,Patrick On Thursday, March 19, 2020, 04:33:45 PM GMT+3, Sebastian Wagner wagner@cert.at wrote:
Hi, On 3/19/20 12:27 PM, patric sungura wrote:
Find output below ; sha256sum /usr/lib/python3.5/csv.py 6d96e56ec22d1603ffed0f8d20f5b5090fb6ae4c7417126d09afc45447b34a64 /usr/lib/python3.5/csv.py
Great, so we now fixed the Python problem :)
On 3/19/20 2:29 PM, patric sungura wrote:
2020-03-19 16:14:27,444 - ShadowServer-Parser - ERROR - Bot has found a problem. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict self.csv_fieldnames = csv_reader.fieldnames File "/usr/lib/python3.5/csv.py", line 96, in fieldnames self._fieldnames = next(self.reader) _csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode? In the previous mail the error was different - so what changed? 2020-03-19 14:02:16,409 - ShadowServer-Parser - ERROR - Failed to parse line. Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 951, in process events = list(filter(bool, value)) File"/usr/local/lib/python3.5/dist-packages/intelmq/bots/parsers/shadowserver/parser.py", line 111, in parse_line ' format or misconfiguration.'.format(shadowkey, self.feedname)) ValueError: Required column 'timestamp' not found in feed 'Blacklisted-IP'. Possible change in data format or misconfiguration. Sebastian -- // Sebastian Wagner wagner@cert.at - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg -- Listen-Einstellungen: https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users
Patric,
what is your locale? (Test with entering `locale`)
Bernhard
Hi Made this time conversion to GMT for understanding.I belong to GMT+3. Let me know if i replied it correctly. kr,Patrick On Thursday, March 19, 2020, 04:56:10 PM GMT+3, Bernhard Reiter bernhard@intevation.de wrote:
Patric,
what is your locale? (Test with entering `locale`)
Bernhard
Hi Team,Find below; LANG=en_US.UTF-8LANGUAGE=LC_CTYPE="en_US.UTF-8"LC_NUMERIC="en_US.UTF-8"LC_TIME="en_US.UTF-8"LC_COLLATE="en_US.UTF-8"LC_MONETARY="en_US.UTF-8"LC_MESSAGES="en_US.UTF-8"LC_PAPER="en_US.UTF-8"LC_NAME="en_US.UTF-8"LC_ADDRESS="en_US.UTF-8"LC_TELEPHONE="en_US.UTF-8"LC_MEASUREMENT="en_US.UTF-8"LC_IDENTIFICATION="en_US.UTF-8"LC_ALL=
kr,Patrick On Thursday, March 19, 2020, 05:06:56 PM GMT+3, patric sungura patsung87@yahoo.com wrote:
Hi Made this time conversion to GMT for understanding.I belong to GMT+3. Let me know if i replied it correctly. kr,Patrick On Thursday, March 19, 2020, 04:56:10 PM GMT+3, Bernhard Reiter bernhard@intevation.de wrote:
Patric,
what is your locale? (Test with entering `locale`)
Bernhard
Team,May you please assist to share inputs to fix this,I have created new one collector bot and below are errors; Accessible SMB Service Report:\r\n 2020-03-17", "feed.provider": "SHADOWSERVER", "extra.email_message_id": "20200318131314.MluRQ9xWN%autoreports@shadowserver.org", "feed.name": "via IMAP", "feed.accuracy": 100.0}", "source_queue": "ShadowServer-Parser-3-queue", "traceback": [ "Traceback (most recent call last):\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start\n self.process()\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process\n for line in self.parse(report):\n", " File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 884, in parse_csv_dict\n self.csv_fieldnames = csv_reader.fieldnames\n", " File "/usr/lib/python3.5/csv.py", line 96, in fieldnames\n self._fieldnames = next(self.reader)\n", "_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?\n" ] }
2020-03-19 18:34:27,164 - ShadowServer-Parser-3 - ERROR - Bot has found a problem.Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 267, in start self.process() File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 941, in process for line in self.parse(report): File "/usr/local/lib/python3.5/dist-packages/intelmq/lib/bot.py", line 886, in parse_csv_dict for line in csv_reader: File "/usr/lib/python3.5/csv.py", line 110, in __next__ row = next(self.reader)_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?2020-03-19 18:34:27,165 - ShadowServer-Parser-3 - INFO - Dumping message to dump file.
kr,Patrick On Thursday, March 19, 2020, 05:19:36 PM GMT+3, patric sungura patsung87@yahoo.com wrote:
Hi Team,Find below; LANG=en_US.UTF-8LANGUAGE=LC_CTYPE="en_US.UTF-8"LC_NUMERIC="en_US.UTF-8"LC_TIME="en_US.UTF-8"LC_COLLATE="en_US.UTF-8"LC_MONETARY="en_US.UTF-8"LC_MESSAGES="en_US.UTF-8"LC_PAPER="en_US.UTF-8"LC_NAME="en_US.UTF-8"LC_ADDRESS="en_US.UTF-8"LC_TELEPHONE="en_US.UTF-8"LC_MEASUREMENT="en_US.UTF-8"LC_IDENTIFICATION="en_US.UTF-8"LC_ALL=
kr,Patrick On Thursday, March 19, 2020, 05:06:56 PM GMT+3, patric sungura patsung87@yahoo.com wrote:
Hi Made this time conversion to GMT for understanding.I belong to GMT+3. Let me know if i replied it correctly. kr,Patrick On Thursday, March 19, 2020, 04:56:10 PM GMT+3, Bernhard Reiter bernhard@intevation.de wrote:
Patric,
what is your locale? (Test with entering `locale`)
Bernhard
