[Intelmq-users] MAIL ATTACHMENT FETCHER FAILS TO READ/PARSE CSV FILE.
patric sungura
patsung87 at yahoo.com
Wed Mar 18 17:02:13 CET 2020
Dear Bernhard and Team;
>>version of intelmq
Also find below output;
# dpkg -S /usr/lib/python3.5/csv.pylibpython3.5-stdlib:amd64: /usr/lib/python3.5/csv.py
# dpkg -S /usr/lib/python3.5/csv.pylibpython3.5-stdlib:amd64: /usr/lib/python3.5/csv.py
# dpkg -l libpython3.5-stdlib | catDesired=Unknown/Install/Remove/Purge/Hold| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)||/ Name Version Architecture Description+++-=========================-======================-============-===============================================================================ii libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64 Interactive high-level object-oriented language (standard library, version 3.5)
This is now OK after I changed the pymisp version.# intelmqctl checkReading configuration files.Checking defaults configuration.Checking runtime configuration.Checking runtime and pipeline configuration.Orphaned queues found: 'Elasticsearch-Output-queue', 'feodo-tracker-browse-parser-queue', 'TCP-Output-queue'. Possible leftover from past reconfigurations without cleanup. Have a look at the FAQ at https://github.com/certtools/intelmq/blob/master/docs/FAQ.mdChecking harmonization configuration.Checking for bots.No state file found. Please call 'intelmqctl upgrade-config'.No issues found.
Find below collector and parser for shadow server;
"ShadowServer-Parser": { "parameters": { "time_format": null, "extra.file_name": "%Y-%m-%d-blacklist-tanzania-geo.csv", "feedname": "Blacklisted-IP", "overwrite": false }, "name": "ShadowServer", "group": "Parser", "module": "intelmq.bots.parsers.shadowserver.parser", "description": "ShadowServer Parser is a bot capable of parsing all shadowserver feeds, depending on configuration files. Parameter 'feedname' is used as identifier to chose the correct mapping.", "enabled": true, "run_mode": "continuous", "groupname": "parsers", "bot_id": "ShadowServer-Parser" }, "ShadowServer-Parser-2": { "parameters": { "time_format": null, "extra.file_name": "%Y-%m-%d-cisco_smart_install-tanzania-geo.csv", "feedname": "Accessible-Cisco-Smart-Install", "overwrite": false }, "name": "ShadowServer", "group": "Parser", "module": "intelmq.bots.parsers.shadowserver.parser", "description": "ShadowServer Parser is a bot capable of parsing all shadowserver feeds, depending on configuration files. Parameter 'feedname' is used as identifier to chose the correct mapping.", "enabled": true, "run_mode": "continuous", "groupname": "parsers", "bot_id": "ShadowServer-Parser-2" Mail-Attachment-Fetcher-Collector": { "parameters": { "extract_files": false, "attach_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]", "folder": "INBOX", "mail_host": "mail.xxxx.xxxx.xxxxx", "mail_password": "xxxxxxxxxxxxxx", "mail_ssl": true, "mail_user": "xxxxx.xxxx", "name": "via IMAP", "provider": "SHADOWSERVER", "rate_limit": 300, "subject_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]" }, "name": "Mail Attachment Fetcher", "group": "Collector", "module": "intelmq.bots.collectors.mail.collector_mail_attach", "description": "Monitor IMAP mailboxes and retrieve mail attachments", "enabled": true, "run_mode": "continuous", "groupname": "collectors", "bot_id": "Mail-Attachment-Fetcher-Collector" Mail-Attachment-Fetcher-Collector-2": { "parameters": { "extract_files": false, "attach_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]", "folder": "INBOX", "mail_host": "mail.xxxx.xxxx.xxx", "mail_password": "xxxxxx", "mail_ssl": true, "mail_user": "xxxxxxx", "name": "via IMAP", "provider": "SHADOWSERVER", "rate_limit": 300, "subject_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]" }, "name": "Mail Attachment Fetcher", "group": "Collector", "module": "intelmq.bots.collectors.mail.collector_mail_attach", "description": "Monitor IMAP mailboxes and retrieve mail attachments", "enabled": true, "run_mode": "continuous", "groupname": "collectors", "bot_id": "Mail-Attachment-Fetcher-Collector-2"
Am new user of Intelmq,may you please assist to guide me to finish this intelmq to shadow server integration.
Kr,Patrick
On Wednesday, March 18, 2020, 06:42:34 PM GMT+3, Bernhard Reiter <bernhard at intevation.de> wrote:
Dear Patric,
Am Mittwoch 18 März 2020 15:57:14 schrieb Sebastian Wagner:
> > Description: Ubuntu 16.04.6 LTS
> File "/usr/lib/python3.5/csv.py", line 96, in fieldnames
> self._fieldnames = next(self.reader, dialect=csv.excel_tab)
> NameError: name 'csv' is not defined
as I happen to have a comparable system here, I took a look in the file.
The code on line 96 looks different, so can you double check which version you
have, e.g.
dpkg -S /usr/lib/python3.5/csv.py
and then using the result to query it via dpkg -l,
for my system:
dpkg -l libpython3.5-stdlib | cat
ii libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64
BTW: Which version of intelmq and how did you install it?
Best,
Bernhard
--
www.intevation.de/~bernhard +49 541 33 508 3-3
Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998
Geschäftsführer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner--
Listen-Einstellungen:
https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20200318/5bedfa87/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20200318/5bedfa87/attachment.sig>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1584546595546blob.jpg
Type: image/png
Size: 13070 bytes
Desc: not available
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20200318/5bedfa87/attachment.png>
More information about the Intelmq-users
mailing list