[Intelmq-users] MAIL ATTACHMENT FETCHER FAILS TO READ/PARSE CSV FILE.

patric sungura patsung87 at yahoo.com
Wed Mar 18 17:02:13 CET 2020


 Dear Bernhard and Team;
>>version of intelmq 


Also find below output;

#  dpkg -S /usr/lib/python3.5/csv.pylibpython3.5-stdlib:amd64: /usr/lib/python3.5/csv.py
#  dpkg -S /usr/lib/python3.5/csv.pylibpython3.5-stdlib:amd64: /usr/lib/python3.5/csv.py
# dpkg -l libpython3.5-stdlib | catDesired=Unknown/Install/Remove/Purge/Hold| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)||/ Name                      Version                Architecture Description+++-=========================-======================-============-===============================================================================ii  libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64        Interactive high-level object-oriented language (standard library, version 3.5)
This is now OK after I changed the pymisp version.# intelmqctl checkReading configuration files.Checking defaults configuration.Checking runtime configuration.Checking runtime and pipeline configuration.Orphaned queues found: 'Elasticsearch-Output-queue', 'feodo-tracker-browse-parser-queue', 'TCP-Output-queue'. Possible leftover from past reconfigurations without cleanup. Have a look at the FAQ at https://github.com/certtools/intelmq/blob/master/docs/FAQ.mdChecking harmonization configuration.Checking for bots.No state file found. Please call 'intelmqctl upgrade-config'.No issues found.


Find below collector and parser for shadow server;

"ShadowServer-Parser": {        "parameters": {            "time_format": null,            "extra.file_name": "%Y-%m-%d-blacklist-tanzania-geo.csv",            "feedname": "Blacklisted-IP",            "overwrite": false        },        "name": "ShadowServer",        "group": "Parser",        "module": "intelmq.bots.parsers.shadowserver.parser",        "description": "ShadowServer Parser is a bot capable of parsing all shadowserver feeds, depending on configuration files. Parameter 'feedname' is used as identifier to chose the correct mapping.",        "enabled": true,        "run_mode": "continuous",        "groupname": "parsers",        "bot_id": "ShadowServer-Parser"    },    "ShadowServer-Parser-2": {        "parameters": {            "time_format": null,            "extra.file_name": "%Y-%m-%d-cisco_smart_install-tanzania-geo.csv",            "feedname": "Accessible-Cisco-Smart-Install",            "overwrite": false        },        "name": "ShadowServer",        "group": "Parser",        "module": "intelmq.bots.parsers.shadowserver.parser",        "description": "ShadowServer Parser is a bot capable of parsing all shadowserver feeds, depending on configuration files. Parameter 'feedname' is used as identifier to chose the correct mapping.",        "enabled": true,        "run_mode": "continuous",        "groupname": "parsers",        "bot_id": "ShadowServer-Parser-2"                                        Mail-Attachment-Fetcher-Collector": {        "parameters": {            "extract_files": false,            "attach_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]",            "folder": "INBOX",            "mail_host": "mail.xxxx.xxxx.xxxxx",            "mail_password": "xxxxxxxxxxxxxx",            "mail_ssl": true,            "mail_user": "xxxxx.xxxx",            "name": "via IMAP",            "provider": "SHADOWSERVER",            "rate_limit": 300,            "subject_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]"        },        "name": "Mail Attachment Fetcher",        "group": "Collector",        "module": "intelmq.bots.collectors.mail.collector_mail_attach",        "description": "Monitor IMAP mailboxes and retrieve mail attachments",        "enabled": true,        "run_mode": "continuous",        "groupname": "collectors",        "bot_id": "Mail-Attachment-Fetcher-Collector"                        Mail-Attachment-Fetcher-Collector-2": {        "parameters": {            "extract_files": false,            "attach_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]",            "folder": "INBOX",            "mail_host": "mail.xxxx.xxxx.xxx",            "mail_password": "xxxxxx",            "mail_ssl": true,            "mail_user": "xxxxxxx",            "name": "via IMAP",            "provider": "SHADOWSERVER",            "rate_limit": 300,            "subject_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]"        },        "name": "Mail Attachment Fetcher",        "group": "Collector",        "module": "intelmq.bots.collectors.mail.collector_mail_attach",        "description": "Monitor IMAP mailboxes and retrieve mail attachments",        "enabled": true,        "run_mode": "continuous",        "groupname": "collectors",        "bot_id": "Mail-Attachment-Fetcher-Collector-2"
Am new user of Intelmq,may you please assist to guide me to finish this intelmq to shadow server integration.
Kr,Patrick

    On Wednesday, March 18, 2020, 06:42:34 PM GMT+3, Bernhard Reiter <bernhard at intevation.de> wrote:  
 
 Dear Patric,

Am Mittwoch 18 März 2020 15:57:14 schrieb Sebastian Wagner:
> > Description:    Ubuntu 16.04.6 LTS
>   File "/usr/lib/python3.5/csv.py", line 96, in fieldnames 
>     self._fieldnames = next(self.reader, dialect=csv.excel_tab)
> NameError: name 'csv' is not defined

as I happen to have a comparable system here, I took a look in the file.
The code on line 96 looks different, so can you double check which version you 
have, e.g.
  dpkg -S /usr/lib/python3.5/csv.py
and then using the result to query it via dpkg -l,
for my system:
  
dpkg -l libpython3.5-stdlib | cat
ii  libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64

BTW: Which version of intelmq and how did you install it?

Best,
Bernhard

-- 
www.intevation.de/~bernhard   +49 541 33 508 3-3
Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998
Geschäftsführer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner-- 
Listen-Einstellungen:
 https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-users
  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20200318/5bedfa87/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20200318/5bedfa87/attachment.sig>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1584546595546blob.jpg
Type: image/png
Size: 13070 bytes
Desc: not available
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20200318/5bedfa87/attachment.png>


More information about the Intelmq-users mailing list