Dear Bernhard and Team;

>>version of intelmq 

Inline image

Also find below output;


dpkg -S /usr/lib/python3.5/csv.py
libpython3.5-stdlib:amd64: /usr/lib/python3.5/csv.py

dpkg -S /usr/lib/python3.5/csv.py
libpython3.5-stdlib:amd64: /usr/lib/python3.5/csv.py

# dpkg -l libpython3.5-stdlib | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                      Version                Architecture Description
+++-=========================-======================-============-===============================================================================
ii  libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64        Interactive high-level object-oriented language (standard library, version 3.5)

This is now OK after I changed the pymisp version.
# intelmqctl check
Reading configuration files.
Checking defaults configuration.
Checking runtime configuration.
Checking runtime and pipeline configuration.
Orphaned queues found: 'Elasticsearch-Output-queue', 'feodo-tracker-browse-parser-queue', 'TCP-Output-queue'. Possible leftover from past reconfigurations without cleanup. Have a look at the FAQ at https://github.com/certtools/intelmq/blob/master/docs/FAQ.md
Checking harmonization configuration.
Checking for bots.
No state file found. Please call 'intelmqctl upgrade-config'.
No issues found.



Find below collector and parser for shadow server;


"ShadowServer-Parser": {
        "parameters": {
            "time_format": null,
            "extra.file_name": "%Y-%m-%d-blacklist-tanzania-geo.csv",
            "feedname": "Blacklisted-IP",
            "overwrite": false
        },
        "name": "ShadowServer",
        "group": "Parser",
        "module": "intelmq.bots.parsers.shadowserver.parser",
        "description": "ShadowServer Parser is a bot capable of parsing all shadowserver feeds, depending on configuration files. Parameter 'feedname' is used as identifier to chose the correct mapping.",
        "enabled": true,
        "run_mode": "continuous",
        "groupname": "parsers",
        "bot_id": "ShadowServer-Parser"
    },
    "ShadowServer-Parser-2": {
        "parameters": {
            "time_format": null,
            "extra.file_name": "%Y-%m-%d-cisco_smart_install-tanzania-geo.csv",
            "feedname": "Accessible-Cisco-Smart-Install",
            "overwrite": false
        },
        "name": "ShadowServer",
        "group": "Parser",
        "module": "intelmq.bots.parsers.shadowserver.parser",
        "description": "ShadowServer Parser is a bot capable of parsing all shadowserver feeds, depending on configuration files. Parameter 'feedname' is used as identifier to chose the correct mapping.",
        "enabled": true,
        "run_mode": "continuous",
        "groupname": "parsers",
        "bot_id": "ShadowServer-Parser-2"
        
        
        
        
        Mail-Attachment-Fetcher-Collector": {
        "parameters": {
            "extract_files": false,
            "attach_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]",
            "folder": "INBOX",
            "mail_host": "mail.xxxx.xxxx.xxxxx",
            "mail_password": "xxxxxxxxxxxxxx",
            "mail_ssl": true,
            "mail_user": "xxxxx.xxxx",
            "name": "via IMAP",
            "provider": "SHADOWSERVER",
            "rate_limit": 300,
            "subject_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]"
        },
        "name": "Mail Attachment Fetcher",
        "group": "Collector",
        "module": "intelmq.bots.collectors.mail.collector_mail_attach",
        "description": "Monitor IMAP mailboxes and retrieve mail attachments",
        "enabled": true,
        "run_mode": "continuous",
        "groupname": "collectors",
        "bot_id": "Mail-Attachment-Fetcher-Collector"
        
        
        Mail-Attachment-Fetcher-Collector-2": {
        "parameters": {
            "extract_files": false,
            "attach_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]",
            "folder": "INBOX",
            "mail_host": "mail.xxxx.xxxx.xxx",
            "mail_password": "xxxxxx",
            "mail_ssl": true,
            "mail_user": "xxxxxxx",
            "name": "via IMAP",
            "provider": "SHADOWSERVER",
            "rate_limit": 300,
            "subject_regex": "[A-Za-z:0-9\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\_ \\\\\\\\\\\\\\\\[\\\\\\\\\\\\\\\\]"
        },
        "name": "Mail Attachment Fetcher",
        "group": "Collector",
        "module": "intelmq.bots.collectors.mail.collector_mail_attach",
        "description": "Monitor IMAP mailboxes and retrieve mail attachments",
        "enabled": true,
        "run_mode": "continuous",
        "groupname": "collectors",
        "bot_id": "Mail-Attachment-Fetcher-Collector-2"

Am new user of Intelmq,may you please assist to guide me to finish this intelmq to shadow server integration.

Kr,
Patrick


On Wednesday, March 18, 2020, 06:42:34 PM GMT+3, Bernhard Reiter <bernhard@intevation.de> wrote:


Dear Patric,

Am Mittwoch 18 März 2020 15:57:14 schrieb Sebastian Wagner:
> > Description:    Ubuntu 16.04.6 LTS
>   File "/usr/lib/python3.5/csv.py", line 96, in fieldnames
>     self._fieldnames = next(self.reader, dialect=csv.excel_tab)
> NameError: name 'csv' is not defined


as I happen to have a comparable system here, I took a look in the file.
The code on line 96 looks different, so can you double check which version you
have, e.g.
  dpkg -S /usr/lib/python3.5/csv.py
and then using the result to query it via dpkg -l,
for my system:
 
dpkg -l libpython3.5-stdlib | cat
ii  libpython3.5-stdlib:amd64 3.5.2-2ubuntu0~16.04.9 amd64

BTW: Which version of intelmq and how did you install it?

Best,
Bernhard

--
www.intevation.de/~bernhard   +49 541 33 508 3-3
Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998
Geschäftsführer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner