IntelMQ-users January 2018

intelmq-users@lists.cert.at

5 participants
4 discussions

Shadowserver compromised website parser -ERROR
by Tomislav Protega 02 Jul '18

02 Jul '18

Hi, I ran into this error: Shadowserver-Compromised-Website-Parser - ERROR - Could not convert shadowkey: 'http_host', value: '' via conversion function 'validate_fqdn'. More detailed log is attached. This happens when "http_host" field in the shadowserver origin report contains IP instead of domain which is not something unusual. At the end IntelMQ does produce the output data, but there's no 'source.url' field which should contain merged 'http_host' and 'url' parameters from the origin report. Regards, -- Tomislav

2 2

reverse_dns-expert exception InvalidValue
by Tomislav Protega 02 Jul '18

02 Jul '18

Hi, here's another exception I ran into. If Reverse DNS expert bot runs into IP address which doesn't have DNS PTR record in fqdn form, then it gives exception. In attachment are logs. Example: There are cases when DNS gives two records for PTR. $ nslookup 5.157.80.221 Server: 8.8.8.8 Address: 8.8.8.8#53 Non-authoritative answer: 221.80.157.5.in-addr.arpa name = 5.157.80.221. 221.80.157.5.in-addr.arpa name = aliancys.peopleinc.nl. -------------------------------------------------------- $ nslookup 121.201.38.118 Server: 8.8.8.8 Address: 8.8.8.8#53 Non-authoritative answer: 118.38.201.121.in-addr.arpa name = 121.201.38.118. Regards, -- Tomislav

2 1

New Bots Development Workflow
by Leandro Velasco 18 Jan '18

18 Jan '18

Hello everybody, I have not much experience with intelmq nor python packages managments so while while implementing a couple of new bots, I have experienced the following issues: 1- According to the documentation the best way to proceed is to clone the repo and install intelmq via “pip3 install -e .”. The issue here is that it needs to be done from root, otherwise the install fails. However once it is installed the user intelmq cannot see the intelmq installation (pip3 list) since it is only installed for root. 2- If I Install from pip3 directly (not using a local repo) intelmq works just fine but then, new bot creation fails since according to the documentation new bots needs to be generated reinstalling intelmq. Trying to manually generate the binaries for the bots is not recommended according to previous emails (in the archive). 3- Using virtual env allowed me to install intelmq as recommended for developers (“pip3 install -e .”.) and new bots are generated appropriately . However, some issues appear with the pipeline connection and nowhere in the documentation it is suggested. My question then is, what is the recommended and verified way of implementing new bots for intelmq? Starting from the installation of intelmq to testing the new bots. My environment: VM with Ubuntu 16.04 Thanks in advance for your time! Best regards, Leandro Velasco --- Rijnland 4c 1948 RL Beverwijk Leandro.Velasco(a)dearbytes.nl<mailto:Leandro.Velasco@dearbytes.nl> www.dearbytes.nl<http://www.dearbytes.nl>

2 1

elasticsearch parsing exception
by Tomislav Protega 06 Jan '18

06 Jan '18

Hi, recently I came up into elasticsearch parsing exception. Dump is attached below. It only happens when it processes data from Blueliv Crimeserver and Shadowserver-Open-XDMCP collectors. Not so far ago my elasticsearch output bot didn't throw that exception. Currently I'm using intelmq 1.0.2 and intelmq-manager 0.3.1, all installed from .deb package and python client elasticsearch 6.0.0. Anyone experienced the same? Thanks for the efforts. Regards, -- Tomislav

3 3

2025

2024

2023

2022

2021

2020

2019

2018

2017

IntelMQ-users January 2018