IntelMQ-users April 2018

intelmq-users@lists.cert.at

1 participants
4 discussions

Shadowserver compromised website parser -ERROR
by Tomislav Protega 02 Jul '18

02 Jul '18

Hi, I ran into this error: Shadowserver-Compromised-Website-Parser - ERROR - Could not convert shadowkey: 'http_host', value: '' via conversion function 'validate_fqdn'. More detailed log is attached. This happens when "http_host" field in the shadowserver origin report contains IP instead of domain which is not something unusual. At the end IntelMQ does produce the output data, but there's no 'source.url' field which should contain merged 'http_host' and 'url' parameters from the origin report. Regards, -- Tomislav

2 2

reverse_dns-expert exception InvalidValue
by Tomislav Protega 02 Jul '18

02 Jul '18

Hi, here's another exception I ran into. If Reverse DNS expert bot runs into IP address which doesn't have DNS PTR record in fqdn form, then it gives exception. In attachment are logs. Example: There are cases when DNS gives two records for PTR. $ nslookup 5.157.80.221 Server: 8.8.8.8 Address: 8.8.8.8#53 Non-authoritative answer: 221.80.157.5.in-addr.arpa name = 5.157.80.221. 221.80.157.5.in-addr.arpa name = aliancys.peopleinc.nl. -------------------------------------------------------- $ nslookup 121.201.38.118 Server: 8.8.8.8 Address: 8.8.8.8#53 Non-authoritative answer: 118.38.201.121.in-addr.arpa name = 121.201.38.118. Regards, -- Tomislav

2 1

intelmq-manager release 1.0.0 & 1.0.1
by Sebastian Wagner 23 Apr '18

23 Apr '18

Dear community, I just released a new stable release 1.0.0 and maintenance release 1.0.1 of intelmq-manager. Thanks to Edvard Reijthar, the project is very active again and many changes can be expected for the next versions too! Version 1.0.0 fixes some bugs and adds an interface for `intelmqctl check`. 1.0.1 fixes the version number in the code (shown in the about page). Installation instructions: https://github.com/certtools/intelmq-manager/blob/1.0.1/docs/INSTALL.md The version is compatible with intelmq >= 1.0.3 Combined changelog: ### Backend * Set content type correctly for JSON data in configuration loading (#112) * Fix version number. ### Pages * All pages are now deliverd by php, reducing the amount of duplicate code drastically. #### Landing page * Added a new block for the new check page, changed the formatting a bit. #### Configuration * Fixed handling of special parameter `run_mode` (#150) * Intelmqctl controller may be set via an env variable `INTELMQ_MANGER_CONTROLER_CMD` #### Check * Added, showing the output of `intelmqctl check` (#118). ### Documentation * Note on header Content-Security-Policy (#113) * Note on security considerations in Readme to avoid misunderstandings * Remove compatibility warning from README ### Third-party libraries * reverted update jQuery to 3.2.1 * reverted update metisMenu to 2.7.0 ### Licenses * Licenses of used and included software is now inventarized and properly declared in LICENSES/ (#134) ### Packaging * fix packaging of positions.conf file for deb-packages (#133). ### Known issues * Missing CSRF protection (#111). * Missing copyright notices (#140). * Graph jumps around on "Add edge" bug component (#148). * new runtime parameters with _ not possible (#153). * wrong error message for new bots with existing ID (#152). Sebastian -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

1 0

intelmq release 1.0.4
by Sebastian Wagner 20 Apr '18

20 Apr '18

Dear community, I just released a new maintenance release 1.0.4 of intelmq. It only fixes bugs in the 1.0.x series and may be the last version of the 1.0.x series. Installation instructions: https://github.com/certtools/intelmq/blob/1.0.4/docs/INSTALL.md Upgrade instructions: https://github.com/certtools/intelmq/blob/1.0.4/docs/UPGRADING.md The changes are: - make code style compatible to pycodestyle 2.4.0 - fixed permissions of some files (they were executable but shouldn't be) ### Core - lib/harmonization: * FQDN validation now handles None correctly (raised an Exception). * Fixed several sanitize() methods, the generic sanitation method were called by is_valid, not the sanitize methods (#1219). ### Harmonization ### Bots * Use the new pypi website at https://pypi.org/ everywhere. #### Parsers - Shadowserver parser: * The fields `url` and `http_url` now handle HTTP URL paths and HTTP requests for all feeds (#1204). * The conversion function `validate_fqdn` now handles empty strings correctly. * Feed 'drone (hadoop)': * Correct validation of field `cc_dns`, will now only be added as `destination.fqdn` if correct FQDN, otherwise ignored. Previously this field could be saved in extra containing an IP address. * Adding more mappings for added columns. * A lot of newly added fields and fixed conversions. * Add newly added columns of `Ssl-Scan` feed to parser - Spamhaus CERT parser: * fix parsing and classification for bot names 'openrelay', 'iotrdp', 'sshauth', 'telnetauth', 'iotcmd', 'iotuser', 'wpscanner', 'w_wplogin', 'iotscan' see the NEWS file - Postgresql section - for all changes. - CleanMX phishing parser: handle FQDNs in IP column (#1162). #### Experts - `bots.experts.ripencc_abuse_contact`: Add existing parameter `mode` to BOTS file. ### Tools - intelmqctl check: Fixed and extended message for 'run_mode' check. - `intelmqctl start` botnet. When using `--type json`, no non-json information about wrong bots are output because that would confuse eg. intelmq-manager ### Tests - lib/bot: No dumps will be written during tests (#934). - lib/test: Expand regular expression on python version to match pre-releases (debian testing). ### Packaging * Static data is now included in source tarballs, development files are excluded Sebastian -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

IntelMQ-users April 2018