- IntelMQ-users

IntelMQ version 2.1.2
by Sebastian Wagner 28 Jan '20

28 Jan '20

Dear community, Version 2.1.2, a bugfix release, is out which contains various bugfixes. Install documentation: https://github.com/certtools/intelmq/blob/2.1.2/docs/INSTALL.md Upgrade documentation: https://github.com/certtools/intelmq/blob/2.1.2/docs/UPGRADING.md The full changelog: ### Core - `__init__`: Resolve absolute path for `STATE_FILE_PATH` variable (resolves `..`). - `intelmq.lib.utils`: - log: Do not raise an exception if logging to neither file nor syslog is requested. - logging StreamHandler: Colorize all warning and error messages red. - logging FileHandler: Strip all shell colorizations from the messages (#1436). - `intelmq.lib.message`: - `Message.to_json`: Set `sort_keys=True` to get reproducible results. - `drop_privileges`: Handle situations where the user or group `intelmq` does not exist. - `intelmq.lib.pipeline`: - `Amqp._send` and `Amqp._acknowledge`: Log traceback in debug mode in case of errors and necessary re-connections. - `Amqp._acknowledge`: Reset delivery tag if acknowledge was successful. ### Bots #### Collectors - `intelmq.bots.collectors.misp.collector`: - Add compatibility with current pymisp versions and versions released after January 2020 (PR #1468). #### Parsers - `intelmq.bots.parsers.shadowserver.config`: Add some missing fields for the feed `accessible-rdp` (#1463). - `intelmq.bots.parsers.shadowserver.parser`: - Feed-detection based on file names: The prefixed date is optional now. - Feed-detection based on file names: Re-detect feed for every report received (#1493). #### Experts - `intelmq.bots.experts.national_cert_contact_certat`: Handle empty responses by server (#1467). - `intelmq.bots.experts.maxmind_geoip`: The script `update-geoip-data` now requires a license key as second parameter because of upstream changes (#1484)). #### Outputs - `intelmq.bots.outputs.restapi.output`: Fix logging of response body if response status code was not ok. ### Documentation - Remove some hardcoded `/opt/intelmq/` paths from code comments and program outputs. ### Packaging - debian/rules: Only replace `/opt/intelmq/` with LSB-paths in some certain files, not the whole tree, avoiding wrong replacements. - debian/rules and debian/intelmq.install: Do install the examples configuration directly instead of working around the abandoned examples directory. ### Tests - `lib/test_utils`: Skip some tests on Python 3.4 because `contextlib.redirect_stdout` and `contextlib.redirect_sterr` are not supported on this version. - Travis: Stop running tests with all optional dependencies on Python 3.4, as more and more libraries are dropping support for it. Tests on the core and code without non-optional requirements are not affected. - `tests.bots.parsers.html_table`: Make tests independent of current year. ### Tools - `intelmqctl upgrade-config`: Fix missing substitution in error message "State file %r is not writable.". ### Known issues - bots trapped in endless loop if decoding of raw message fails (#1494) - intelmqctl status of processes: need to check bot id too (#1492) - MongoDB authentication: compatibility on different MongoDB and pymongo versions (#1439) - ctl: shell colorizations are logged (#1436) - http stream collector: retry on regular connection problems? (#1435) - tests: capture logging with context manager (#1342) - Bots started with IntelMQ-Manager stop when the webserver is restarted. (#952) - n6 parser: mapping is modified within each run (#905) - reverse DNS: Only first record is used (#877) - Corrupt dump files when interrupted during writing (#870) -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

1 0

heads up: maxmind license updates (relevant for IntelMQ)
by L. Aaron Kaplan 09 Jan '20

09 Jan '20

Dear IntelMQ users, Sebix mentioned something that might be relevant for everyone using IntelMQ (or actually maxmind for the matter). (quote) "MaxMind has always been committed to an individual’s right to privacy on the internet. We welcome the burgeoning privacy regulations, such as GDPR and CCPA, for the benefit they can provide to internet citizens. However, these new legislative measures place restrictions that impact our ability to continue distributing our GeoLite2 databases on a public page under the Creative Commons Attribution-ShareAlike 4.0 International License." Maxmind has decided to change the download mechanism of the maxmind GeoLite 2 database. You now need to be registered and need a license key. Since I know that many IntelMQ users rely on maxmind for geolocation, you might want to register there and get a new license key and make sure it can be used for the most recent database version. More info at https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-us… All the best and a happy 2020! Aaron. -- // L. Aaron Kaplan <kaplan(a)cert.at> - T: +43 1 5056416 78 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - http://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

2 1

Python 3.4 support coming to an end
by Sebastian Wagner 09 Dec '19

09 Dec '19

Hi, The upstream support of Python 3.4 ended already, this year in March the last bugfix was released. The current oldoldstable Debian version (jessie) uses this version, but I am not aware of any other popular system with an ancient Python version. Even CentOS 7 supports Python 3.6 (previously 3.4). As a consequence, more and more downstream libraries are dropping support for python 3.4, causing more maintenance work for me. I thus came to the point to slowly say Python 3.4 goodbye by -) first only running the CI tests on the core functionalities of IntelMQ for Python 3.4, ignoring other parts with optional requirements -) In the next minor release, only Python >= 3.5 will be supported I hope that this works for everyone.[0] Thanks, Sebastian [0]: If not, please contribute. -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

1 0

Re: [Intelmq-users] IntelMQ Error
by Sebastian Wagner 02 Dec '19

02 Dec '19

Hi Hatibu, I recommend to upgrade to a newer version, the latest release is 2.1.1. If this is a fresh installation, I recommend to remove the old version and start with the newer instead. Otherwise: It looks like your defaults.conf file (/etc/intelmq/defaults.conf or /opt/intelmq/etc/defaults.conf) is missing the `logging_level` parameter (default value INFO), but maybe there are more values missing. Sebastian On 27/11/2019 06.35, hatibu chande wrote: > Hello Sebastian, > > Yes am using version 1.1.2 > > Hatibu. > > On Tue, Nov 26, 2019 at 5:54 PM Sebastian Wagner <wagner(a)cert.at > <mailto:wagner@cert.at>> wrote: > > Hi, > > You are using IntelMQ version 1.1.2? > > Sebastian > > On 11/26/19 12:52 PM, hatibu chande wrote: >> Hello Guys, >> >> I recently install IntelMQ successfully but it gives >> " Error loading bot queues information: Traceback (most recent >> call last): File "/usr/bin/intelmqctl", line 9, >> in load_entry_point('intelmq==1.1.2', 'console_scripts', >> 'intelmqctl')() File >> "/usr/lib/python3/dist-packages/intelmq/bin/intelmqctl.py", line >> 1225, in main x = IntelMQController(interactive=True) File >> "/usr/lib/python3/dist-packages/intelmq/bin/intelmqctl.py", line >> 380, in __init__ log_level = self.parameters.logging_level >> AttributeError: 'Parameters' object has no attribute >> 'logging_level' SyntaxError: Unexpected token T in JSON at >> position 0" >> >> Can someone help me please. >> >> Hatibu >> > -- > // Sebastian Wagner <wagner(a)cert.at> <mailto:wagner@cert.at> - T: +43 1 5056416 7201 > // CERT Austria - https://www.cert.at/ > // Eine Initiative der nic.at <http://nic.at> GmbH - https://www.nic.at/ > // Firmenbuchnummer 172568b, LG Salzburg > -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

4 4

IntelMQ Error
by hatibu chande 26 Nov '19

26 Nov '19

Hello Guys, I recently install IntelMQ successfully but it gives " Error loading bot queues information: Traceback (most recent call last): File "/usr/bin/intelmqctl", line 9, in load_entry_point('intelmq==1.1.2', 'console_scripts', 'intelmqctl')() File "/usr/lib/python3/dist-packages/intelmq/bin/intelmqctl.py", line 1225, in main x = IntelMQController(interactive=True) File "/usr/lib/python3/dist-packages/intelmq/bin/intelmqctl.py", line 380, in __init__ log_level = self.parameters.logging_level AttributeError: 'Parameters' object has no attribute 'logging_level' SyntaxError: Unexpected token T in JSON at position 0" Can someone help me please. Hatibu

2 1

IntelMQ 2.1.1 release
by Sebastian Wagner 11 Nov '19

11 Nov '19

Dear community, we again collected a bunch of bugfixes in the last weeks, coming almost one month after 2.1.0. Install documentation: https://github.com/certtools/intelmq/blob/2.1.1/docs/INSTALL.md Upgrade documentation: https://github.com/certtools/intelmq/blob/2.1.1/docs/UPGRADING.md The full changelog: ### Configuration - Default configuration: - Remove discontinued feed "Feodo Tracker Domains" from default configuration. - Add "Feodo Tracker Browse" feed to default configuration. ### Core - `intelmq.lib.pipeline`: AMQP: using port 15672 as default (like RabbitMQ's defaults) for the monitoring interface for getting statistical data (`intelmqctl_rabbitmq_monitoring_url`). - `intelmq.lib.upgrades`: Added a generic upgrade function for harmonization, checking of all message types, it's fields and their types. - `intelmq.lib.utils`: - `TimeoutHTTPAdapter`: A subclass of `requests.adapters.HTTPAdapter` with the possibility to set the timeout per adapter. - `create_request_session_from_bot`: Use the `TimeoutHTTPAdapter` with the user-defined timeout. Previously the timeout was not functional. ### Bots #### Parsers - `intelmq.bots.parsers.shadowserver.parser`: Fix logging message if the parameter `feedname` is not present. - `intelmq.bots.parsers.shodan.parser`: Also add field `classification.identifier` (`'network-scan'`) in minimal mode. - `intelmq.bots.parsers.spamhaus.parser_cert`: Add support for category `'misc'`. - `intelmq.bots.parsers.cymru.parser_cap_program`: - Add support for phishing events without URL. - Add support for protocols >= 143 (unassigned, experiments, testing, reserved), saving the number to extra, as the data would be bogus. - `intelmq.bots.parsers.microsoft.parser_bingmurls`: - Save the `Tags` data as `source.geolocation.cc`. #### Experts - `intelmq.bots.experts.modify.expert`: Fix bug with setting non-string values (#1460). #### Outputs - `intelmq.bots.outputs.smtp`: - Allow non-existent field in text formatting by using a default value `None` instead of throwing errors. - Fix Authentication (#1464). - Fix sending to multiple recipients (#1464). ### Documentation - Feeds: - Fix configuration of `Feodo Tracker Browse` feed. - Bots: - Sieve expert: Document behavior of `!=` with lists. ### Tests - Adaption and extension of the test cases to the changes. ### Tools - `intelmq.bin.intelmqctl`: - check: Check if running the upgrade function for harmonization is necessary. - upgrade-config: Run the upgrade function for harmonization. - `intelmqctl restart` did throw an error as the message for restarting was not defined (#1465). ### Known issues - MongoDB authentication: compatibility on different MongoDB and pymongo versions (#1439) - ctl: shell colorizations are logged (#1436) - http stream collector: retry on regular connection problems? (#1435) - tests: capture logging with context manager (#1342) - Bots started with IntelMQ-Manager stop when the webserver is restarted. (#952) - n6 parser: mapping is modified within each run (#905) - reverse DNS: Only first record is used (#877) - Corrupt dump files when interrupted during writing (#870) -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

1 0

IntelMQ 2.1.0 release
by Sebastian Wagner 15 Oct '19

15 Oct '19

Dear community, Given the vast amount of changes, additions and new features, it's time to mark a new feature release! Thanks to all the contributors who participate in this community project! IntelMQ gained a lot of new bots and features in the last months. Install documentation: https://github.com/certtools/intelmq/blob/2.1.0/docs/INSTALL.md Upgrade documentation: https://github.com/certtools/intelmq/blob/2.1.0/docs/UPGRADING.md Full changelog: ### Core - `intelmq.lib.harmonization`: - Use correct parent classes. - Add `DateTime.convert` as interface for all existing conversion functions. - add `DateTime.convert_from_format`. - add `DateTime.convert_from_format_midnight`. - add `DateTime.convert_fuzzy`. - `intelmq.lib.pipeline`: - Redis: Use single connection client if calling bot is not multithreaded. Gives a small speed advantage. - Require the bot instance as parameter for all pipeline classes. - New internal variable `_has_message` to keep the state of the pipeline. - Split receive and acknowledge into public-facing and private methods. - Add `reject_message` method to the Pipeline class for explicit requeue of messages. - AMQP: - Make exchange configurable. - If exchange is set, the queues are not declared, the queue name is for routing used by exchanges. - `intelmq.lib.bot`: - Log message after successful bot initialization, no log message anymore for ready pipeline. - Use existing current message if receive is called and the current message still exists. - Fix handling of received messaged after a sighup that happend during a blocking receving connection using explicit rejection (#1438). - New method `_parse_common_parameters` called before `init` to parse commonly used argument. Currently supported: `extract_files`. - `intelmq.lib.test`: - Fix the tests broker by providing the testing pipeline. - `intelmq.lib.utils`: - `unzip`: - new parameter `return_names` to optionally return the file names. - support for zip - new parameters `try_zip`, `try_gzip` and `try_tar` to control which compressions are tried. - rewritten to an iterative approach - add `file_name_from_response` to extract a file name from a Response object for downloaded files. - `intelmq.lib.upgrades`: Added `v210_deprecations` for deprecated parameters. ### Harmonization - Add extra to reports. ### Bots #### Collectors - `intelmq.bots.collectors.http.collector_http`: - More extensive usage of `intelmq.lib.utils.unzip`. - Save the file names in the report if files have been extracted form an archive. - `intelmq.bots.collectors.rt.collector_rt`: - Save ticket information/metadata in the extra fields of the report. - Support for RT 3.8 and RT 4.4. - New parameters `extract_attachment` and `extract_download` for generic archive extraction and consistency. The parameter `unzip_attachment` is deprecated. - `intelmq.bots.collectors.mail.*`: Save email information/metadata in the extra fields of the report. See the bots documentation for a complete list of provided data. - `intelmq.bots.collectors.mail.collector_mail_attach`: Check for existence/validity of the `attach_regex` parameter. - Use the lib's `unzip` function for uncompressing attachments and use the . - `intelmq.bots.collectors.mail.collector_mail_url`: Save the file name of the downloaded file as `extra.file_name`. - `intelmq.bots.collectors.amqp.collector_amqp`: New collector to collect data from (remote) AMQP servers, for bot IntelMQ as well as external data. - use default SSL context for client purposes, fixes compatibility with python < 3.6 if TLS is used. #### Parsers - `intelmq.bot.parsers.html_table.parser`: * New parameter "html_parser". * Use time conversion functions directly from `intelmq.lib.harmonization.DateTime.convert`. - Limit lxml dependency on 3.4 to < 4.4.0 (incompatibility). - `intelmq.bots.parsers.netlab_360.parser`: Add support for hajime scanners. - `intelmq.bots.parsers.hibp.parser_callback`: A new parser to parse data retrieved from a HIBP Enterprise Subscription. - `intelmq.bots.parsers.shadowserver.parser`: - Ability to detect the feed base on the reports's field `extra.file_name`, so the parameter `feedname` is no longer required and one configured parser can parse any feed (#1442). #### Experts - Add geohash expert. - `intelmq.bot.experts.generic_db_lookup.expert` - new optional parameter `engine` with `postgresql` (default) and `sqlite` (new) as possible values. #### Outputs - Add `intelmq.bots.outputs.touch.output`. - `intelmq.bot.outputs.postgresql.output`: - deprecated in favor of `intelmq.bot.outputs.sql.output` - Compatibility shim will be available in the 2.x series. - `intelmq.bot.outputs.sql.output` added generic SQL output bot. Comparted to - new optional parameter `engine` with `postgresql` (default) and `sqlite` (new) as possible values. - `intelmq.bots.outputs.stomp.output`: New parameters `message_hierarchical_output`, `message_jsondict_as_string`, `message_with_type`, `single_key`. ### Documentation - Feeds: - Add ViriBack feed. - Add Have I Been Pwned Enterprise Callback. - `intelmq.tests.bots.outputs.amqptopic.test_output`: Added. - Move the documentation of most bots from separate README files to the central Bots.md and feeds.yaml files. ### Tests - Travis: - Use UTC timezone. - Tests for `utils.unzip`. - Add a new asset: Zip archive with two files, same as with tar.gz archive. - Added tests for the Mail Attachment & Mail URL collectors. - Ignore logging-tests on Python 3.7 temporarily (#1342). ### Tools - intelmqctl: - Use green and red text color for some interactive output to indicate obvious errors or the absence of them. - intelmqdump: - New edit action `v` to modify a message saved in the dump (#1284). ### Contrib * malware name mapping: * Add support for MISP treat actors data, see it's README for more information. * And handle empty synonyms in misp's galxies data. * Move apply-Script to the new EventDB directory * EventDB: Scripts for applying malware name mapping and domain suffixes to an EventDB. ### Known issues - MongoDB authentication: compatibility on different MongoDB and pymongo versions (#1439) - ctl: shell colorizations are logged (#1436) - http stream collector: retry on regular connection problems? (#1435) - tests: capture logging with context manager (#1342) - Bots started with IntelMQ-Manager stop when the webserver is restarted. (#952) - n6 parser: mapping is modified within each run (#905) - reverse DNS: Only first record is used (#877) - Corrupt dump files when interrupted during writing (#870) -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

1 0

IntelMQ Manager Release 2.1.0
by Sebastian Wagner 15 Oct '19

15 Oct '19

Dear community, The new 2.1.0 version of the intelmq manager improves the error reporting in the frontend and allows to connect collectors to experts and experts to parsers. Thanks to Edvard for his work on this tool! Installation instructions: https://github.com/certtools/intelmq-manager/blob/2.0.0/docs/INSTALL.md Full changelog: ### Backend - Fix mispelling of the environmental variable `INTELMQ_MANGER_CONTROLER_CMD` to `INTELMQ_MANGER_CONTROLLER_CMD` (you might be required to add the double 'l' to your nginx/apache server configuration) - When displaying a command to be replicated by debugging user, the string "sudo -u {webserver user}" string is prepended so that linux user do not have to bother with sudoing himself on the commonly used user "www-data" – which often can't be sudoed to (no bash provided due to good security measures). (Used in monitor and error reporting.) ### Frontend - Error reporting - Click will enlarge the dialog that contains much more useful info, notably the very command that failed so that it can be easily reproduced. - Error messages are shuffled only when minimized, not when maximized. That would disturb the user trying to read the details. - Invalid Syntax Error message removed. Till now, all error messages generated the string that JSON received is invalid – that wasn't needed, we knew it's invalid because it contained string message. - Double click does not close log window anymore since it would interfere with the user trying to select whole text by mouse. - Escape minimizes the reporting. - For common seen errors, a tip is displayed (preferable with a link to the Github manual). ### Pages #### Configuration - Node group Collector now may connect to Expert and Expert can connect to Parser, however you receive a warning that it is not very common. -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 1 5056416 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

1 0

IntelMQ 2.0.2 Bugfix release
by Sebastian Wagner 14 Oct '19

14 Oct '19

Dear community, Today I released version 2.0.2 of intelmq with a bunch of bugfixes, see the changelog below. A feature release is coming soon too. Install documentation: https://github.com/certtools/intelmq/blob/2.0.2/docs/INSTALL.md Upgrade documentation: https://github.com/certtools/intelmq/blob/2.0.2/docs/UPGRADING.md As always: read the NEWS file, upgrade according to the documentation and have fun! If you get any errors, please report them here or in the bug tracker. Sebastian Full changelog: ### Core - `intelmq.lib.bot.CollectorBot`: Support the deprecated parameter `feed` until version 2.2 as the documentation was not properly updated (#1445). - `intelmq.lib.bot.Bot`: - `_dump_message`: Wait for up to 60 seconds instead of 50 if the dump file is locked (the log message was said 60, but the code was for only 50). - `intelmq.lib.upgrades.v202_fixes` - Migration of deprecated parameter `feed` for Collectors. - Ripe expert parameter `query_ripe_stat_ip` was not correctly configured in `v110_deprecations`, now use `query_ripe_stat_asn` as default if it does not exist. - `intelmq.lib.upgrades.v110_deprecations`: Fix upgrade of ripe expert configuration. - `intelmq.lib.bot_debugger`: - Fix handling of empty messages generated by parser when user wanted to show the result by "--show-sent" flag. - Fix handling of sent messages for bots using the `path_permissive` paramter (#1453). - `intelmq.lib.pipeline.Amqp`: - use default SSL context for client purposes, fixes compatibility with python < 3.6 if TLS is used. - Reconnect once on sending messages if disconnect detected. ### Bots #### Collectors - `intelmq.bots.collectors.api.collector_api`: - Handle non-existing IO loop in shutdown. - Close socket on shutdown, fixes reloading. - Marked as non-threadable. - `intelmq.bots.collectors.rt.collector_rt`: Check for matching URLs if no `attachment_regex` is given. - `intelmq.bots.collectors.stomp.collector_stomp`: Handle disconnects by actively reconnecting. #### Parsers - `intelmq.bots.cymru.parser_cap_program`: Fix parsing of the new `$certname_$date.txt` report format (#1443): - Support protocol ICMP. - Fix error message for unsupported protocols. - Support fields `destination_port_numbers`, `port`. - Support for all proxy types without ports. - Use Country Code of AS as `source.geolocation.cc`. - Support for 'scanner' and 'spam' categories. - Handle bogus lines with missing separator. - Fix bug preventing use of old format after using the new format. - Handle postfix ` (total_count:..)` for destination port numbers. #### Experts - `intelmq.bots.experts.cymru_whois.expert`: Add optional parameter `overwrite`, current behavior was `True`, default if not given is `True` now, will change to `False` in 3.0.0 (#1452, #1455). - `intelmq.bots.experts.modify.expert`: Add optional parameter `overwrite`, current behavior was `True`, default if not given is `True` now, will change to `False` in 3.0.0 (#1452, #1455). - `intelmq.bots.experts.reverse_dns.expert`: Add optional parameter `overwrite`, current behavior was `True`, default if not given is `True` now, will change to `False` in 3.0.0 (#1452, #1455). #### Outputs - `intelmq.bots.outputs.amqptopic.output`: use default SSL context for client purposes, fixes compatibility with python < 3.6 if TLS is used. ### Packaging - Rules: - Exclude intelmqsetup tool in packages - Include update-rfiprisk-data in packages ### Tests - Tests for `intelmq.lib.upgrades.v202_fixes`. - Tests for `intelmq.lib.upgrades.v110_deprecations`. - Extended tests for `intelmq.bots.parser.cymru.parser_cap_program`. ### Tools - intelmqctl: - More and more precise logging messages for botnet starting and restarting, enable and disable. - No error message for disabled bots on botnet reload. - Fix `upgrade-conf` is state file is empty or not existing. - Use arpgarse's `store_true` action for flags instead of `store_const`. - If the loading of the defaults configuration failed, a variable definition was missing and causing an exception (#1456). ### Contrib - Check MK Statistics Cronjob: - Use `statistics_*` parameters. - Make file executable - Handle None values in `*.temporary.*` keys and treat them as 0. - systemd: - Add `PIDFile` parameter to service file. ### Known issues - MongoDB authentication: compatibility on different MongoDB and pymongo versions (#1439) - ctl: shell colorizations are logged (#1436) - http stream collector: retry on regular connection problems? (#1435) - tests: capture logging with context manager (#1342) - Bots started with IntelMQ-Manager stop when the webserver is restarted. (#952) - n6 parser: mapping is modified within each run (#905) - reverse DNS: Only first record is used (#877) - Corrupt dump files when interrupted during writing (#870) -- // Sebastian Wagner <wagner(a)cert.at> - T: +43 1 50564167201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg

1 0

Permissions error
by Vaclav Bruzek 28 Aug '19

28 Aug '19

Hi mailing list, I've recently upgraded intelmq from version 1.1.2 to 2.0.1. My intelmq runs in Docker. I have a slight issue with the botnet startup on the new version (the previous version works ok). Intelmq fails to start because of the exception present at the bottom of the mail. Upon further investigation, when the folder was manually created, other issues started to pop up. All of them were due to a permission error on either the files in /opt/intelmq/etc/ or some else that intelmq tried to access during startup. Did something change in the setup requirements that I've overlooked? 28/08/2019 09:48:272019-08-28 07:48:27,796 DEBG 'intelmq' stdout output: 28/08/2019 09:48:27Traceback (most recent call last): 28/08/2019 09:48:27 File "/usr/local/lib/python3.6/dist-packages/intelmq-2.0.1-py3.6.egg/intelmq/bin/intelmqctl.py", line 657, in __init__ 28/08/2019 09:48:27 logging_level_stream=logging_level_stream) 28/08/2019 09:48:27 File "/usr/local/lib/python3.6/dist-packages/intelmq-2.0.1-py3.6.egg/intelmq/lib/utils.py", line 334, in log 28/08/2019 09:48:27 handler = FileHandler("%s/%s.log" % (log_path, name)) 28/08/2019 09:48:27 File "/usr/lib/python3.6/logging/__init__.py", line 1032, in __init__ 28/08/2019 09:48:27 StreamHandler.__init__(self, self._open()) 28/08/2019 09:48:27 File "/usr/lib/python3.6/logging/__init__.py", line 1061, in _open 28/08/2019 09:48:27 return open(self.baseFilename, self.mode, encoding=self.encoding) 28/08/2019 09:48:27FileNotFoundError: [Errno 2] No such file or directory: '/opt/intelmq/var/log/intelmqctl.log' 28/08/2019 09:48:27 28/08/2019 09:48:27During handling of the above exception, another exception occurred: 28/08/2019 09:48:27 28/08/2019 09:48:27Traceback (most recent call last): 28/08/2019 09:48:27 File "/usr/local/bin/intelmqctl", line 11, in <module> 28/08/2019 09:48:27 load_entry_point('intelmq==2.0.1', 'console_scripts', 'intelmqctl')() 28/08/2019 09:48:27 File "/usr/local/lib/python3.6/dist-packages/intelmq-2.0.1-py3.6.egg/intelmq/bin/intelmqctl.py", line 1710, in main 28/08/2019 09:48:27 x = IntelMQController(interactive=True) 28/08/2019 09:48:27 File "/usr/local/lib/python3.6/dist-packages/intelmq-2.0.1-py3.6.egg/intelmq/bin/intelmqctl.py", line 661, in __init__ 28/08/2019 09:48:27 logging_level_stream=logging_level_stream) 28/08/2019 09:48:27 File "/usr/local/lib/python3.6/dist-packages/intelmq-2.0.1-py3.6.egg/intelmq/lib/utils.py", line 345, in log 28/08/2019 09:48:27 raise ValueError("Invalid configuration, neither log_path is given nor syslog is used.") 28/08/2019 09:48:27ValueError: Invalid configuration, neither log_path is given nor syslog is used. Sincerely, Václav Brůžek

2 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

IntelMQ-users