IntelMQ-dev October 2021

intelmq-dev@lists.cert.at

5 participants
4 discussions

Help for fixing configuration of intelmq-manager 3.0.1-1
by Mika Silander 03 Nov '21

03 Nov '21

Hi, Seems I've gotten stuck on the same problem than before. However, this time around the order of installation was (on a Ubuntu 20.04 LTS/Apache 2): apt-get install intelmq apt-get install intelmq-api apt-get install intelmq-manager and the versions were: intelmq 3.0.2-1 intelmq-api 3.0.1-1 intelmq-manager 3.0.1-1 I'm able to access the intelmq-manager start page, https://ourimq.domain/intelmq-manager/index.html but when opening the Configuration tab https://ourimq.domain/intelmq-manager/configs.html, expected frames appear but no bots are shown in the frame (normally) allowing editing of the bot network. Instead an endless loop starts with the error shown further below. All hints for troubleshooting & debugging are welcome. Br, Mika P.S: No SELinux lurking in the shadows on the install host. --- 16:42:30 115× Error loading bot queues information: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not Found</h1> <p>The requested URL was not found on this server.</p> <hr> <address>Apache/2.4.41 (Ubuntu) Server at ourimq.domain Port 443</address> </body></html> Not Found 16:40:38 Get an error error, Not Found when trying to obtain config file properly https://ourimq.domain/intelmq/v1/api/positions . 16:40:38 Get an error error, Not Found when trying to obtain config file properly https://ourimq.domain/intelmq/v1/api/runtime . 16:40:37 Get an error error, Not Found when trying to obtain config file properly https://ourimq.domain/intelmq/v1/api/bots

2 6

Potential attribute name clashes in bot parameters in intelmq 3.1.x ?
by Mika Silander 02 Nov '21

02 Nov '21

Hi, At long last I updated my develop branch from the 2.3. days to 3.1.0 and noticed my tests fail due to the fact that bots don't have anymore a "parameters" attribute. At a closer look it seems all parameters from runtime.yaml are turned into direct attributes of self (correct?), e.g. if one defines a conf parameter "myparam", this shows up in the bot as self.myparam and not accessible as getattr(self.parameters, "myparam") as it did before. After experimenting with my own bot tests I ended up in a situation where there's a potential attribute name clash. Assume a bot is tested like self.input_message = some_event_here self.run_bot() self.assertSomethingHere() In the above test, the bot gets initiated with a big number of default attributes, e.g. accuracy, group, enabled, logger, run_mode etc etc. Assume then that as a developer I want to use parameters with matching names for the needs of my own bot like self.input_message = some_event_here self.run_bot(parameters={ 'group': 'Plumber', 'run_mode': 'disruptive' }) self.assertSomethingHere() I can set those parameters and they happily override the defaults and I imagine this can mean trouble ahead. Is this intended behaviour and if yes, is there a way to prevent it? With the earlier self.parameters construct this did not happen and one could have e.g. a "group" attribute for the needs of intelmq's internal operation (self.group) separate from a bot developer's "group" attribute (that ended up under self.parameters) - no clashes despite equal names. Thus, to avoid the above in intelmq 3.0.x this means every bot developer needs to check her own bot won't use any of the 'reserved' attributes before defining their own ones, right? I might and I definitively hope I'm wrong here but please let me know if this is the case. Br, Mika

4 8

Systemd startup scripts
by Mika Silander 02 Nov '21

02 Nov '21

Hi, Afaik the bots can currently be managed with the command intelmqctl only. Has anyone implemented systemd startup scripts for intelmq? Br, Mika

3 5

Building packages for intelmq 3.0.2 stumbles on two tests
by Mika Silander 29 Oct '21

29 Oct '21

Hi, Tried building the intelmq debian packages on a Ubuntu 18 LTS, python 3.6 with dpkg-buildpackage -us -uc -b -d Branch was maintenance, tag 3.0.2 and the option -d above was added because Cerberus 1.3.2 was installed with pip3 and not with apt from python3-cerberus. Seems two tests make the build fail, see below. A known problem? Any hints for troubleshooting this? Airing this here before starting to debug on my own ... Cheers, Mika ====================================================================== FAIL: test_multiple_bots (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Let's simulate multiple IntelMQ instances are pushing the events at once! ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 175, in test_multiple_bots self.run_bot(iterations=client_count + 1) File "/home/mika/intelmq/intelmq/lib/test.py", line 361, in run_bot self.assertLessEqual(len(re.findall(' - ERROR - ', self.loglines_buffer)), allowed_error_count) AssertionError: 2 not less than or equal to 0 ====================================================================== FAIL: test_random_input (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Check how we handle a random input, coming from an unknown source. We should put all the data to report['raw']. ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 106, in test_random_input self.run_bot() File "/home/mika/intelmq/intelmq/lib/test.py", line 363, in run_bot self.assertLessEqual(len(re.findall(' - WARNING - ', self.loglines_buffer)), allowed_warning_count) AssertionError: 1 not less than or equal to 0 ---------------------------------------------------------------------- Ran 1404 tests in 125.121s FAILED (failures=2, skipped=274) Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> error: Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> E: pybuild pybuild:336: test: plugin distutils failed with: exit code=1: python3.6 setup.py test dh_auto_test: pybuild --test -i python{version} -p 3.6 returned exit code 13 debian/rules:23: recipe for target 'build' failed make: *** [build] Error 25 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2

2 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

IntelMQ-dev October 2021