Hi Sebastian,
Thanks for your quick reply. I'm not familiar with the nitty-gritty details of the automated builds vs. local builds but I don't see environment variables as a reliable method for separating these two cases. One option that comes to mind is trying to deduce whether dpkg-buildpackage is run from a terminal or not and implement this test inside the skip_build_environment method in [ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... ] . I'm probably unaware of other constraints and details affecting this, so let me know if this is a bad idea.
Br, MIka
From: "Sebastian Wagner" wagner@cert.at To: "Mika Silander" mika.silander@csc.fi, "intelmq-dev" intelmq-dev@lists.cert.at Sent: Tuesday, 5 October, 2021 12:09:16 Subject: Re: [IntelMQ-dev] Building packages for intelmq 3.0.2 stumbles on two tests
Dear Mika,
These two tests are known to fail in build environments. I guess that their intentional limitations on networking cause some troubles here, but I don't know the exact reason. Therefore we skip these two tests in such environments (see the two links) but a local build with dpkg-buildpackage is obviously not covered yet by that logic.
[ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... ] [ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... ]
Do you know if there's some env variable set which we could check on? Then we can add that to the skip_build_environment definition.
Hope that helps
Sebastian On 10/5/21 10:55 AM, Mika Silander wrote:
Hi,
Tried building the intelmq debian packages on a Ubuntu 18 LTS, python 3.6 with
dpkg-buildpackage -us -uc -b -d
Branch was maintenance, tag 3.0.2 and the option -d above was added because Cerberus 1.3.2 was installed with pip3 and not with apt from python3-cerberus. Seems two tests make the build fail, see below.
A known problem? Any hints for troubleshooting this? Airing this here before starting to debug on my own ...
Cheers, Mika
====================================================================== FAIL: test_multiple_bots (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Let's simulate multiple IntelMQ instances are pushing the events at once! ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 175, in test_multiple_bots self.run_bot(iterations=client_count + 1) File "/home/mika/intelmq/intelmq/lib/test.py", line 361, in run_bot self.assertLessEqual(len(re.findall(' - ERROR - ', self.loglines_buffer)), allowed_error_count) AssertionError: 2 not less than or equal to 0
====================================================================== FAIL: test_random_input (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Check how we handle a random input, coming from an unknown source. We should put all the data to report['raw']. ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 106, in test_random_input self.run_bot() File "/home/mika/intelmq/intelmq/lib/test.py", line 363, in run_bot self.assertLessEqual(len(re.findall(' - WARNING - ', self.loglines_buffer)), allowed_warning_count) AssertionError: 1 not less than or equal to 0
---------------------------------------------------------------------- Ran 1404 tests in 125.121s
FAILED (failures=2, skipped=274) Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> error: Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> E: pybuild pybuild:336: test: plugin distutils failed with: exit code=1: python3.6 setup.py test dh_auto_test: pybuild --test -i python{version} -p 3.6 returned exit code 13 debian/rules:23: recipe for target 'build' failed make: *** [build] Error 25 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 _______________________________________________ IntelMQ-dev mailing list [ https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev | https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev ] [ https://intelmq.readthedocs.io/ | https://intelmq.readthedocs.io/ ]