[IntelMQ-dev] Building packages for intelmq 3.0.2 stumbles on two tests

Tue Oct 5 14:31:18 CEST 2021

Hi, 

Answering myself here: I got the below going by checking the availability of the terminal inside skip_build_environment, so manual builds from a CLI with dpkg-buildpackage seem to succeed for branch maintenance, tag 3.0.2. Build stats included 1404 tests run, 286 skipped ok (for various reasons). Let me know if those numbers are not reasonable. 

Br, Mika 

From: "Mika Silander" <mika.silander at csc.fi> 
To: "intelmq-dev" <intelmq-dev at lists.cert.at> 
Sent: Tuesday, 5 October, 2021 12:25:50 
Subject: Re: [IntelMQ-dev] Building packages for intelmq 3.0.2 stumbles on two tests 

Hi Sebastian, 

Thanks for your quick reply. I'm not familiar with the nitty-gritty details of the automated builds vs. local builds but I don't see environment variables as a reliable method for separating these two cases. One option that comes to mind is trying to deduce whether dpkg-buildpackage is run from a terminal or not and implement this test inside the skip_build_environment method in [ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe3d1080/intelmq/lib/test.py#L102 | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe3d1080/intelmq/lib/test.py#L102 ] . I'm probably unaware of other constraints and details affecting this, so let me know if this is a bad idea. 

Br, MIka 

From: "Sebastian Wagner" <wagner at cert.at> 
To: "Mika Silander" <mika.silander at csc.fi>, "intelmq-dev" <intelmq-dev at lists.cert.at> 
Sent: Tuesday, 5 October, 2021 12:09:16 
Subject: Re: [IntelMQ-dev] Building packages for intelmq 3.0.2 stumbles on two tests 

Dear Mika, 

These two tests are known to fail in build environments. I guess that their intentional limitations on networking cause some troubles here, but I don't know the exact reason. Therefore we skip these two tests in such environments (see the two links) but a local build with dpkg-buildpackage is obviously not covered yet by that logic. 

[ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe3d1080/intelmq/tests/bots/collectors/tcp/test_collector.py#L68 | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe3d1080/intelmq/tests/bots/collectors/tcp/test_collector.py#L68 ] 
[ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe3d1080/intelmq/lib/test.py#L102 | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe3d1080/intelmq/lib/test.py#L102 ] 

Do you know if there's some env variable set which we could check on? Then we can add that to the skip_build_environment definition. 

Hope that helps 

Sebastian 
On 10/5/21 10:55 AM, Mika Silander wrote: 

Hi,

 Tried building the intelmq debian packages on a Ubuntu 18 LTS, python 3.6 with

dpkg-buildpackage -us -uc -b -d

 Branch was maintenance, tag 3.0.2 and the option -d above was added because Cerberus 1.3.2 was installed with pip3
and not with apt from python3-cerberus. Seems two tests make the build fail, see below.

 A known problem? Any hints for troubleshooting this? Airing this here before starting to debug on my own ...

Cheers, Mika

======================================================================
FAIL: test_multiple_bots (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot)
Let's simulate multiple IntelMQ instances are pushing the events at once!
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 175, in test_multiple_bots
    self.run_bot(iterations=client_count + 1)
  File "/home/mika/intelmq/intelmq/lib/test.py", line 361, in run_bot
    self.assertLessEqual(len(re.findall(' - ERROR - ', self.loglines_buffer)), allowed_error_count)
AssertionError: 2 not less than or equal to 0

======================================================================
FAIL: test_random_input (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot)
Check how we handle a random input, coming from an unknown source. We should put all the data to report['raw'].
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 106, in test_random_input
    self.run_bot()
  File "/home/mika/intelmq/intelmq/lib/test.py", line 363, in run_bot
    self.assertLessEqual(len(re.findall(' - WARNING - ', self.loglines_buffer)), allowed_warning_count)
AssertionError: 1 not less than or equal to 0

----------------------------------------------------------------------
Ran 1404 tests in 125.121s

FAILED (failures=2, skipped=274)
Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2>
error: Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2>
E: pybuild pybuild:336: test: plugin distutils failed with: exit code=1: python3.6 setup.py test 
dh_auto_test: pybuild --test -i python{version} -p 3.6 returned exit code 13
debian/rules:23: recipe for target 'build' failed
make: *** [build] Error 25
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
_______________________________________________
IntelMQ-dev mailing list [ https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev | https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev ] [ https://intelmq.readthedocs.io/ | https://intelmq.readthedocs.io/ ] 

-- 
// Sebastian Wagner [ mailto:wagner at cert.at | <wagner at cert.at> ] - T: +43 676 898 298 7201
// CERT Austria - [ https://www.cert.at/ | https://www.cert.at/ ] // Eine Initiative der nic.at GmbH - [ https://www.nic.at/ | https://www.nic.at/ ] // Firmenbuchnummer 172568b, LG Salzburg 

_______________________________________________ 
IntelMQ-dev mailing list 
https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev 
https://intelmq.readthedocs.io/ 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20211005/f7937381/attachment.htm>