[IntelMQ-dev] Building packages for intelmq 3.0.2 stumbles on two tests

Sebastian Wagner wagner at cert.at
Fri Oct 29 19:29:19 CEST 2021 

In the Ubuntu 18.04 package builds 1404 tests are running, with
skipped=285
(https://build.opensuse.org/public/build/home:sebix:intelmq/xUbuntu_18.04/x86_64/intelmq/_log
at 617s), so the numbers are perfectly legit.

To detect if the tests are running inside the dpkg-buildpackage
environment, I extended the detection logic by checking for
DEB_BUILD_ARCH:
https://github.com/certtools/intelmq/commit/7ebb8e16d821c372a44b077dd18a151c07f75807

Sebastian

On 10/5/21 2:31 PM, Mika Silander wrote:
> Hi,
>
>  Answering myself here: I got the below going by checking the
> availability of the terminal inside skip_build_environment, so manual
> builds from a CLI with dpkg-buildpackage seem to succeed for branch
> maintenance, tag 3.0.2. Build stats included 1404 tests run, 286
> skipped ok (for various reasons). Let me know if those numbers are not
> reasonable.
>
> Br, Mika
> ------------------------------------------------------------------------
> *From: *"Mika Silander" <mika.silander at csc.fi>
> *To: *"intelmq-dev" <intelmq-dev at lists.cert.at>
> *Sent: *Tuesday, 5 October, 2021 12:25:50
> *Subject: *Re: [IntelMQ-dev] Building packages for intelmq 3.0.2
> stumbles on two tests
>
> Hi Sebastian,
>
>  Thanks for your quick reply. I'm not familiar with the nitty-gritty
> details of the automated builds vs. local builds but I don't see
> environment variables as a reliable method for separating these two
> cases. One option that comes to mind is trying to deduce whether
> dpkg-buildpackage is run from a terminal or not and implement this
> test inside the skip_build_environment method
> in https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe3d1080/intelmq/lib/test.py#L102.
> I'm probably unaware of other constraints and details affecting this,
> so let me know if this is a bad idea.
>
> Br, MIka
>
> ------------------------------------------------------------------------
> *From: *"Sebastian Wagner" <wagner at cert.at>
> *To: *"Mika Silander" <mika.silander at csc.fi>, "intelmq-dev"
> <intelmq-dev at lists.cert.at>
> *Sent: *Tuesday, 5 October, 2021 12:09:16
> *Subject: *Re: [IntelMQ-dev] Building packages for intelmq 3.0.2
> stumbles on two tests
>
> Dear Mika,
>
> These two tests are known to fail in build environments. I guess that
> their intentional limitations on networking cause some troubles here,
> but I don't know the exact reason. Therefore we skip these two tests
> in such environments (see the two links) but a local build with
> dpkg-buildpackage is obviously not covered yet by that logic.
>
> https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe3d1080/intelmq/tests/bots/collectors/tcp/test_collector.py#L68
> https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe3d1080/intelmq/lib/test.py#L102
>
> Do you know if there's some env variable set which we could check on?
> Then we can add that to the skip_build_environment definition.
>
> Hope that helps
>
> Sebastian
>
> On 10/5/21 10:55 AM, Mika Silander wrote:
>
>     Hi,
>
>      Tried building the intelmq debian packages on a Ubuntu 18 LTS, python 3.6 with
>
>     dpkg-buildpackage -us -uc -b -d
>
>      Branch was maintenance, tag 3.0.2 and the option -d above was added because Cerberus 1.3.2 was installed with pip3
>     and not with apt from python3-cerberus. Seems two tests make the build fail, see below.
>
>      A known problem? Any hints for troubleshooting this? Airing this here before starting to debug on my own ...
>
>     Cheers, Mika
>
>     ======================================================================
>     FAIL: test_multiple_bots (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot)
>     Let's simulate multiple IntelMQ instances are pushing the events at once!
>     ----------------------------------------------------------------------
>     Traceback (most recent call last):
>       File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 175, in test_multiple_bots
>         self.run_bot(iterations=client_count + 1)
>       File "/home/mika/intelmq/intelmq/lib/test.py", line 361, in run_bot
>         self.assertLessEqual(len(re.findall(' - ERROR - ', self.loglines_buffer)), allowed_error_count)
>     AssertionError: 2 not less than or equal to 0
>
>     ======================================================================
>     FAIL: test_random_input (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot)
>     Check how we handle a random input, coming from an unknown source. We should put all the data to report['raw'].
>     ----------------------------------------------------------------------
>     Traceback (most recent call last):
>       File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 106, in test_random_input
>         self.run_bot()
>       File "/home/mika/intelmq/intelmq/lib/test.py", line 363, in run_bot
>         self.assertLessEqual(len(re.findall(' - WARNING - ', self.loglines_buffer)), allowed_warning_count)
>     AssertionError: 1 not less than or equal to 0
>
>     ----------------------------------------------------------------------
>     Ran 1404 tests in 125.121s
>
>     FAILED (failures=2, skipped=274)
>     Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2>
>     error: Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2>
>     E: pybuild pybuild:336: test: plugin distutils failed with: exit code=1: python3.6 setup.py test 
>     dh_auto_test: pybuild --test -i python{version} -p 3.6 returned exit code 13
>     debian/rules:23: recipe for target 'build' failed
>     make: *** [build] Error 25
>     dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
>     _______________________________________________
>     IntelMQ-dev mailing list
>     https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
>     https://intelmq.readthedocs.io/
>
> -- 
> // Sebastian Wagner <wagner at cert.at> - T: +43 676 898 298 7201
> // CERT Austria - https://www.cert.at/
> // Eine Initiative der nic.at GmbH - https://www.nic.at/
> // Firmenbuchnummer 172568b, LG Salzburg
>
>
> _______________________________________________
> IntelMQ-dev mailing list
> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
> https://intelmq.readthedocs.io/
>
> _______________________________________________
> IntelMQ-dev mailing list
> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
> https://intelmq.readthedocs.io/

-- 
// Sebastian Wagner <wagner at cert.at> - T: +43 676 898 298 7201
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - https://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20211029/894f46bc/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20211029/894f46bc/attachment-0001.sig>

More information about the IntelMQ-dev mailing list