Hi,
Tried building the intelmq debian packages on a Ubuntu 18 LTS, python 3.6 with
dpkg-buildpackage -us -uc -b -d
Branch was maintenance, tag 3.0.2 and the option -d above was added because Cerberus 1.3.2 was installed with pip3 and not with apt from python3-cerberus. Seems two tests make the build fail, see below.
A known problem? Any hints for troubleshooting this? Airing this here before starting to debug on my own ...
Cheers, Mika
====================================================================== FAIL: test_multiple_bots (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Let's simulate multiple IntelMQ instances are pushing the events at once! ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 175, in test_multiple_bots self.run_bot(iterations=client_count + 1) File "/home/mika/intelmq/intelmq/lib/test.py", line 361, in run_bot self.assertLessEqual(len(re.findall(' - ERROR - ', self.loglines_buffer)), allowed_error_count) AssertionError: 2 not less than or equal to 0
====================================================================== FAIL: test_random_input (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Check how we handle a random input, coming from an unknown source. We should put all the data to report['raw']. ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 106, in test_random_input self.run_bot() File "/home/mika/intelmq/intelmq/lib/test.py", line 363, in run_bot self.assertLessEqual(len(re.findall(' - WARNING - ', self.loglines_buffer)), allowed_warning_count) AssertionError: 1 not less than or equal to 0
---------------------------------------------------------------------- Ran 1404 tests in 125.121s
FAILED (failures=2, skipped=274) Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> error: Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> E: pybuild pybuild:336: test: plugin distutils failed with: exit code=1: python3.6 setup.py test dh_auto_test: pybuild --test -i python{version} -p 3.6 returned exit code 13 debian/rules:23: recipe for target 'build' failed make: *** [build] Error 25 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
Dear Mika,
These two tests are known to fail in build environments. I guess that their intentional limitations on networking cause some troubles here, but I don't know the exact reason. Therefore we skip these two tests in such environments (see the two links) but a local build with dpkg-buildpackage is obviously not covered yet by that logic.
https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe...
Do you know if there's some env variable set which we could check on? Then we can add that to the skip_build_environment definition.
Hope that helps
Sebastian
On 10/5/21 10:55 AM, Mika Silander wrote:
Hi,
Tried building the intelmq debian packages on a Ubuntu 18 LTS, python 3.6 with
dpkg-buildpackage -us -uc -b -d
Branch was maintenance, tag 3.0.2 and the option -d above was added because Cerberus 1.3.2 was installed with pip3 and not with apt from python3-cerberus. Seems two tests make the build fail, see below.
A known problem? Any hints for troubleshooting this? Airing this here before starting to debug on my own ...
Cheers, Mika
====================================================================== FAIL: test_multiple_bots (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Let's simulate multiple IntelMQ instances are pushing the events at once!
Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 175, in test_multiple_bots self.run_bot(iterations=client_count + 1) File "/home/mika/intelmq/intelmq/lib/test.py", line 361, in run_bot self.assertLessEqual(len(re.findall(' - ERROR - ', self.loglines_buffer)), allowed_error_count) AssertionError: 2 not less than or equal to 0
====================================================================== FAIL: test_random_input (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Check how we handle a random input, coming from an unknown source. We should put all the data to report['raw'].
Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 106, in test_random_input self.run_bot() File "/home/mika/intelmq/intelmq/lib/test.py", line 363, in run_bot self.assertLessEqual(len(re.findall(' - WARNING - ', self.loglines_buffer)), allowed_warning_count) AssertionError: 1 not less than or equal to 0
Ran 1404 tests in 125.121s
FAILED (failures=2, skipped=274) Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> error: Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> E: pybuild pybuild:336: test: plugin distutils failed with: exit code=1: python3.6 setup.py test dh_auto_test: pybuild --test -i python{version} -p 3.6 returned exit code 13 debian/rules:23: recipe for target 'build' failed make: *** [build] Error 25 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 _______________________________________________ IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://intelmq.readthedocs.io/
Hi Sebastian,
Thanks for your quick reply. I'm not familiar with the nitty-gritty details of the automated builds vs. local builds but I don't see environment variables as a reliable method for separating these two cases. One option that comes to mind is trying to deduce whether dpkg-buildpackage is run from a terminal or not and implement this test inside the skip_build_environment method in [ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... ] . I'm probably unaware of other constraints and details affecting this, so let me know if this is a bad idea.
Br, MIka
From: "Sebastian Wagner" wagner@cert.at To: "Mika Silander" mika.silander@csc.fi, "intelmq-dev" intelmq-dev@lists.cert.at Sent: Tuesday, 5 October, 2021 12:09:16 Subject: Re: [IntelMQ-dev] Building packages for intelmq 3.0.2 stumbles on two tests
Dear Mika,
These two tests are known to fail in build environments. I guess that their intentional limitations on networking cause some troubles here, but I don't know the exact reason. Therefore we skip these two tests in such environments (see the two links) but a local build with dpkg-buildpackage is obviously not covered yet by that logic.
[ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... ] [ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... ]
Do you know if there's some env variable set which we could check on? Then we can add that to the skip_build_environment definition.
Hope that helps
Sebastian On 10/5/21 10:55 AM, Mika Silander wrote:
Hi,
Tried building the intelmq debian packages on a Ubuntu 18 LTS, python 3.6 with
dpkg-buildpackage -us -uc -b -d
Branch was maintenance, tag 3.0.2 and the option -d above was added because Cerberus 1.3.2 was installed with pip3 and not with apt from python3-cerberus. Seems two tests make the build fail, see below.
A known problem? Any hints for troubleshooting this? Airing this here before starting to debug on my own ...
Cheers, Mika
====================================================================== FAIL: test_multiple_bots (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Let's simulate multiple IntelMQ instances are pushing the events at once! ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 175, in test_multiple_bots self.run_bot(iterations=client_count + 1) File "/home/mika/intelmq/intelmq/lib/test.py", line 361, in run_bot self.assertLessEqual(len(re.findall(' - ERROR - ', self.loglines_buffer)), allowed_error_count) AssertionError: 2 not less than or equal to 0
====================================================================== FAIL: test_random_input (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Check how we handle a random input, coming from an unknown source. We should put all the data to report['raw']. ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 106, in test_random_input self.run_bot() File "/home/mika/intelmq/intelmq/lib/test.py", line 363, in run_bot self.assertLessEqual(len(re.findall(' - WARNING - ', self.loglines_buffer)), allowed_warning_count) AssertionError: 1 not less than or equal to 0
---------------------------------------------------------------------- Ran 1404 tests in 125.121s
FAILED (failures=2, skipped=274) Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> error: Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> E: pybuild pybuild:336: test: plugin distutils failed with: exit code=1: python3.6 setup.py test dh_auto_test: pybuild --test -i python{version} -p 3.6 returned exit code 13 debian/rules:23: recipe for target 'build' failed make: *** [build] Error 25 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 _______________________________________________ IntelMQ-dev mailing list [ https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev | https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev ] [ https://intelmq.readthedocs.io/ | https://intelmq.readthedocs.io/ ]
Hi,
Answering myself here: I got the below going by checking the availability of the terminal inside skip_build_environment, so manual builds from a CLI with dpkg-buildpackage seem to succeed for branch maintenance, tag 3.0.2. Build stats included 1404 tests run, 286 skipped ok (for various reasons). Let me know if those numbers are not reasonable.
Br, Mika
From: "Mika Silander" mika.silander@csc.fi To: "intelmq-dev" intelmq-dev@lists.cert.at Sent: Tuesday, 5 October, 2021 12:25:50 Subject: Re: [IntelMQ-dev] Building packages for intelmq 3.0.2 stumbles on two tests
Hi Sebastian,
Thanks for your quick reply. I'm not familiar with the nitty-gritty details of the automated builds vs. local builds but I don't see environment variables as a reliable method for separating these two cases. One option that comes to mind is trying to deduce whether dpkg-buildpackage is run from a terminal or not and implement this test inside the skip_build_environment method in [ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... ] . I'm probably unaware of other constraints and details affecting this, so let me know if this is a bad idea.
Br, MIka
From: "Sebastian Wagner" wagner@cert.at To: "Mika Silander" mika.silander@csc.fi, "intelmq-dev" intelmq-dev@lists.cert.at Sent: Tuesday, 5 October, 2021 12:09:16 Subject: Re: [IntelMQ-dev] Building packages for intelmq 3.0.2 stumbles on two tests
Dear Mika,
These two tests are known to fail in build environments. I guess that their intentional limitations on networking cause some troubles here, but I don't know the exact reason. Therefore we skip these two tests in such environments (see the two links) but a local build with dpkg-buildpackage is obviously not covered yet by that logic.
[ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... ] [ https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... | https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... ]
Do you know if there's some env variable set which we could check on? Then we can add that to the skip_build_environment definition.
Hope that helps
Sebastian On 10/5/21 10:55 AM, Mika Silander wrote:
Hi,
Tried building the intelmq debian packages on a Ubuntu 18 LTS, python 3.6 with
dpkg-buildpackage -us -uc -b -d
Branch was maintenance, tag 3.0.2 and the option -d above was added because Cerberus 1.3.2 was installed with pip3 and not with apt from python3-cerberus. Seems two tests make the build fail, see below.
A known problem? Any hints for troubleshooting this? Airing this here before starting to debug on my own ...
Cheers, Mika
====================================================================== FAIL: test_multiple_bots (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Let's simulate multiple IntelMQ instances are pushing the events at once! ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 175, in test_multiple_bots self.run_bot(iterations=client_count + 1) File "/home/mika/intelmq/intelmq/lib/test.py", line 361, in run_bot self.assertLessEqual(len(re.findall(' - ERROR - ', self.loglines_buffer)), allowed_error_count) AssertionError: 2 not less than or equal to 0
====================================================================== FAIL: test_random_input (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Check how we handle a random input, coming from an unknown source. We should put all the data to report['raw']. ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 106, in test_random_input self.run_bot() File "/home/mika/intelmq/intelmq/lib/test.py", line 363, in run_bot self.assertLessEqual(len(re.findall(' - WARNING - ', self.loglines_buffer)), allowed_warning_count) AssertionError: 1 not less than or equal to 0
---------------------------------------------------------------------- Ran 1404 tests in 125.121s
FAILED (failures=2, skipped=274) Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> error: Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> E: pybuild pybuild:336: test: plugin distutils failed with: exit code=1: python3.6 setup.py test dh_auto_test: pybuild --test -i python{version} -p 3.6 returned exit code 13 debian/rules:23: recipe for target 'build' failed make: *** [build] Error 25 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 _______________________________________________ IntelMQ-dev mailing list [ https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev | https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev ] [ https://intelmq.readthedocs.io/ | https://intelmq.readthedocs.io/ ]
In the Ubuntu 18.04 package builds 1404 tests are running, with skipped=285 (https://build.opensuse.org/public/build/home:sebix:intelmq/xUbuntu_18.04/x86... at 617s), so the numbers are perfectly legit.
To detect if the tests are running inside the dpkg-buildpackage environment, I extended the detection logic by checking for DEB_BUILD_ARCH: https://github.com/certtools/intelmq/commit/7ebb8e16d821c372a44b077dd18a151c...
Sebastian
On 10/5/21 2:31 PM, Mika Silander wrote:
Hi,
Answering myself here: I got the below going by checking the availability of the terminal inside skip_build_environment, so manual builds from a CLI with dpkg-buildpackage seem to succeed for branch maintenance, tag 3.0.2. Build stats included 1404 tests run, 286 skipped ok (for various reasons). Let me know if those numbers are not reasonable.
Br, Mika
*From: *"Mika Silander" mika.silander@csc.fi *To: *"intelmq-dev" intelmq-dev@lists.cert.at *Sent: *Tuesday, 5 October, 2021 12:25:50 *Subject: *Re: [IntelMQ-dev] Building packages for intelmq 3.0.2 stumbles on two tests
Hi Sebastian,
Thanks for your quick reply. I'm not familiar with the nitty-gritty details of the automated builds vs. local builds but I don't see environment variables as a reliable method for separating these two cases. One option that comes to mind is trying to deduce whether dpkg-buildpackage is run from a terminal or not and implement this test inside the skip_build_environment method in https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe.... I'm probably unaware of other constraints and details affecting this, so let me know if this is a bad idea.
Br, MIka
*From: *"Sebastian Wagner" wagner@cert.at *To: *"Mika Silander" mika.silander@csc.fi, "intelmq-dev" intelmq-dev@lists.cert.at *Sent: *Tuesday, 5 October, 2021 12:09:16 *Subject: *Re: [IntelMQ-dev] Building packages for intelmq 3.0.2 stumbles on two tests
Dear Mika,
These two tests are known to fail in build environments. I guess that their intentional limitations on networking cause some troubles here, but I don't know the exact reason. Therefore we skip these two tests in such environments (see the two links) but a local build with dpkg-buildpackage is obviously not covered yet by that logic.
https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe... https://github.com/certtools/intelmq/blob/5962fa9efb1c9d1ae454cc85d4aa05ddfe...
Do you know if there's some env variable set which we could check on? Then we can add that to the skip_build_environment definition.
Hope that helps
Sebastian
On 10/5/21 10:55 AM, Mika Silander wrote:
Hi, Tried building the intelmq debian packages on a Ubuntu 18 LTS, python 3.6 with dpkg-buildpackage -us -uc -b -d Branch was maintenance, tag 3.0.2 and the option -d above was added because Cerberus 1.3.2 was installed with pip3 and not with apt from python3-cerberus. Seems two tests make the build fail, see below. A known problem? Any hints for troubleshooting this? Airing this here before starting to debug on my own ... Cheers, Mika ====================================================================== FAIL: test_multiple_bots (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Let's simulate multiple IntelMQ instances are pushing the events at once! ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 175, in test_multiple_bots self.run_bot(iterations=client_count + 1) File "/home/mika/intelmq/intelmq/lib/test.py", line 361, in run_bot self.assertLessEqual(len(re.findall(' - ERROR - ', self.loglines_buffer)), allowed_error_count) AssertionError: 2 not less than or equal to 0 ====================================================================== FAIL: test_random_input (intelmq.tests.bots.collectors.tcp.test_collector.TestTCPCollectorBot) Check how we handle a random input, coming from an unknown source. We should put all the data to report['raw']. ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mika/intelmq/intelmq/tests/bots/collectors/tcp/test_collector.py", line 106, in test_random_input self.run_bot() File "/home/mika/intelmq/intelmq/lib/test.py", line 363, in run_bot self.assertLessEqual(len(re.findall(' - WARNING - ', self.loglines_buffer)), allowed_warning_count) AssertionError: 1 not less than or equal to 0 ---------------------------------------------------------------------- Ran 1404 tests in 125.121s FAILED (failures=2, skipped=274) Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> error: Test failed: <unittest.runner.TextTestResult run=1404 errors=0 failures=2> E: pybuild pybuild:336: test: plugin distutils failed with: exit code=1: python3.6 setup.py test dh_auto_test: pybuild --test -i python{version} -p 3.6 returned exit code 13 debian/rules:23: recipe for target 'build' failed make: *** [build] Error 25 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 _______________________________________________ IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://intelmq.readthedocs.io/-- // Sebastian Wagner wagner@cert.at - T: +43 676 898 298 7201 // CERT Austria - https://www.cert.at/ // Eine Initiative der nic.at GmbH - https://www.nic.at/ // Firmenbuchnummer 172568b, LG Salzburg
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://intelmq.readthedocs.io/
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://intelmq.readthedocs.io/