<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body>
<p>Hi Aaron,</p>
<p>Before you start something new, let me add that we already
created a bunch of vagrant+ansible scripts to test that deb/rpm
packages work, including the API and the Manager. As you propose,
it already includes configuring and starting bots and checking if
they are running. Initially it was meant as just a test, and it
got bigger and more complete over the last months. Now even
package upgrades (2.2 -> 2.3) are tested. All of that needs to
be done locally as of now, but that is already sufficient for our
purposes and probably it can be integrated in GitHub Actions.<br>
</p>
<p>The repository containing everything is currently located at
<a class="moz-txt-link-freetext" href="https://github.com/schacht-certat/intelmq-vagrant">https://github.com/schacht-certat/intelmq-vagrant</a><br>
You can find the tests as ansible scripts in ansible/tasks/<br>
As the README states, it is still very young and alpha, but it
works well. Any ideas how to proceed with it and any contributions
are welcome.<br>
</p>
<p>Many thanks to Birger for creating all that! :)</p>
<p>best regards<br>
Sebastian<br>
</p>
<div class="moz-cite-prefix">On 2/19/21 12:32 PM, L. Aaron Kaplan
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:8B1B2867-77BA-4A97-8B37-770A07FBE4E4@lo-res.org">
<pre class="moz-quote-pre" wrap="">Hi Mika,
this is an old idea which I discussed with Sebastian in ~ 2014 (?) or 2015 already.
Somehow it never got implemented. But I agree, it's really important to test correctness.
I think what you would need to do is the following:
1) define a pipeline which has as collector some file collectors.
The input shall come via a file
2) define the run parameters for this
3) define filters (filter out certain fields from the events before storing)
4) store the results of the flow into a known output file (file output bot)
5) compare these output file with known good output files.
6) return error on diff
I would say we could call this a "unit-test-flow"
I think this could be easily implemented as a script / docker image which gets deployed via CI/CD.
Gitlab supports this approach. I guess gitHUB supports it as well.
How about writing a small proposal for this together?
Then we can put this proposal in an issue and see that it gets implemented.
Best,
a.
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">On 19.02.2021, at 12:25, Mika Silander <a class="moz-txt-link-rfc2396E" href="mailto:mika.silander@csc.fi"><mika.silander@csc.fi></a> wrote:
Hi,
While writing tests to individual bots seems quite straightforward, what would be the recommended way for writing (unit?) tests for a chain of bots? Ideally, I'd like to feed in individual test events to the first bot and only check what is returned by the last bot (event or whatever the expected outcome should be).
I suppose there are many bad and clumsy ways of doing this so that's why I air this question on the dev list.
Thanks for all help in advance, Mika
_______________________________________________
IntelMQ-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:IntelMQ-dev@lists.cert.at">IntelMQ-dev@lists.cert.at</a>
<a class="moz-txt-link-freetext" href="https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev">https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev</a>
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
IntelMQ-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:IntelMQ-dev@lists.cert.at">IntelMQ-dev@lists.cert.at</a>
<a class="moz-txt-link-freetext" href="https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev">https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
// Sebastian Wagner <a class="moz-txt-link-rfc2396E" href="mailto:wagner@cert.at"><wagner@cert.at></a> - T: +43 1 5056416 7201
// CERT Austria - <a class="moz-txt-link-freetext" href="https://www.cert.at/">https://www.cert.at/</a>
// Eine Initiative der nic.at GmbH - <a class="moz-txt-link-freetext" href="https://www.nic.at/">https://www.nic.at/</a>
// Firmenbuchnummer 172568b, LG Salzburg</pre>
</body>
</html>