[IntelMQ-users] Custom harmonization configuration in tests

[Kamil Mankowski]

True, you could also - if you had time - propose a change in the base 
BotTestCase class to allow overriding the harmonization loader without 
subclassing. It was my plan, but just don't have time for that at the 
moment.

Best regards

// Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
// CERT Austria - https://www.cert.at/
// CERT.at GmbH, FB-Nr. 561772k, HG Wien

On 2/28/24 09:31, Vaclav Bruzek wrote:
> Hi,
> thanks for the suggestions. So there are two options
> 1. overwrite the file in intelmq/etc/.
> 2. merge the loaded harmonization in the specific test cases with the 
> custom event fields.
> 
> This might be a good addition to the documentation.
> 
> Sincerely,
> Václav Brůžek
> 
> 
> On Wed, 28 Feb 2024 at 09:04, Kamil Mankowski <mankowski at cert.at 
> <mailto:mankowski at cert.at>> wrote:
> 
>     Hi,
> 
>     I overcome such a problem by creating a custom test base, look at those
>     snippets as an example:
> 
>     import intelmq.lib.test as test
> 
>     ADDITIONAL_HARMONIZATION =
>     ["contrib/constituency.harmonization.part.json"]
> 
>     def load_harmonization():
>           harmonization = pkg_resources.resource_filename(
>               "intelmq_extensions", "etc/harmonization.conf"
>           )
>           extensions = []
>           for file in ADDITIONAL_HARMONIZATION:
>               with open(pathlib.Path(__file__).parent.parent.parent /
>     file) as f:
>                   extensions.append(json.load(f))
>           return merge_harmonization(extensions, harmonization)
> 
>     class TestCaseMixin:
>           harmonization = load_harmonization()
> 
>     class BotTestCase(TestCaseMixin, test.BotTestCase):
>           """Provides test class with additional changes required for
>     extension bots"""
> 
>     In my case, I'm merging the default harmonization with additional
>     fields, but you can just load your own.
> 
>     Best regards
> 
>     // Kamil Mańkowski <mankowski at cert.at <mailto:mankowski at cert.at>> -
>     T: +43 676 898 298 7204
>     // CERT Austria - https://www.cert.at/ <https://www.cert.at/>
>     // CERT.at GmbH, FB-Nr. 561772k, HG Wien
> 
>     On 2/28/24 08:52, Sebix wrote:
>      > Dear Václav,
>      >
>      > Which harmonization file did you change?
>      >
>      > The test library loads the harmonization config from the package
>     itself:
>      >
>      >
>     https://github.com/certtools/intelmq/blob/ac6aa4e306b1431b6db89158e25ed4c2c5a356bd/intelmq/lib/test.py#L190-L191 <https://github.com/certtools/intelmq/blob/ac6aa4e306b1431b6db89158e25ed4c2c5a356bd/intelmq/lib/test.py#L190-L191>
>      >
>      >      harmonization =
>      > utils.load_configuration(pkg_resources.resource_filename('intelmq',
>      > 'etc/harmonization.conf'))
>      >
>      > best regards
>      > Sebastian
>      >
>      > Institute for Common Good Technology
>      > gemeinnütziger Kulturverein - nonprofit cultural society
>      > https://commongoodtechnology.org/ <https://commongoodtechnology.org/>
>      > ZVR 1510673578
>      >
>      > On 2/28/24 08:39, Vaclav Bruzek wrote:
>      >> Dear IntelMQ Developers & Users,
>      >> I'm trying to incorporate my custom fields in Events and Reports as
>      >> well as test cases for my custom bots in unit tests. What I¨m still
>      >> encountering is that the Github actions are failing since my custom
>      >> fields in harmonization are not known. I've added step which should
>      >> create the harmonization file in /opt/intelmq/etc/ like so which is
>      >> the only change to the action:
>      >>
>      >>    - name: Include custom harmonization names
>      >>       run: mkdir -p /opt/intelmq/etc/ && cp
>      >> docker/fast/harmonization.conf /opt/intelmq/etc/
>      >>
>      >> However, intelmq still falls back to the default one. If I try
>     to run
>      >>
>     manually load_configuration("/opt/intelmq/etc/harmonization.conf") the
>      >> correct result is displayed.
>      >>
>      >> Does anybody know how to supply the custom configuration to unit
>     tests?
>      >>
>      >> Sincerely,
>      >> Václav Brůžek
>      >>
>      >
> 
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20240228/242f2206/attachment.sig>