[IntelMQ-users] Custom harmonization configuration in tests

[Vaclav Bruzek]

Hi,
thanks for the suggestions. So there are two options
1. overwrite the file in intelmq/etc/.
2. merge the loaded harmonization in the specific test cases with the
custom event fields.

This might be a good addition to the documentation.

Sincerely,
Václav Brůžek


On Wed, 28 Feb 2024 at 09:04, Kamil Mankowski <mankowski at cert.at> wrote:

> Hi,
>
> I overcome such a problem by creating a custom test base, look at those
> snippets as an example:
>
> import intelmq.lib.test as test
>
> ADDITIONAL_HARMONIZATION = ["contrib/constituency.harmonization.part.json"]
>
> def load_harmonization():
>      harmonization = pkg_resources.resource_filename(
>          "intelmq_extensions", "etc/harmonization.conf"
>      )
>      extensions = []
>      for file in ADDITIONAL_HARMONIZATION:
>          with open(pathlib.Path(__file__).parent.parent.parent / file) as
> f:
>              extensions.append(json.load(f))
>      return merge_harmonization(extensions, harmonization)
>
> class TestCaseMixin:
>      harmonization = load_harmonization()
>
> class BotTestCase(TestCaseMixin, test.BotTestCase):
>      """Provides test class with additional changes required for
> extension bots"""
>
> In my case, I'm merging the default harmonization with additional
> fields, but you can just load your own.
>
> Best regards
>
> // Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
> // CERT Austria - https://www.cert.at/
> // CERT.at GmbH, FB-Nr. 561772k, HG Wien
>
> On 2/28/24 08:52, Sebix wrote:
> > Dear Václav,
> >
> > Which harmonization file did you change?
> >
> > The test library loads the harmonization config from the package itself:
> >
> >
> https://github.com/certtools/intelmq/blob/ac6aa4e306b1431b6db89158e25ed4c2c5a356bd/intelmq/lib/test.py#L190-L191
> >
> >      harmonization =
> > utils.load_configuration(pkg_resources.resource_filename('intelmq',
> > 'etc/harmonization.conf'))
> >
> > best regards
> > Sebastian
> >
> > Institute for Common Good Technology
> > gemeinnütziger Kulturverein - nonprofit cultural society
> > https://commongoodtechnology.org/
> > ZVR 1510673578
> >
> > On 2/28/24 08:39, Vaclav Bruzek wrote:
> >> Dear IntelMQ Developers & Users,
> >> I'm trying to incorporate my custom fields in Events and Reports as
> >> well as test cases for my custom bots in unit tests. What I¨m still
> >> encountering is that the Github actions are failing since my custom
> >> fields in harmonization are not known. I've added step which should
> >> create the harmonization file in /opt/intelmq/etc/ like so which is
> >> the only change to the action:
> >>
> >>    - name: Include custom harmonization names
> >>       run: mkdir -p /opt/intelmq/etc/ && cp
> >> docker/fast/harmonization.conf /opt/intelmq/etc/
> >>
> >> However, intelmq still falls back to the default one. If I try to run
> >> manually load_configuration("/opt/intelmq/etc/harmonization.conf") the
> >> correct result is displayed.
> >>
> >> Does anybody know how to supply the custom configuration to unit tests?
> >>
> >> Sincerely,
> >> Václav Brůžek
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20240228/722670c2/attachment.htm>