[IntelMQ-users] Custom harmonization configuration in tests

[Vaclav Bruzek]

Hi,
I'll try to prepare something.

Sincerely,
Václav Brůžek


On Wed, 28 Feb 2024 at 09:39, Kamil Mankowski <mankowski at cert.at> wrote:

> True, you could also - if you had time - propose a change in the base
> BotTestCase class to allow overriding the harmonization loader without
> subclassing. It was my plan, but just don't have time for that at the
> moment.
>
> Best regards
>
> // Kamil Mańkowski <mankowski at cert.at> - T: +43 676 898 298 7204
> // CERT Austria - https://www.cert.at/
> // CERT.at GmbH, FB-Nr. 561772k, HG Wien
>
> On 2/28/24 09:31, Vaclav Bruzek wrote:
> > Hi,
> > thanks for the suggestions. So there are two options
> > 1. overwrite the file in intelmq/etc/.
> > 2. merge the loaded harmonization in the specific test cases with the
> > custom event fields.
> >
> > This might be a good addition to the documentation.
> >
> > Sincerely,
> > Václav Brůžek
> >
> >
> > On Wed, 28 Feb 2024 at 09:04, Kamil Mankowski <mankowski at cert.at
> > <mailto:mankowski at cert.at>> wrote:
> >
> >     Hi,
> >
> >     I overcome such a problem by creating a custom test base, look at
> those
> >     snippets as an example:
> >
> >     import intelmq.lib.test as test
> >
> >     ADDITIONAL_HARMONIZATION =
> >     ["contrib/constituency.harmonization.part.json"]
> >
> >     def load_harmonization():
> >           harmonization = pkg_resources.resource_filename(
> >               "intelmq_extensions", "etc/harmonization.conf"
> >           )
> >           extensions = []
> >           for file in ADDITIONAL_HARMONIZATION:
> >               with open(pathlib.Path(__file__).parent.parent.parent /
> >     file) as f:
> >                   extensions.append(json.load(f))
> >           return merge_harmonization(extensions, harmonization)
> >
> >     class TestCaseMixin:
> >           harmonization = load_harmonization()
> >
> >     class BotTestCase(TestCaseMixin, test.BotTestCase):
> >           """Provides test class with additional changes required for
> >     extension bots"""
> >
> >     In my case, I'm merging the default harmonization with additional
> >     fields, but you can just load your own.
> >
> >     Best regards
> >
> >     // Kamil Mańkowski <mankowski at cert.at <mailto:mankowski at cert.at>> -
> >     T: +43 676 898 298 7204
> >     // CERT Austria - https://www.cert.at/ <https://www.cert.at/>
> >     // CERT.at GmbH, FB-Nr. 561772k, HG Wien
> >
> >     On 2/28/24 08:52, Sebix wrote:
> >      > Dear Václav,
> >      >
> >      > Which harmonization file did you change?
> >      >
> >      > The test library loads the harmonization config from the package
> >     itself:
> >      >
> >      >
> >
> https://github.com/certtools/intelmq/blob/ac6aa4e306b1431b6db89158e25ed4c2c5a356bd/intelmq/lib/test.py#L190-L191
> <
> https://github.com/certtools/intelmq/blob/ac6aa4e306b1431b6db89158e25ed4c2c5a356bd/intelmq/lib/test.py#L190-L191
> >
> >      >
> >      >      harmonization =
> >      >
> utils.load_configuration(pkg_resources.resource_filename('intelmq',
> >      > 'etc/harmonization.conf'))
> >      >
> >      > best regards
> >      > Sebastian
> >      >
> >      > Institute for Common Good Technology
> >      > gemeinnütziger Kulturverein - nonprofit cultural society
> >      > https://commongoodtechnology.org/ <
> https://commongoodtechnology.org/>
> >      > ZVR 1510673578
> >      >
> >      > On 2/28/24 08:39, Vaclav Bruzek wrote:
> >      >> Dear IntelMQ Developers & Users,
> >      >> I'm trying to incorporate my custom fields in Events and Reports
> as
> >      >> well as test cases for my custom bots in unit tests. What I¨m
> still
> >      >> encountering is that the Github actions are failing since my
> custom
> >      >> fields in harmonization are not known. I've added step which
> should
> >      >> create the harmonization file in /opt/intelmq/etc/ like so which
> is
> >      >> the only change to the action:
> >      >>
> >      >>    - name: Include custom harmonization names
> >      >>       run: mkdir -p /opt/intelmq/etc/ && cp
> >      >> docker/fast/harmonization.conf /opt/intelmq/etc/
> >      >>
> >      >> However, intelmq still falls back to the default one. If I try
> >     to run
> >      >>
> >     manually load_configuration("/opt/intelmq/etc/harmonization.conf")
> the
> >      >> correct result is displayed.
> >      >>
> >      >> Does anybody know how to supply the custom configuration to unit
> >     tests?
> >      >>
> >      >> Sincerely,
> >      >> Václav Brůžek
> >      >>
> >      >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20240228/bcf7d824/attachment.htm>