[Intelmq-dev] usage of extra field made easier

[Sebastian Wagner]

Dear contributors,

using the extra.* fields was not so easy and could easily lead to errors
with overwriting[1]. In PR#1046[2] I proposed a new behavior for this field.

This introduces a new harmonization type 'JSONDict' which is the same as
'JSON' but requires the content to be a dictionary.
The harmonization now allows "subfields" as they are used by the new
'JSONDict' type, including validity checks for the values.
The message library now handles any read/write access to extra.*
transparently, no workarounds are needed. That's the main feature.
The field is properly exploded, see bug #791[3].
All bots currently present in certtools/develop are converted to use the
new simpler behavior, including the tests.

This change requires backwards compatibility:
Bots can still use the old interfaces (reading and writing the whole
field), that behavior should either be removed in 2.0 or be made
consistent for all hierarchical fields.
Old configurations (type of extra is JSON) still work, but raises a
warning in the `intelmqctl check` tool. This behavior will be removed in
2.0 too.
So compatibility with 1.0 is achieved, both for code and configuration.

Do you see any issues with this approach?

Sebastian

[1]: For example here:
https://github.com/certtools/intelmq/pull/1004#discussion_r121417044
[2]: https://github.com/certtools/intelmq/pull/1046
[3]: https://github.com/certtools/intelmq/issues/791

-- 
// Sebastian Wagner <wagner at cert.at> - T: +43 1 5056416 7201
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - https://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20170816/6aef6926/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20170816/6aef6926/attachment.sig>