[IntelMQ-dev] config format (Is: IEP01: IntelMQ Configuration Handling)

[Filip Pokorny]

Hi everyone,

> So the question behind this is the weight of the different use cases.
> Personally I'll find the wireing of a graph easier in an editor
> and IntelMQ Manager will certainly used by a number of people for this.
> So JSON is a good, modern fit for IntelMQ. 
> I also consider it okay to write with a text editor.

I believe people using IntelMQ Manager for wiring the graph don't really
need to care what kind of configuration format is used. And for those
(like me) using the text editor YAML feels superior.

> _If_ there is a different format to be chosen because of the sum of the weight 
> of the text file and out of band comments use cases, I advise against YAML. 
> YAML is too complicated, which makes it hard to write and parse correctly by 
> tool and humans. (The Python proposal lists the problems with the format.)
> Of all presented options (YAML,INI,TOML) I'd go with TOML.

I agree that YAML is complicated and it's features introduced security
issues in the past. I would like to suggest strictyaml for consideration
( https://hitchdev.com/strictyaml/ ) which removes a lot of the
complicated stuff and preserves comments across read/write operations.
The project seems active. For me YAML is much easier to read and write
than JSON or TOML, but this is obviously a very subjective matter.

As for the Storage proposal, I generally like it. With the multiple "-c"
option it might get confusing really quickly so I would also suggest an
option for printing the final configuration.

Best Regards,

Filip Pokorny
CSIRT.CZ

On 12/14/20 10:42 AM, Bernhard Reiter wrote:
> Hi Birger,
> 
> Am Donnerstag 10 Dezember 2020 13:17:45 schrieb Birger Schacht:
>> # IntelMQ Configuration Handling (IntelMQ Enhancement Proposal 01)
>> ## Format
> 
>> The downside of JSON is, that is is hard to read
>> and and write for humans and it cannot contain comments.
> 
> JSON can contain comments, if they are part of the data itself, e.g.
> { "parameter1": true,
>   "parameter1-comment": "better to have this enabled ;)"
> }
> or
> [  { "comment":"this really should be considered 2020-12-14ber",
>      "param1":false },
>    { "param2":"Bernhard", "comment:"my name in 2020" }
> ]
> 
> this is a good thing, if data is to be handled mainly by tools and frontends.
> Because otherwise the comments are not accessible or visible there.
> 
> It is a drawback for workflows where text files are mainly worked upon 
> manually, saved, delopyed and diffed with SCM like tools or text editors.
> 
> So the question behind this is the weight of the different use cases.
> Personally I'll find the wireing of a graph easier in an editor
> and IntelMQ Manager will certainly used by a number of people for this.
> So JSON is a good, modern fit for IntelMQ. 
> I also consider it okay to write with a text editor.
> 
> _If_ there is a different format to be chosen because of the sum of the weight 
> of the text file and out of band comments use cases, I advise against YAML. 
> YAML is too complicated, which makes it hard to write and parse correctly by 
> tool and humans. (The Python proposal lists the problems with the format.)
> Of all presented options (YAML,INI,TOML) I'd go with TOML.
> 
> Best Regards,
> Bernhard
> 
> 
> _______________________________________________
> IntelMQ-dev mailing list
> IntelMQ-dev at lists.cert.at
> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0x8C1607AE1371C607.asc
Type: application/pgp-keys
Size: 4825 bytes
Desc: not available
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20201215/3668e56b/attachment.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20201215/3668e56b/attachment.sig>