[IntelMQ-dev] config format (Is: IEP01: IntelMQ Configuration Handling)

[L. Aaron Kaplan]

> On 15.12.2020, at 17:12, Filip Pokorny <filip.pokorny at csirt.cz> wrote:
> 
> Signed PGP part
> Hi everyone,
> 
>> So the question behind this is the weight of the different use cases.
>> Personally I'll find the wireing of a graph easier in an editor
>> and IntelMQ Manager will certainly used by a number of people for this.
>> So JSON is a good, modern fit for IntelMQ.
>> I also consider it okay to write with a text editor.
> 
> I believe people using IntelMQ Manager for wiring the graph don't really
> need to care what kind of configuration format is used. And for those
> (like me) using the text editor YAML feels superior.

I agree here with Filip.

The point I think is that the current config is way too complex and not even
intrinsically allows for (natural) comments. YAML would.
It would be one particular way of expressing meaning and comments
and instructions to people editing config files.
JSON comments as described are a bit of a clutch/work-around.

I just want to add my main point:
I don't really care which config language is used (YAML, etc)
as long as we

* reduce complexity (i.e. I don't want to read a very long config file)
* follow the KISS principle (keep it simple and stupid)
* we can explain things there via comments.


So, YAML is just an idea.
We could also go for something else.

But in the past, the very long JSON (no comments) format has been , well..
a bit cumbersome.

How do other projects do their config language for large and complex configs?
conf.d/ style directories are one way to address the problem
Any other ideas on how to reduce complexity?

Best,
a.


> 
>> _If_ there is a different format to be chosen because of the sum of the weight
>> of the text file and out of band comments use cases, I advise against YAML.
>> YAML is too complicated, which makes it hard to write and parse correctly by
>> tool and humans. (The Python proposal lists the problems with the format.)
>> Of all presented options (YAML,INI,TOML) I'd go with TOML.
> 
> I agree that YAML is complicated and it's features introduced security
> issues in the past. I would like to suggest strictyaml for consideration
> ( https://hitchdev.com/strictyaml/ ) which removes a lot of the
> complicated stuff and preserves comments across read/write operations.
> The project seems active. For me YAML is much easier to read and write
> than JSON or TOML, but this is obviously a very subjective matter.
> 
> As for the Storage proposal, I generally like it. With the multiple "-c"
> option it might get confusing really quickly so I would also suggest an
> option for printing the final configuration.
> 
> Best Regards,
> 
> Filip Pokorny
> CSIRT.CZ
> 
> On 12/14/20 10:42 AM, Bernhard Reiter wrote:
>> Hi Birger,
>> 
>> Am Donnerstag 10 Dezember 2020 13:17:45 schrieb Birger Schacht:
>>> # IntelMQ Configuration Handling (IntelMQ Enhancement Proposal 01)
>>> ## Format
>> 
>>> The downside of JSON is, that is is hard to read
>>> and and write for humans and it cannot contain comments.
>> 
>> JSON can contain comments, if they are part of the data itself, e.g.
>> { "parameter1": true,
>>  "parameter1-comment": "better to have this enabled ;)"
>> }
>> or
>> [  { "comment":"this really should be considered 2020-12-14ber",
>>     "param1":false },
>>   { "param2":"Bernhard", "comment:"my name in 2020" }
>> ]
>> 
>> this is a good thing, if data is to be handled mainly by tools and frontends.
>> Because otherwise the comments are not accessible or visible there.
>> 
>> It is a drawback for workflows where text files are mainly worked upon
>> manually, saved, delopyed and diffed with SCM like tools or text editors.
>> 
>> So the question behind this is the weight of the different use cases.
>> Personally I'll find the wireing of a graph easier in an editor
>> and IntelMQ Manager will certainly used by a number of people for this.
>> So JSON is a good, modern fit for IntelMQ.
>> I also consider it okay to write with a text editor.
>> 
>> _If_ there is a different format to be chosen because of the sum of the weight
>> of the text file and out of band comments use cases, I advise against YAML.
>> YAML is too complicated, which makes it hard to write and parse correctly by
>> tool and humans. (The Python proposal lists the problems with the format.)
>> Of all presented options (YAML,INI,TOML) I'd go with TOML.
>> 
>> Best Regards,
>> Bernhard
>> 
>> 
>> _______________________________________________
>> IntelMQ-dev mailing list
>> IntelMQ-dev at lists.cert.at
>> https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
>> 
> <OpenPGP_0x8C1607AE1371C607.asc>
> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20201215/a9fc16e2/attachment.sig>