<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Consolas","serif";
color:black;
mso-fareast-language:EN-US;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-AU" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:windowtext">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">And: what use cases do we have?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#002060">My particular use case at the moment is to have lists of IP addresses, IP networks and possibly FQDN’s.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#002060"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">How to define the types of the values inside the list?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#002060">The values will be those that conform to IPAddress, IPNetwork and FQDN for their respective type. It could be represented as a vertical bar or comma separated list within a string or it could be a proper python
list.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">How should the "API" look like<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#002060">The API should function as a regular python list. That being said, I don’t imagine doing any complex operations with the list – I will have access to all the values within the parser and will be able to add them
all to the event at once. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#002060"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">When should the list be converted to a string (or maybe also a JSON-list)?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#002060">My main usage will be outputting the events to Mongo – in that case a JSON-list will work. But overall I am happy to use strings to represent the list for all outputs if it makes it easier. I can simply split
the values out after receiving the event on the other end. <o:p></o:p></span></p>
<div style="mso-element:para-border-div;border:none;border-bottom:solid windowtext 1.0pt;padding:0cm 0cm 1.0pt 0cm">
<p class="MsoNormal" style="border:none;padding:0cm"><span style="color:windowtext"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#002060">My end use case is marking up the events as indicators in STIX. One of the teams most vital sources will have many source IPs/Networks/FQDNs per indicator, and thus I would like to be able to send a list of these
values as one event.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#002060"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">Alex<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:EN-AU">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:EN-AU">
Sebastian Wagner [mailto:wagner@cert.at] <br>
<b>Sent:</b> Wednesday, 8 November 2017 10:59 PM<br>
<b>To:</b> Knight, Alexander; intelmq-dev@lists.cert.at<br>
<b>Subject:</b> Re: [Intelmq-dev] Data Harmonization - Fields with multiple values<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi,<br>
<br>
On 11/03/2017 06:26 AM, Knight, Alexander wrote:<br>
<br>
<o:p></o:p></p>
<p class="MsoNormal">At the Deepsec conference Sebastian mentioned updating the harmonization to allow for fields with multiple values. Has this issue been progressed at all?<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif";mso-fareast-language:EN-AU">The use case was the field abuse_contact which could be a list and then be concatenated (if necessary) with commas.<br>
Technically it is not hard to do it. In the develop branch I already have something similar (and more complex): a dictionary type named JSONDict.<br>
So, not directly, but some changes that should make a change easier.<br>
<br>
There are some questions popping up that need to be clarified first:<br>
* How to define the types of the values inside the list? E.g. for the abuse_contact it has to be a list of strings/email addresses<br>
* How should the "API" look like, or in other words: what should happen for the in and setitem-operations etc<br>
* When should the list be converted to a string (or maybe also a JSON-list)? E.g. for postgres output the abuse_contact could either be a json-list or a comma separated list, depending on the table's definition, but for NoSQL-databases and files it can be just
the list itself.<br>
<br>
And: what use cases do we have? That's good to know before thinking about how we implement that all:<br>
<br>
<o:p></o:p></span></p>
<p class="MsoNormal">We will require multiple values for some fields in our events,<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif";mso-fareast-language:EN-AU">What is in these fields? (type and/or example values) Where do you put that that and how do you want to work with in (inside intelmq)?<br>
<br>
I'd like to hear opinions of other users and developers too!<br>
<br>
Sebastian<br>
P.S.: I do have specific ideas, but don't want to bias others ;)<br>
<br>
<o:p></o:p></span></p>
<pre>-- <o:p></o:p></pre>
<pre>// Sebastian Wagner <a href="mailto:wagner@cert.at"><wagner@cert.at></a> - T: +43 1 5056416 7201<o:p></o:p></pre>
<pre>// CERT Austria - <a href="https://www.cert.at/">https://www.cert.at/</a><o:p></o:p></pre>
<pre>// Eine Initiative der nic.at GmbH - <a href="https://www.nic.at/">https://www.nic.at/</a><o:p></o:p></pre>
<pre>// Firmenbuchnummer 172568b, LG Salzburg<o:p></o:p></pre>
</div>
"This e-mail and any attachments to it (the "Communication") is, unless otherwise stated, confidential, may contain copyright material and is for the use only of the intended recipient. If you receive the Communication in error, please notify the sender immediately
by return e-mail, delete the Communication and the return e-mail, and do not read, copy, retransmit or otherwise deal with it. Any views expressed in the Communication are those of the individual sender only, unless expressly stated to be those of Australia
and New Zealand Banking Group Limited ABN 11 005 357 522, or any of its related entities including ANZ Bank New Zealand Limited (together "ANZ"). ANZ does not accept liability in connection with the integrity of or errors in the Communication, computer virus,
data corruption, interference or delay arising from or in respect of the Communication."
</body>
</html>