[Intelmq-dev] Destination host in malware feeds

[L. Aaron Kaplan]

> On 12 Apr 2017, at 12:54, Sebastian Wagner <wagner at cert.at> wrote:
> 
> Hi,
> 
> we have this IP vs FQDN problem in some parsers, not only the
> shadowserver. Stripping the port there can be simply achieved by use a
> conversion function. But the main problem is IP/FQDN.
> 
> Instead of implementing the logic in many parsers we could add this
> "intelligence" in the libs.

I am not sure if I like that approach.
Usually the particularities of the "messiness" are best placed in the parser.
Even if the logic repeats itself a bit amongst different parsers.
We could of course have a function in lib/ to clear this up, but then each parser which thinks it needs that cleanup part must call the cleanup function in lib/.
But: other parsers MUST NOT call that cleanup function.

Because the http host dest fields might contain totally different (crap) in other feeds.
So...
I would *not* try to impose a default behaviour for all parsers here.

I believe the shadowserver parser should be extended in a way as Thomas suggested.

> One possibility: If the parser tries to add an FQDN as IP, save the IP.
> But I don't like this simple approach as this implicitness raises other
> problems.
yup

> Other possibility: Use a new "logic" (actually non-existing) field, e.g.
> `destination.host-info`,
how about calling it destination.http_host ?

> same applies to source. If some data is added
> to this field, the data will be parsed and added to ip, fqdn, port
> (,network?)
> 
> Example 1:
> event['destination.host-info'] = 'example.com:8080'
> results in:
> {'destination.fqdn': 'example.com', 'destination.port': 8080}
> Example2:
> event['destination.host-info'] = '10.0.0.1'
> results in:
> {'source.ip': '10.0.0.1'}
> 
but again, if you have destination.http_host there, then again it would make sense to parse it and put the info into destination.ip, destination.port etc, ...

> Sebastian
> 
> --
> // Sebastian Wagner <wagner at cert.at> - T: +43 1 50564167201
> // CERT Austria - https://www.cert.at/
> // Eine Initiative der nic.at GmbH - https://www.nic.at/
> // Firmenbuchnummer 172568b, LG Salzburg
> 
> 
> _______________________________________________
> Intelmq-dev mailing list
> Intelmq-dev at lists.cert.at
> http://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev


--
// L. Aaron Kaplan <kaplan at cert.at> - T: +43 1 5056416 78
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - http://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg






-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20170608/80d057c1/attachment.sig>