[IntelMQ-dev] A suitable collector for the Abusech Feodo Tracker feed

[Sebix]

Dear Mika

You can find the documentation for the HTTP collector (aka Generic URL 
Fetcher) here:
https://docs.intelmq.org/latest/user/bots/#generic-url-fetcher
and especially the paragraph on the parameter extract_files.

 > Not 100% sure but it looks to me that collector_http.py for example, 
expects the incoming data to be in zip format since in the sources one 
can see unzipping being done. Correct?

No, the collector does expect zipped data unless the parameter 
extract_files is in use.
What you may have mis-interpreted in the source code is the automatic 
unzipping if the input data is a valid zip-archive.
btw: extract_files works with zip, gzip (gz), tar and tar.gz archives.

 > Therefore, my question was, what would be the recommended collector 
to be used to push reports to the Abusech Feodo Tracker parser?

The correct parser to use the Abusech Feodo Tracker feed, is the HTTP 
collector/Generic URL Fetcher.
See also our feed documentation: 
https://docs.intelmq.org/latest/user/feeds/#feodo-tracker

Hope that helps

Sebastian

Institute for Common Good Technology
gemeinnütziger Kulturverein - nonprofit cultural society
https://commongoodtechnology.org/
ZVR 1510673578

On 12/16/24 12:37 PM, Mika Silander wrote:
> Hi Sebastian, all,
>
>  Seems I rushed when sending out a message to the list (once again, I 
> shouldn't have). Yes, I checked the feed's current contents after 
> clicking "send" and, as you said, there were no events.
> What comes to my comment on http collectors manipulating data, a 
> better wording would have been "the http collectors make assumptions 
> on the structure of the incoming data". Not 100% sure but it looks to 
> me that collector_http.py for example, expects the incoming data to be 
> in zip format since in the sources one can see unzipping being done. 
> Correct?
>
>  I hadn't tried to fetch the ipblocklist.json with any collector yet 
> since I thought the collector_http.py would not be suitable due to the 
> unzipping. Therefore, my question was, what would be the recommended 
> collector to be used to push reports to the Abusech Feodo Tracker 
> parser? I expect the parser to be fine as long as its incoming reports 
> are plain JSON(?)
>
>  If you hear something concerning the Feodo tracker feed, please let 
> me know. Meanwhile, I'll look for other candidate sources for vuln info.
>
> Br, Mika
>
> ------------------------------------------------------------------------
> *From: *"Sebix" <sebix at sebix.at>
> *To: *"Mika Silander" <mika.silander at csc.fi>, "intelmq-dev" 
> <intelmq-dev at lists.cert.at>
> *Sent: *Friday, 13 December, 2024 19:48:23
> *Subject: *Re: [IntelMQ-dev] A suitable collector for the Abusech 
> Feodo Tracker feed
>
> On 12/13/24 6:38 PM, Sebix wrote:
>
>     On 12/13/24 1:29 PM, Mika Silander via IntelMQ-dev wrote:
>
>           I'm attempting to find a suitable collector for retrieving
>         the Abusech Feodo Tracker feed
>         (https://feodotracker.abuse.ch/downloads/ipblocklist.json).
>         Afaiks, the ready-made Abusech Feodo Tracker parser expects
>         reports in plain JSON but the available http collectors are
>         manipulating the retrieved information in one way or the other
>         before passing it on to the parser.
>
>
>     Not sure what you mean with the http collector data manipulation,
>     but to me it appears that the feodotracker is either dysfunctional
>     or dead. Not one of the data feed files contains actual data.
>
> Never mind, the other feeds are empty because there's simply no data.😇️
>
> Parsing the mentioned
> https://feodotracker.abuse.ch/downloads/ipblocklist.json
> works fine with
> intelmq.bots.parsers.abusech.parser_feodotracker
> as documented in https://docs.intelmq.org/latest/user/feeds/#feodo-tracker
>
> Could you please describe what erroneous behavior you see?
>
> best regards
> Sebastian
>
> -- 
> Institute for Common Good Technology
> gemeinnütziger Kulturverein - nonprofit cultural society
> https://commongoodtechnology.org/
> ZVR 1510673578
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-dev/attachments/20241216/61c117f3/attachment.htm>