[Intelmq-dev] Intelmq-dev newletter 201605

[L. Aaron Kaplan]

= Intelmq-dev-news 05-2016

Issue 5/2016

== Topics == 

 # Summary of IHAP meeting in April
 # Status update Intevation
 # Status update CERT.at
 # Status update misc


== May 2016 ==

Dear Intelmq-dev mailing list readers,
  this is the second issue of intelmq developer news.
We hope it's useful. 


TL;DR and important changes
-----------------------------

The syntax of intelmqcli was changed to a new format:

  intelmqctl {start,stop} bot_id.

This breaks compatibilty with existing scripts.
If you put intelmqctl into some script, please adapt it. Also 
please be sure to check out the latest version of the intelmq-manager
in case you use it.


Lots of open issues. Progress with intelmqcli (to connect postgresql to the RT ticket system).

/ TL;DR





=== How to contribute to this newsletter? ===

 -> contact Aaron, Dustin for future input


=== Summary of IHAP meeting in April ===

 In April the IHAP Meeting took place in Vienna.

 * A Hacksession the night before the meeting was used by Raphael and Aaron in 
order to bridge MISP and IntelMQ.
 * Connections between Abusehelper and IntelMQ are on some CERTs wish list. 
XMPP is a good start. Unfortunately the XMPP Bot upstream was not fit for 
production.


=== Status report Intevation === 

 * Still working on the KontaktDB, we appreciate the discussions that started 
on IHAP Meeting.
   We received a Pull Request from Cert.at and are currently reviewing it.
 * We have Scripts to import Data into the KontaktDB.
   Nevertheless there is some work left.
 * Demonstrated installation from packages on Ubuntu 14.04 on IHAP-Meeting.
   We propose to host the **signed** packages on our public apt-repositories.
 * Working on a tool similar to intelmqcli, intended to process events from 
the eventdb. Instead of using RT they are sent by e-mail.
   The tool has the working title "event-processor" and can be found here 
(https://github.com/Intevation/event-processor)
 * We did not start with support for IODEF or X-ARF yet.

=== Status report CERT.at developments ===

* we moved to python3 only. Intelmq dropped python2 support 
(https://github.com/certtools/intelmq/commit/2cbb42f1458a7e90539a443ec5e50eec7a5f1795). 
This does not apply yet to the certat repo (github.com/certat/intelmq), which 
still supports python2.7 but only for the intelmqcli tool. 
* New active contributor: pedro m. reis! Welcome and thanks for working so 
hard on the Bitsight collector 
(https://github.com/certtools/intelmq/pull/493)
* intelmqcli tool now supports a lot of new flags: 
https://github.com/certat/intelmq/issues/52 This was necessary for CERT.at 
since we use intelmqcli via cron job to connect to the (postgresql) eventDB , 
pull out all of the new data and use RT (ticket system) to send stuff out. 
Added flags --quiet --batch. Now intelmqcli sends via cronjob.
These flags now allow CERT.at to run intelmq in full auto-mode! intelmqcli is 
started via cron and sends out all events to all ISPs.


=== Requests ===

  * Intevation searches for testers for the packages.
  * We'd like to have some nice graphs in the intelmq-manager: events/sec , parse-failures/sec, etc.
  * implementation of whitelisting of events (filter out events based on whitelists). See
    https://github.com/certtools/intelmq/issues/426
  * A good CSS design for the web page

=== Community ===

  * RIPE abuse-c contacts can be done locally. RIPE might be able to export 
abuse-c infos publicly (fingers crossed).

  * more command line options for intelmqcli (see the 
https://github.com/certat/intelmq repo)

  * Aaron gave a presentation at the ENISA workshop "CSIRTs in Europe", 11th of May.
    Slides will be shared on the ENISA page.


==== intelmq.org ==== 
The website intelmq.org is now online, but we would like to have more content and a proper 
design.
Do you want to contribute to intelmq, but you are not a programmer? This is 
your chance!


Current ToDos:
 * Create Website Content: How-Tos / Installation Instructions,  Success 
Stories
 ** How-Tos / Instructions: If you are using a special feature of IntelMQ, for 
instance an expert bot, try to find some time to write down a short article 
how you managed to get it to work and why you are using it.
 * Website Design


== Wishlist ==
  * **we need more test-cases!!!**
  * a specific config logic for ASNs: do this and that (for example sett ttl = 
1 month) if event is in ASN xyz. Or "ignore" if event is in ASN xyz. This 
should support some kind of more-specific-less-specific inheritance, 
similarly to Apache directory settings. The most specific setting wins. The 
order could be: country code -> ASN -> netblock -> ip (/32). Open questions: 
what's more relevant if both domains and numbers (ASN, IPs, net blocks) exist 
in an event?
  * block based processing: for example block based team cymru lookups
  * parallelisation: We need to revisit this topic

== Important Discussions ==
 In case you missed something, here are the headlines of some discussion we 
consider interesting / important.
 
=== Mailing Lists ===
 * [Intelmq-dev] Packaging Strategy for Bots with dependencies
 * [Intelmq-dev] Discussion on intelmq output / transformation architecture
 * [Intelmq-dev] Output format to syslog/splunk (PR#503)


== Communication ==
Chat: irc #intelmq on freenode or webchat: 
[[https://webchat.freenode.net/?channels=intelmq]]

Follow on twitter: @intelmqorg

Weekly Conference Call every Tuesday: Dial in via the known conference bridge number. It is 
[[https://en.wikipedia.org/wiki/Telephone_number_mapping|ENUM]] enabled. Ask 
Aaron or Dustin for the number if you want to participate.