[Intelmq-dev] Monthly newsletter 3/2016

[L. Aaron Kaplan]

= Intelmq-dev-news 

Issue 3/2016

== Topics == 

 # First Developer News
 # IHAP meeting in April
 # Status update Intevation
 # Status update community


== March 2016 ==

Hi Folks,

this is the first issue of intelmq developer news.
[[https://github.com/aaronkaplan/|Aaron]] and [[https://github.com/dmth/|Dustin]] plan to send this newsletter on a monthly basis now.

We intend to achieve the following:
 * Generate a "helicopter view" for all developers.
   Everyone should be capable of knowing what is planned for the next iterations of intelmq and which challenges community and software are currently facing.
 * Reduce the risk of parallel implementation of features
 * Foster improvements of the software
 * Create the possibility to discuss 


=== How to contribute to this newsletter? ===

 -> contact Aaron, Dustin for future input
 
=== Rules ===
 In case you feel the urge to discuss a topic of this newsletter (which is what we intend!),
 please create a new post on this mailing list, in which you refer to this newsletter.
 For the sake of finding infos quickly: please start a separate thread in this case instead of answering to this mail. Thank you.
 

=== Frequency ===

The planned frequency is one newsletter / month.


=== Status report Intevation === 

* Started to created *.deb packages. We "successfully" created *.deb packages for the intelmq-manager. There are some warnings left, for example for the fonts. The core is still work in progress. 
* We refactored the existing code of the XMPP bot. It still has to be tested with existing feeds.
* Currently we are spending most of our time on a local database to store abuse contacts and rules on how to notify them.
* We suggest to modify intelmq_psql_initdb.py in order to support PostgreSQLs native JSON data type.
* Currently working on a concept for integration tests.
* Started to map AbuseHelper and IntelMQ semantics.

=== Status report Community developments ===

* python3 only? Currently there is a discussion to drop the support for python2. In case there are no objections until end of March 2016, we will continue and move to python3 only. This simplifies a lot. Last chance to object
* [[https://github.com/certtools/intelmq/commit/04ccc93340158cc7f6aaf3900cde78bcb3eb9b39|generic csv parser]] by [[https://github.com/robcza|robcza]]
* [[https://github.com/certtools/intelmq/pull/455|ftp(s) bot]] submission by [[https://github.com/robcza|robcza]]
* thx to [[https://github.com/sebix|sebix]], abusix now seems to support [[https://github.com/certtools/intelmq/commit/e66effaa844977aabe38b1d54227c81926cea1fc|ipv6]]
* interesting discussion on running bots via cronjob: https://github.com/certtools/intelmq/issues/464 
* We would like to change the syntax of intelmqcli to a new format: intelmqctl {start,stop} bot_id. Unless there are major objections we will continue. Note well: this will break compatibility with scripts. 
* IHAP meeting in April: if you want to attend and have not registered yet, get in contact with Aaron pls.
* discussions on how to make intelmq-manager more useful: add events/sec (rate), failed events/sec (failure-rate), total failed events/sec as extra columns in the monitoring page.
* if you are using the n6stomp bot, there were some hick-ups upstream. You might have to restart your n6stomp bot.

== Planned for April 2016 ==

=== Meeting April ===

  * If you have not registered for the IHAP meeting in April, please do so: [[http://doodle.com/poll/6hmhwahhp9sp2q5c#table|doodle]]
  * On day two of the meeting we will have a hackathon: we can form small groups and work on specific, nice little tasks. Please think about topics you are interest in.

=== Intevation === 

  * *.deb packages should exist now.
  * A local contact database can be used to enrich events with contact information and instructions how they have to be informed.
  * Input and Output formats:
  ** X-ARF
  ** IODEF

=== Community ===

  * RIPE abuse-c contacts can be done locally. RIPE might be able to export abuse-c infos publicly (fingers crossed).
  * more command line options for intelmqcli (see the https://github.com/certat/intelmq repo)
  * activate intelmq.org homepage


== Wishlist ==

  * **we need more test-cases!!!**
  * a specific config logic for ASNs: do this and that (for example sett ttl = 1 month) if event is in ASN xyz. Or "ignore" if event is in ASN xyz. This should support some kind of more-specific-less-specific inheritance, similarly to Apache directory settings. The most specific setting wins. The order could be: country code -> ASN -> netblock -> ip (/32). Open questions: what's more relevant if both domains and numbers (ASN, IPs, net blocks) exist in an event?
  * block based processing: for example block based team cymru lookups
  * parallelisation: need to revisit this topic

== Communication ==

Chat: irc #intelmq on freenode or webchat: [[https://webchat.freenode.net/?channels=intelmq]]

Weekly Conference Call: Dial in via the known conference bridge number. It is [[https://en.wikipedia.org/wiki/Telephone_number_mapping|ENUM]] enabled. Ask Aaron or Dustin for the number if you want to participate.
The next weekly conf call is on March, 30th, 16:00 UTC+1