Hello List,
as you all probably know, the code for the XMPP bot that was sent upstream was
broken, this shouldn't have happened. What we think happened was this: prior
to opening the pull request we pulled the commits to another repository and
rebased them. During this process we seem to have lost a couple of commits. I
will prepare a pull request that fixes these problems.
To avoid situations such as this arising in the future we will perform
additional tests before sending our code upstream.
Best regards,
Roland
--
Roland Geider | ++49-541-335 083 - 256 | http://www.intevation.de/
Intevation GmbH, 49074 Osnabrück, DE | AG Osnabrück, HR B 18998
Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner
= Intelmq-dev-news
Issue 3/2016
== Topics ==
# First Developer News
# IHAP meeting in April
# Status update Intevation
# Status update community
== March 2016 ==
Hi Folks,
this is the first issue of intelmq developer news.
[[https://github.com/aaronkaplan/|Aaron]] and [[https://github.com/dmth/|Dustin]] plan to send this newsletter on a monthly basis now.
We intend to achieve the following:
* Generate a "helicopter view" for all developers.
Everyone should be capable of knowing what is planned for the next iterations of intelmq and which challenges community and software are currently facing.
* Reduce the risk of parallel implementation of features
* Foster improvements of the software
* Create the possibility to discuss
=== How to contribute to this newsletter? ===
-> contact Aaron, Dustin for future input
=== Rules ===
In case you feel the urge to discuss a topic of this newsletter (which is what we intend!),
please create a new post on this mailing list, in which you refer to this newsletter.
For the sake of finding infos quickly: please start a separate thread in this case instead of answering to this mail. Thank you.
=== Frequency ===
The planned frequency is one newsletter / month.
=== Status report Intevation ===
* Started to created *.deb packages. We "successfully" created *.deb packages for the intelmq-manager. There are some warnings left, for example for the fonts. The core is still work in progress.
* We refactored the existing code of the XMPP bot. It still has to be tested with existing feeds.
* Currently we are spending most of our time on a local database to store abuse contacts and rules on how to notify them.
* We suggest to modify intelmq_psql_initdb.py in order to support PostgreSQLs native JSON data type.
* Currently working on a concept for integration tests.
* Started to map AbuseHelper and IntelMQ semantics.
=== Status report Community developments ===
* python3 only? Currently there is a discussion to drop the support for python2. In case there are no objections until end of March 2016, we will continue and move to python3 only. This simplifies a lot. Last chance to object
* [[https://github.com/certtools/intelmq/commit/04ccc93340158cc7f6aaf3900cde78b… csv parser]] by [[https://github.com/robcza|robcza]]
* [[https://github.com/certtools/intelmq/pull/455|ftp(s) bot]] submission by [[https://github.com/robcza|robcza]]
* thx to [[https://github.com/sebix|sebix]], abusix now seems to support [[https://github.com/certtools/intelmq/commit/e66effaa844977aabe38b1d54227c81…]]
* interesting discussion on running bots via cronjob: https://github.com/certtools/intelmq/issues/464
* We would like to change the syntax of intelmqcli to a new format: intelmqctl {start,stop} bot_id. Unless there are major objections we will continue. Note well: this will break compatibility with scripts.
* IHAP meeting in April: if you want to attend and have not registered yet, get in contact with Aaron pls.
* discussions on how to make intelmq-manager more useful: add events/sec (rate), failed events/sec (failure-rate), total failed events/sec as extra columns in the monitoring page.
* if you are using the n6stomp bot, there were some hick-ups upstream. You might have to restart your n6stomp bot.
== Planned for April 2016 ==
=== Meeting April ===
* If you have not registered for the IHAP meeting in April, please do so: [[http://doodle.com/poll/6hmhwahhp9sp2q5c#table|doodle]]
* On day two of the meeting we will have a hackathon: we can form small groups and work on specific, nice little tasks. Please think about topics you are interest in.
=== Intevation ===
* *.deb packages should exist now.
* A local contact database can be used to enrich events with contact information and instructions how they have to be informed.
* Input and Output formats:
** X-ARF
** IODEF
=== Community ===
* RIPE abuse-c contacts can be done locally. RIPE might be able to export abuse-c infos publicly (fingers crossed).
* more command line options for intelmqcli (see the https://github.com/certat/intelmq repo)
* activate intelmq.org homepage
== Wishlist ==
* **we need more test-cases!!!**
* a specific config logic for ASNs: do this and that (for example sett ttl = 1 month) if event is in ASN xyz. Or "ignore" if event is in ASN xyz. This should support some kind of more-specific-less-specific inheritance, similarly to Apache directory settings. The most specific setting wins. The order could be: country code -> ASN -> netblock -> ip (/32). Open questions: what's more relevant if both domains and numbers (ASN, IPs, net blocks) exist in an event?
* block based processing: for example block based team cymru lookups
* parallelisation: need to revisit this topic
== Communication ==
Chat: irc #intelmq on freenode or webchat: [[https://webchat.freenode.net/?channels=intelmq]]
Weekly Conference Call: Dial in via the known conference bridge number. It is [[https://en.wikipedia.org/wiki/Telephone_number_mapping|ENUM]] enabled. Ask Aaron or Dustin for the number if you want to participate.
The next weekly conf call is on March, 30th, 16:00 UTC+1
Hey Folks,
who would join a conf-call today?
Possible Topics:
* Quality of Data:
what is the most reliable source for Country information?
Maxmind or NCC?
* Developer Newsletter
We'd like to test palava.tv (Browser 2 Browser Video)
Feel free to join https://palava.tv/intelmq at 16:00 UTC+1
Best Regards
Dustin
--
dustin.demuth(a)intevation.de https://intevation.de/ OpenPGP key: B40D2EFF
Intevation GmbH, Neuer Graben 17, 49074 Osnabrück; AG Osnabrück, HR B 18998
Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner
Hi all,
do plans exist for a conference call today?
Here is a list of alternatives to Skype:
We did not test most of them
Based on WebRTC
- appear.in
- palava.tv (should work)
"Classical" Voice
- mumble
Unknown / Other
- zoom.us
Best Regards
Dustin
--
dustin.demuth(a)intevation.de https://intevation.de/ OpenPGP key: B40D2EFF
Intevation GmbH, Neuer Graben 17, 49074 Osnabrück; AG Osnabrück, HR B 18998
Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner
Dear community,
At our meeting in Osnabrueck, the point about legacy python
compatibility came up again. From a developer perspective, dropping
support for Legacy Python would be great. As Python now has native
unicode support, handling of strings became much easier and unproblematic.
Does anyone have concerns or blocking issues about dropping support for
the 2.7 series? Current Python 3.x is available on all plattforms.
Also, what versions of 3.x are available on your plattform? Currently we
run tests for 3.4 and 3.5.
Sebastian
--
// Sebastian Wagner <wagner(a)cert.at> - T: +43 1 50564167201
// CERT Austria - http://www.cert.at/
// Eine Initiative der nic.at GmbH - http://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg
Hi Folks,
today we created a pull request for an XMPP Collector and an XMPP Output bot.
The bots are based on the previous PRs 158 and 197.
This can be a first step to create a bridge to abusehelper.
BR
Dustin
--
dustin.demuth(a)intevation.de https://intevation.de/ OpenPGP key: B40D2EFF
Intevation GmbH, Neuer Graben 17, 49074 Osnabrück; AG Osnabrück, HR B 18998
Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner