IntelMQ-dev April 2016

intelmq-dev@lists.cert.at

7 participants
8 discussions

Packaging Strategy for Bots with dependencies
by Dustin Demuth 07 Jul '16

07 Jul '16

Hey Folks, Whilst checking the dependencies, I've found out that a lot of Bots have their own requirements-file. From a packagers point of view, I think this is hard to maintain. I think, we should discuss if it is reasonable to split the software into a CORE package, were all packages have the same dependencies, and several BOTS packages with their own dependencies. This would create the additional need to rethink the BOTS json-file. It would be possible to have all JSON configs for the Bots in a bots.d directory and search for them in this directory. I know, that this might create additional complexity in some points. BR Dustin -- dustin.demuth(a)intevation.de https://intevation.de/ OpenPGP key: B40D2EFF Intevation GmbH, Neuer Graben 17, 49074 Osnabrück; AG Osnabrück, HR B 18998 Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

8 15

Discussion on intelmq output / transformation architecture
by L. Aaron Kaplan 24 May '16

24 May '16

Hi folks, an interesting discussion started on https://github.com/certtools/intelmq/issues/487 Basically, pedro would like to send data to ArcSite in CEF format. I would like to propose that we enhance our architecture to include "transformer bots". Apart from the funny name which reminds me of http://transformers.hasbro.com/en-us/bots, this has a nice symmetry to the collector-> parser chain we have on the input side. So what are we talking about? Input side: ``` Collector ---> Parser ---> .... pipeline ``` Currently on the output side it's like this: ``` ... pipeline -> expert1 -> expert2 -> ... -> output bot 1 ``` But this basically leaves the work of transcoding/transforming the DHO format to the output bots. So far, this was enough. In the general case, it would be great to transform the DHO to some format. Let's assume CEF or IODEF. A generic output bot could then take that data and send it over whatever mechanism it knows to a destination. Example: ``` ... pipeline -> expert1 -> transform 2 IODEF -> mail bot or: ... pipeline -> expert1 -> transform 2 IODEF -> mail bot \ \-> sftp bot ``` The question in https://github.com/certtools/intelmq/issues/487 arose on where to store the output data. I'd suggest to create a new "raw_output" field in the DHO for this. A transformer bot shall write its format to the raw_output field (in base64) and an output bot shall take that data, decode the base64 andd send it via its own mechanism. What do you think? Would this work? Do you see any serious problem with this approach? Best, a.

3 2

Possible License Conflict for ASN-Lookup Bot
by Dustin Demuth 02 May '16

02 May '16

Dear all, whilst packaging we've had a look at the licenses. Most things are ok, but there might be a problem with the asn_lookup expert. It makes use of pyasn (MIT-License) which builds on other products with different lisenses. One of them (mrtd) uses a 4-Clause BSD license which might be incompatible to AGPL. Do you know of alternatives to pyasn? Best Regards Dustin -- dustin.demuth(a)intevation.de https://intevation.de/ OpenPGP key: B40D2EFF Intevation GmbH, Neuer Graben 17, 49074 Osnabrück; AG Osnabrück, HR B 18998 Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

4 5

Conf-Call 26.04.16
by Dustin Demuth 26 Apr '16

26 Apr '16

Hey Folks, This are the minutes of today's call. Participants: Sebastian, Aaron, Dustin Topics: _ Pull-Request ContactDB https://github.com/Intevation/intelmq/pull/3/commits/902896a96725ff288419a9… Aaron sent a PR for the Contact DB The Handles are URIs to the API of first.org or TI. * For TI the public service should be used as an example. * In Ripe an ORG-Handle exists Roles: New Table is supposed to model tech-c, abuse-c, etc. AS: ripe_aut_num refers to the the number in RIPE Map Organization to ASN instead of Contacts, bc. this mirrors the RIPE-Data more closely -> Problem: notification intervall per contact is not possible anymore. CONCEPT: Fill a Shadow-Database from ftp://ftp.ripe.net/ripe/dbase/ See tables in split directory _ Contact-DB Classification - This is similar to the incidents' classification in eventDB - Classification will need a little overhaul to reflect the taxonomy _ Newsletter * Sent Newsletter on Friday * Review on Thursday _ Merges from Sebastian * Redis Output Bot was successfully Merged * Several Bugfixes * New Bitsight collector _ Debian-Packages * Use HTTP Apt-Repository from intevation to provide the packages. _ XMPP - Bot * Intevation needs to do some more testing and a quick internal review. But the bot should already work. _ PyASN vs. Other Technologies * See upcoming Blog post _ Linux Standards * Logging: Syslog is currenntly in test * Configuration: runtime.d, bots.d * To think about: start intelmq with systemd in intelmq2 _ "Coding Desaster Contingency Plan" * Intevation and Cert.at will create backups of github repositories on a daily basis. BR Dustin -- dustin.demuth(a)intevation.de https://intevation.de/ OpenPGP key: B40D2EFF Intevation GmbH, Neuer Graben 17, 49074 Osnabrück; AG Osnabrück, HR B 18998 Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

1 0

reminder: intelmq-dev conf call today
by L. Aaron Kaplan 26 Apr '16

26 Apr '16

Dear list members, this is a short reminder: last time we skipped the intelmq-dev conf call since we had the physical meeting in Vienna anyway. This week / this Tuesday we'd like to resume the regular calls. When? Today, 26th of April 2016, 16:00 UTC+2 Dial in-number: please ask me if you plan to attend. (not posting it since this is a public list with a public archive). Best, a.

1 0

Deb Packages for Ubuntu
by Dustin Demuth 25 Apr '16

25 Apr '16

Hey Folks, Sascha has been sucessful in creating *.deb Packages for intelmq and intelmq-manager. They work on Ubuntu 14.04. (with a little bit of tweaking) If you are interested in packages, have a look at: https://github.com/Intevation/intelmq/tree/deb-packaging and https://github.com/Intevation/intelmq-manager We needed to downgrade[1] a bunch of packages in intelmqs setup.py. ** Some testing will be required. ** Some packages are _not_ available in Ubuntu 14.04 but they can be installed from 15.10: - python3-redis_2.10.3-3ubuntu1 - python3-tz_2014.10dfsg1-0ubuntu2 - python3-termstyle_0.1.10-1 - python3-unicodecsv_0.13.0-2 The package intelmq-manager depends on intelmq. The dependency on sudo still exists. Packages do not follow the Debian packaging guidelines. Which must be improved. [1] https://github.com/Intevation/intelmq/commit/81b6af9ef0d08c4c6726e4c4feeaa8… BR Dustin -- dustin.demuth(a)intevation.de https://intevation.de/ OpenPGP key: B40D2EFF Intevation GmbH, Neuer Graben 17, 49074 Osnabrück; AG Osnabrück, HR B 18998 Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

3 5

input for our intelmq newsletter April 2016
by L. Aaron Kaplan 20 Apr '16

20 Apr '16

Hi folks, Dustin and me are starting to collect input for our monthly intelmq newsletter. Please be so kind and send us your input. We write down as much as we know about of course, but I'd be helpful to still get your input upfront. Otherwise it might not be in the newletter. Best, a.

1 0

Current status of XMPP bot
by Roland Geider 16 Apr '16

16 Apr '16

Hello List, as you all probably know, the code for the XMPP bot that was sent upstream was broken, this shouldn't have happened. What we think happened was this: prior to opening the pull request we pulled the commits to another repository and rebased them. During this process we seem to have lost a couple of commits. I will prepare a pull request that fixes these problems. To avoid situations such as this arising in the future we will perform additional tests before sending our code upstream. Best regards, Roland -- Roland Geider | ++49-541-335 083 - 256 | http://www.intevation.de/ Intevation GmbH, 49074 Osnabrück, DE | AG Osnabrück, HR B 18998 Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

IntelMQ-dev April 2016