Dear list,
in pull request #944 (netlab 360 enh [0]) by navtej an issue came up
which can't be solved trivially:
The feed Netlab 360 DGA[1] - which is already included in intelmq -
provides a validity time frame for each domain. Most of those (~90%) end
in 2030 while the start date is the current day at 00:00.
So both start and end time are artificial. And the source claims the
event is valid in the future, which is a very odd. And does it actually
make sense to forward this kind of information?
Also, we can't really handle this time information using the current
harmonization.
One idea would be to set time.source to time.observation if the
time.source is in the future. So time.source <= time.observation does
always apply.
What do you think?
Sebastian
[0]: https://github.com/certtools/intelmq/pull/944
[1]: http://data.netlab.360.com/feeds/dga/dga.txt - attention, quite
big! The domains at the beginning have a very near end date.
--
// Sebastian Wagner <wagner(a)cert.at> - T: +43 1 5056416 7201
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - https://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg
Hi all,
Recently, I have installed IntelMQ in a CentOS 7.4 host (fully
patched) and I see some "errors" in official IntelMQ's rpm packages
installed from http://download.opensuse.org/repositories/home:/sebix:/intelmq/CentOS_7/.
a/ /etc/cron.d/intelmq-update-data. Content is:
# /etc/cron.d/intelmq-update-data: crontab fragment for intelmq
# This updates the data files used by some expert bots.
#
# m h dom mon dow command
# Update data for tor_nodes bot:
11 0 * * * intelmq /usr/bin/update-tor-nodes
/var/lib/intelmq/bots/tor_nodes/tor_nodes.dat
# Update data for maxmind_geoip bot:
17 0 * * * intelmq /usr/bin/update-geoip-data
/var/lib/intelmq/bots/maxmind_geoip/GeoLite2-City.mmdb
# Update data for asn_lookup bot:
23 0 * * * intelmq /usr/bin/update-asn-data
/var/lib/intelmq/bots/asn_lookup/ipasn.dat
# Update data for the RIPE DB abuse_c offline contact lookup
25 6 * * * intelmq /usr/bin/update-ripencc_abuse_contact_offline
/var/lib/intelmq/bots/ripencc_abuse_contact_offline/
Where are these scripts: update-tor-nodes, update-geoip-data,
update-asn-data and update-ripencc_abuse_contact_offline? They don't
exist in my system. But exists intelmq-update-asn-data,
intelmq-update-geoip-data and intelmq-update-tor-nodes (not ripe).
b/ /etc/logrotate.d/intelmq. Content is:
compress
delaycompress
copytruncate
create 640 intelmq intelmq
/var/log/intelmq/*.log {
su intelmq intelmq
daily
maxsize 10M
rotate 60
notifempty
sharedscripts
postrotate
/usr/bin/intelmqctl reload --quiet
endscript
}
/var/lib/intelmq/bots/file-output/*.txt {
su intelmq intelmq
daily
maxsize 10M
rotate 60
notifempty
sharedscripts
postrotate
/usr/bin/intelmqctl reload file-output --quiet
endscript
}
... but returns the following email error:
From root(a)cosintelmq.mydomain.com Mon Nov 13 08:29:04 2017
Return-Path: <root(a)cosintelmq.mydomain.com>
X-Original-To: root
Delivered-To: root(a)cosintelmq.mydomain.com
From: Anacron <root(a)cosintelmq.mydomain.com>
To: root(a)cosintelmq.mydomain.com
Content-Type: text/plain; charset="UTF-8"
Subject: Anacron job 'cron.daily' on cosintelmq.mydomain.com
Date: Mon, 13 Nov 2017 08:29:04 +0000 (UTC)
Status: R
/etc/cron.daily/logrotate:
intelmqctl: Running intelmqctl as root is highly discouraged!
usage: intelmqctl [-h] [-v] [--type {text,json}] [--quiet]
{list,check,clear,log,run,help,start,stop,restart,reload,status,enable,disable}
...
intelmqctl: error: unrecognized arguments: --quiet
error: error running shared postrotate script for '/var/log/intelmq/*.log '
Maybe is it more correct to do this:
- /usr/bin/intelmqctl reload --quiet
+ su -m intelmq -c ' /usr/bin/intelmqctl reload --quiet'
- /usr/bin/intelmqctl reload file-output --quiet
+ su -m intelmq -c '/usr/bin/intelmqctl reload file-output --quiet' ??
Thanks.
Hi All,
I am currently in the process of deciding whether ANZ should incorporate IntelMQ into its Threat Intelligence ingestion and sharing platform.
At the Deepsec conference Sebastian mentioned updating the harmonization to allow for fields with multiple values. Has this issue been progressed at all? We will require multiple values for some fields in our events, and I was considering adding this functionality (perhaps in a hacky way) to my own fork, but I would like an update on the progress on the work on the master before doing so.
Regards,
Alex Knight | ANZ | ISO | Cyber Security Engineering
Level 8, 55 Collins Street, Melbourne 3000
Phone: +61 386 545 888 | www.anz.com<http://www.anz.com/>
"This e-mail and any attachments to it (the "Communication") is, unless otherwise stated, confidential, may contain copyright material and is for the use only of the intended recipient. If you receive the Communication in error, please notify the sender immediately by return e-mail, delete the Communication and the return e-mail, and do not read, copy, retransmit or otherwise deal with it. Any views expressed in the Communication are those of the individual sender only, unless expressly stated to be those of Australia and New Zealand Banking Group Limited ABN 11 005 357 522, or any of its related entities including ANZ Bank New Zealand Limited (together "ANZ"). ANZ does not accept liability in connection with the integrity of or errors in the Communication, computer virus, data corruption, interference or delay arising from or in respect of the Communication."
Dear community,
I just pushed the version 1.0.2 to pypi and the build servers.
Installation documentation:
https://github.com/certtools/intelmq/blob/1.0.0/docs/INSTALL.md
Upgrade documentation:
https://github.com/certtools/intelmq/blob/develop/docs/UPGRADING.md
### Core
- `lib.message.add`: parameter force has finally been removed, should
have been gone in 1.0.0.rc1 already
### Bots
- `collectors.mail.collector_mail_url`: Fix bug which prevented marking
emails seen due to disconnects from server (#852).
- `parsers.spamhaus.parser_cert`: Handle/ignore 'AS?' in feed (#1111)
### Packaging
- The following changes have been in effect for the built packages
already since version 1.0.0
- Support building for more distributions, now supported: CentOS 7,
Debian 8 and 9, Fedora 25 and 26, RHEL 7, openSUSE Leap 42.2 and 42.3
and Tumbleweed, Ubuntu 14.04 and 16.04
- Use LSB-paths for created packages (/etc/intelmq/, /var/lib/intelmq/,
/run/intelmq/) (#470). Does does not affect installations with
setuptools/pip.
- Change the debian package format from native to quilt
- Fix problems in postint and postrm scripts
- Use systemd-tmpfile for creation of /run/intelmq/
### Documentation
- Add disclaimer on maxmind database in bot documentation and code and
the cron-job (#1110)
Sebastian
--
// Sebastian Wagner <wagner(a)cert.at> - T: +43 1 5056416 7201
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - https://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg