<div dir="ltr"><div>Elastic, like many other datastores, is not very frequently updated. Organizations stay on older version for longer duration. Can we keep both versions? We can focus maintenance efforts only on the latest version.</div><div>As for shipping to ES via LogStash, an alternative could be to use filebeat with file-output bot. I have seen setups where out of ELK, L(ogstatsh) is not used.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 30, 2020 at 9:44 AM Sebastian Wagner <<a href="mailto:wagner@cert.at">wagner@cert.at</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi all,</p>
<p>Good news, Stone Zach is able to make required changes for
ElasticSearch 7, see
<a href="https://github.com/certtools/intelmq/issues/1479#issuecomment-605454294" target="_blank">https://github.com/certtools/intelmq/issues/1479#issuecomment-605454294</a>
and <a href="https://github.com/certtools/intelmq/pull/1513" target="_blank">https://github.com/certtools/intelmq/pull/1513</a> - Thanks stone!<br>
</p>
<p>However the code for ES 7 is incompatible to the code for ES 5
and Stone can only support the ES 7 code. That would mean that -
if nothing changes - the support for ES 5 would be probably
dropped in the next feature release (2.2.0).</p>
<p>In parallel to these efforts, I welcome the contribution by Filip
to document how IntelMQ can be used to feed ES via Redis +
Logstash. Thanks!<br>
</p>
<p>Sebastian<br>
</p>
<div>On 3/20/20 4:20 PM, Sebastian Wagner
wrote:<br>
</div>
<blockquote type="cite">
<pre>Dear community,
The ElasticSearch bots, tests and tools in IntelMQ need some maintenance
which I am unable to provide. As ES is a very common tool I am sure that
there is know-how available in the community and we are able to continue
the support for it.
The oldest know issue is a broken unittest:
<a href="https://github.com/certtools/intelmq/issues/1480" target="_blank">https://github.com/certtools/intelmq/issues/1480</a>
But there are also incompatibilties with current ElasticSearch version,
e.g. I had problems with the elasticmapper tool using ES 7.6.1 (maybe
easy to fix).
Using 7.5.0 failed on the indices tests
<a href="https://github.com/certtools/intelmq/issues/1479" target="_blank">https://github.com/certtools/intelmq/issues/1479</a>
Further, the only supported elasticsearch python library version is
currently 'elasticsearch>=5.0.0,<6.0.0' while the latest release is 7.6.0.
Please consider contributing
best regards
Sebastian
</pre>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Intelmq-dev mailing list
<a href="mailto:Intelmq-dev@lists.cert.at" target="_blank">Intelmq-dev@lists.cert.at</a>
<a href="https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev" target="_blank">https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev</a>
</pre>
</blockquote>
<pre cols="72">--
// Sebastian Wagner <a href="mailto:wagner@cert.at" target="_blank"><wagner@cert.at></a> - T: +43 1 5056416 7201
// CERT Austria - <a href="https://www.cert.at/" target="_blank">https://www.cert.at/</a>
// Eine Initiative der <a href="http://nic.at" target="_blank">nic.at</a> GmbH - <a href="https://www.nic.at/" target="_blank">https://www.nic.at/</a>
// Firmenbuchnummer 172568b, LG Salzburg</pre>
</div>
_______________________________________________<br>
Intelmq-dev mailing list<br>
<a href="mailto:Intelmq-dev@lists.cert.at" target="_blank">Intelmq-dev@lists.cert.at</a><br>
<a href="https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev" rel="noreferrer" target="_blank">https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev</a><br>
</blockquote></div>