[Intelmq-users] Max retries issue

[Sebastian Wagner]

HI Vaclav,

What's your default configuration and the runtime configuration for this
bot?

Sebastian

On 05.03.19 12:31, Vaclav Bruzek wrote:
> Hi mailinglist,
> I've went over intelmq logs today and that some bots keep retrying
> indefinitely when they encounter a failure. However each of those bots
> have *error_max_retries* set to 1 (and *error_retry_delay *to 60).
> Which in my mind should limit the bot to just try one more time and
> than give up and sleep (if sleep is set up) however this is not
> happening. Am I doing something wrong or does this functionality work
> differently? My version of intelmq is 1.1.1 in Docker on Ubuntu 18.04
> as base.
>
> Following is an example from logs os collector collecting from phishtank:
> 2019-03-01 20:07:51,747 - phishtank-collector - INFO - Downloading
> report from 'https://data.phishtank.com/data/online-valid.csv'.
> 2019-03-01 20:07:52,139 - phishtank-collector - ERROR - Bot has found
> a problem.
> Traceback (most recent call last):
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/lib/bot.py",
> line 167, in start
>     self.process()
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/bots/collectors/http/collector_http.py",
> line 76, in process
>     raise ValueError('HTTP response status code was %i.' %
> resp.status_code)
> ValueError: HTTP response status code was 509.
> 2019-03-01 20:07:52,140 - phishtank-collector - INFO - Current
> Message(event): None.
> 2019-03-01 20:07:52,140 - phishtank-collector - INFO - Downloading
> report from 'https://data.phishtank.com/data/online-valid.csv'.
> 2019-03-01 20:07:52,298 - phishtank-collector - ERROR - Bot has found
> a problem.
> Traceback (most recent call last):
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/lib/bot.py",
> line 167, in start
>     self.process()
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/bots/collectors/http/collector_http.py",
> line 76, in process
>     raise ValueError('HTTP response status code was %i.' %
> resp.status_code)
> ValueError: HTTP response status code was 509.
> 2019-03-01 20:07:52,299 - phishtank-collector - INFO - Current
> Message(event): None.
> 2019-03-01 20:07:52,299 - phishtank-collector - INFO - Bot will
> continue in 60 seconds.
> 2019-03-01 20:08:52,359 - phishtank-collector - INFO - Downloading
> report from 'https://data.phishtank.com/data/online-valid.csv'.
> 2019-03-01 20:08:52,654 - phishtank-collector - ERROR - Bot has found
> a problem.
> Traceback (most recent call last):
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/lib/bot.py",
> line 167, in start
>     self.process()
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/bots/collectors/http/collector_http.py",
> line 76, in process
>     raise ValueError('HTTP response status code was %i.' %
> resp.status_code)
> ValueError: HTTP response status code was 509.
> 2019-03-01 20:08:52,656 - phishtank-collector - INFO - Current
> Message(event): None.
> 2019-03-01 20:08:52,656 - phishtank-collector - INFO - Downloading
> report from 'https://data.phishtank.com/data/online-valid.csv'.
> 2019-03-01 20:08:52,831 - phishtank-collector - ERROR - Bot has found
> a problem.
> Traceback (most recent call last):
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/lib/bot.py",
> line 167, in start
>     self.process()
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/bots/collectors/http/collector_http.py",
> line 76, in process
>     raise ValueError('HTTP response status code was %i.' %
> resp.status_code)
> ValueError: HTTP response status code was 509.
> 2019-03-01 20:08:52,832 - phishtank-collector - INFO - Current
> Message(event): None.
> 2019-03-01 20:08:52,832 - phishtank-collector - INFO - Bot will
> continue in 60 seconds.
> 2019-03-01 20:09:52,891 - phishtank-collector - INFO - Downloading
> report from 'https://data.phishtank.com/data/online-valid.csv'.
> 2019-03-01 20:09:53,138 - phishtank-collector - ERROR - Bot has found
> a problem.
> Traceback (most recent call last):
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/lib/bot.py",
> line 167, in start
>     self.process()
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/bots/collectors/http/collector_http.py",
> line 76, in process
>     raise ValueError('HTTP response status code was %i.' %
> resp.status_code)
> ValueError: HTTP response status code was 509.
> 2019-03-01 20:09:53,139 - phishtank-collector - INFO - Current
> Message(event): None.
> 2019-03-01 20:09:53,139 - phishtank-collector - INFO - Downloading
> report from 'https://data.phishtank.com/data/online-valid.csv'.
> 2019-03-01 20:09:53,315 - phishtank-collector - ERROR - Bot has found
> a problem.
> Traceback (most recent call last):
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/lib/bot.py",
> line 167, in start
>     self.process()
>   File
> "/usr/local/lib/python3.6/dist-packages/intelmq-1.1.1-py3.6.egg/intelmq/bots/collectors/http/collector_http.py",
> line 76, in process
>     raise ValueError('HTTP response status code was %i.' %
> resp.status_code)
> ValueError: HTTP response status code was 509.
> 2019-03-01 20:09:53,317 - phishtank-collector - INFO - Current
> Message(event): None.
> 2019-03-01 20:09:53,318 - phishtank-collector - INFO - Bot will
> continue in 60 seconds.
>
>
> Sincerely,
> Václav Brůžek
>
-- 
// Sebastian Wagner <wagner at cert.at> - T: +43 1 5056416 7201
// CERT Austria - https://www.cert.at/
// Eine Initiative der nic.at GmbH - https://www.nic.at/
// Firmenbuchnummer 172568b, LG Salzburg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20190311/ecae84aa/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cert.at/pipermail/intelmq-users/attachments/20190311/ecae84aa/attachment.sig>