<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi Mika,<br>
</p>
<div class="moz-cite-prefix">On 6/22/22 1:58 PM, Mika Silander
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:772486281.3259230.1655899096165.JavaMail.zimbra@csc.fi">
<pre class="moz-quote-pre" wrap=""> Tracking down a rather elusive bug led me to suspect that the command
intelmqctl reload your_bot_name_here
does not necessarily rerun the bot's init method.</pre>
</blockquote>
<p>It does, but not directly.</p>
<p>reloading relies on sending a signal to the process. But signals
interrupt system calls, which is normally not an issue, except the
process is currently doing something on the network, more
specifically with the pipeline (sending or receiving data). Let's
assume that the bot sends an event to redis, get's interrupted.
What to do now? Run init, and then? we can't just resume the
previously running process() call. Therefore, the behaviour is not
completely straighforward, but not magic.<br>
The signal is caught (<span class="pl-en"><span class="pl-token"
data-hydro-click="{"event_type":"code_navigation.click_on_symbol","payload":{"action":"click_on_symbol","repository_id":21160389,"ref":"develop","language":"Python","backend":"ALEPH_PRECISE","code_nav_context":"BLOB_VIEW","retry_backend":"","originating_url":"https://github.com/certtools/intelmq/find-definition?q=__handle_sighup_signal&blob_path=intelmq%2Flib%2Fbot.py&ref=develop&language=Python&row=261&col=8&code_nav_context=BLOB_VIEW","user_id":199050}}"
data-hydro-click-hmac="c6bc6f638306d84793d4657e8b83b1670a38c63185fe79368283f1f41424dadd">__handle_sighup_signal)</span></span>
and python is commanded to resume system calls instead of
interrupted (<span class="pl-s1"><span class="pl-token"
data-hydro-click="{"event_type":"code_navigation.click_on_symbol","payload":{"action":"click_on_symbol","repository_id":21160389,"ref":"develop","language":"Python","backend":"ALEPH_PRECISE","code_nav_context":"BLOB_VIEW","retry_backend":"","originating_url":"https://github.com/certtools/intelmq/find-definition?q=signal&blob_path=intelmq%2Flib%2Fbot.py&ref=develop&language=Python&row=221&col=16&code_nav_context=BLOB_VIEW","user_id":199050}}"
data-hydro-click-hmac="32c0854bd7114a27c9dd223c8f7fbb2f5c2e49be482016a3116805defc0c12a1">signal</span></span>.<span
class="pl-en"><span class="pl-token"
data-hydro-click="{"event_type":"code_navigation.click_on_symbol","payload":{"action":"click_on_symbol","repository_id":21160389,"ref":"develop","language":"Python","backend":"ALEPH_PRECISE","code_nav_context":"BLOB_VIEW","retry_backend":"","originating_url":"https://github.com/certtools/intelmq/find-definition?q=siginterrupt&blob_path=intelmq%2Flib%2Fbot.py&ref=develop&language=Python&row=221&col=23&code_nav_context=BLOB_VIEW","user_id":199050}}"
data-hydro-click-hmac="728af2055d6c4ce7f124a3f8fc93ce5778bc3ff74386b265211c01f76c5f41d3">siginterrupt</span></span>(<span
class="pl-s1"><span class="pl-token"
data-hydro-click="{"event_type":"code_navigation.click_on_symbol","payload":{"action":"click_on_symbol","repository_id":21160389,"ref":"develop","language":"Python","backend":"ALEPH_PRECISE","code_nav_context":"BLOB_VIEW","retry_backend":"","originating_url":"https://github.com/certtools/intelmq/find-definition?q=signal&blob_path=intelmq%2Flib%2Fbot.py&ref=develop&language=Python&row=221&col=36&code_nav_context=BLOB_VIEW","user_id":199050}}"
data-hydro-click-hmac="b51a305cc1992ab42677894044db09c2653259b36e0211bb02d7ca9c5c9fab39">signal</span></span>.<span
class="pl-v"><span class="pl-token"
data-hydro-click="{"event_type":"code_navigation.click_on_symbol","payload":{"action":"click_on_symbol","repository_id":21160389,"ref":"develop","language":"Python","backend":"ALEPH_PRECISE","code_nav_context":"BLOB_VIEW","retry_backend":"","originating_url":"https://github.com/certtools/intelmq/find-definition?q=SIGHUP&blob_path=intelmq%2Flib%2Fbot.py&ref=develop&language=Python&row=221&col=43&code_nav_context=BLOB_VIEW","user_id":199050}}"
data-hydro-click-hmac="c74f531f51bdc3d37a8b1247275cc4dcb132d2011f73ac38b2e35c5cda28c827">SIGHUP</span></span>,
<span class="pl-c1">False</span>) in __init__)<br>
Once there is time to handle the reload safely (before/after
process, during sleeps, when waiting for messages), the bot is
actually reloaded and __init__ called (<span class="pl-en"><span
class="pl-token active"
data-hydro-click="{"event_type":"code_navigation.click_on_symbol","payload":{"action":"click_on_symbol","repository_id":21160389,"ref":"develop","language":"Python","backend":"ALEPH_PRECISE","code_nav_context":"BLOB_VIEW","retry_backend":"","originating_url":"https://github.com/certtools/intelmq/find-definition?q=__handle_sighup&blob_path=intelmq%2Flib%2Fbot.py&ref=develop&language=Python&row=271&col=8&code_nav_context=BLOB_VIEW","user_id":199050}}"
data-hydro-click-hmac="3c49d7f5b13ed14bb8280c503b8c5e48411aae0d91032e1cab548e34c8152ec1">__handle_sighup).</span></span></p>
<blockquote type="cite"
cite="mid:772486281.3259230.1655899096165.JavaMail.zimbra@csc.fi">
<pre class="moz-quote-pre" wrap="">At least I see an exception related to undefined variables (NoneType)</pre>
</blockquote>
In the code of the core or in the bot's code? If it's the bot's
code: What bot is it? In any case, the stacktrace helps in
explaining the issue, because unfortunately crystal ball broke last
night when my neighbors' dragon chased my little dinosaur....<br>
<blockquote type="cite"
cite="mid:772486281.3259230.1655899096165.JavaMail.zimbra@csc.fi">
<pre class="moz-quote-pre" wrap="">when I reload the bot in question and let it chew a new event. Is this expected behaviour? Still, skimming through the code, it seems to me that the __init__ method in lib/bot.py gets invoked and should invoke the bot's init method as well.</pre>
</blockquote>
<p>Yes, it shoud/does:</p>
<p><a class="moz-txt-link-freetext" href="https://github.com/certtools/intelmq/blob/79cae294a74ab77f09b1f371a19f04ad65abab12/intelmq/lib/bot.py#L216">https://github.com/certtools/intelmq/blob/79cae294a74ab77f09b1f371a19f04ad65abab12/intelmq/lib/bot.py#L216</a>
</p>
<blockquote type="cite"
cite="mid:772486281.3259230.1655899096165.JavaMail.zimbra@csc.fi">
</blockquote>
<pre class="moz-signature" cols="72">--
Institute for Common Good Technology
<a class="moz-txt-link-freetext" href="http://commongoodtechnology.org/">http://commongoodtechnology.org/</a>
ZVR 1510673578</pre>
</body>
</html>