Hi,
Bothering you again with a probably simple question. I've created half a dozen of bots on a development machine and now I'd like to create an installation system, Makefile or whatever to install those bots + configurations to the production host. It appears to me that the section "Update" (https://intelmq.readthedocs.io/en/latest/dev/guide.html#id12) contains the recommended steps to follow and assumes that one has the development environment (below /opt/dev_intelmq) available on the same host where one intends to run the production set up (under /opt/intelmq), correct?
So, what do you recommend? Should I copy over the entire /opt/dev_intelmq to the production host and then "Update" there according to the above instructions or are there other ways to just copy/install the relevant bots+configurations from below /opt/dev_intelmq to the production host?
Best regards and thanks again, Mika
Hmmm ... I should have been clearer on my aims and elaborated, so here's a retake:
I'd like to install from the package management the following to the production host:
intelmq intelmq-manager
From the development machine I'd like to copy+install the bots, runtime.conf, pipeline.conf and the BOTS conf file to the production machine so as to be able to manage them there using intelmq-manager. Intelmq-manager would be an easier tool for the other admins to adopt than intelmqctl.
Is this a reasonable and feasible aim? Or should I discard the usage of intelmq-manager altogether and copy over /opt/dev_intelmq to the production host and use it as the production setup without intelmq-manager? Earlier I tried to use intelmq-manager (installed from the package management) to manage the bots created under /opt/dev_intelmq but despite a bunch of hacks I failed to get it working. I admit that there is potentially a version incompatibility between the intelmq and intelmq-manager packages with respect to the bots and tools from /opt/dev_intelmq so I'm maybe pushing the limits too much here.
Cheers, Mika
----- Original Message ----- From: "Mika Silander" mika.silander@csc.fi To: "intelmq-dev" intelmq-dev@lists.cert.at Sent: Monday, 1 March, 2021 09:44:13 Subject: [IntelMQ-dev] Installing new bots from devel host to production host
Hi,
Bothering you again with a probably simple question. I've created half a dozen of bots on a development machine and now I'd like to create an installation system, Makefile or whatever to install those bots + configurations to the production host. It appears to me that the section "Update" (https://intelmq.readthedocs.io/en/latest/dev/guide.html#id12) contains the recommended steps to follow and assumes that one has the development environment (below /opt/dev_intelmq) available on the same host where one intends to run the production set up (under /opt/intelmq), correct?
So, what do you recommend? Should I copy over the entire /opt/dev_intelmq to the production host and then "Update" there according to the above instructions or are there other ways to just copy/install the relevant bots+configurations from below /opt/dev_intelmq to the production host?
Best regards and thanks again, Mika _______________________________________________ IntelMQ-dev mailing list IntelMQ-dev@lists.cert.at https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev
Am Montag, 1. März 2021, 10:43:05 CET schrieb Mika Silander:
From the development machine I'd like to copy+install the bots, runtime.conf, pipeline.conf and the BOTS conf file to the production machine so as to be able to manage them there using intelmq-manager. Intelmq-manager would be an easier tool for the other admins to adopt than intelmqctl.
Is this a reasonable and feasible aim?
Yes, using an initial setup and then IntelMQ-Manager is a supported scenario and makes sense if the other admins want to tweak things.
(As always it need to be set up and testet, note that between 2.2 and 2.3 there is a technology shift around IntelMQ Manager moving from php to python and changing how the backend works. So when setting something up now, use the upcoming versions.)
Regards, Bernhard
Ok, thanks for confirming the idea is feasible. Where can I find the newer intelmq-manager that is compatible with the development version (2.3.x)? I see no signs of intelmq-manager source code within /opt/dev_intelmq or have I missed something obvious?
Cheers, Mika
----- Original Message ----- From: "Bernhard Reiter" bernhard@intevation.de To: "intelmq-dev" intelmq-dev@lists.cert.at Sent: Tuesday, 2 March, 2021 10:48:48 Subject: Re: [IntelMQ-dev] Installing new bots from devel host to production host
Am Montag, 1. März 2021, 10:43:05 CET schrieb Mika Silander:
From the development machine I'd like to copy+install the bots, runtime.conf, pipeline.conf and the BOTS conf file to the production machine so as to be able to manage them there using intelmq-manager. Intelmq-manager would be an easier tool for the other admins to adopt than intelmqctl.
Is this a reasonable and feasible aim?
Yes, using an initial setup and then IntelMQ-Manager is a supported scenario and makes sense if the other admins want to tweak things.
(As always it need to be set up and testet, note that between 2.2 and 2.3 there is a technology shift around IntelMQ Manager moving from php to python and changing how the backend works. So when setting something up now, use the upcoming versions.)
Regards, Bernhard
Hi Mika,
On 3/2/21 10:57 AM, Mika Silander wrote:
Ok, thanks for confirming the idea is feasible. Where can I find the newer intelmq-manager that is compatible with the development version (2.3.x)?
In the unstable deb/rpm repositories[0] and in the git-repository of the intelmq-manager in the develop branch[1].
If you install intelmq-manager from the (unstable) repository - and therefore the required API as well - you probably need to adapt the API's configuration in regards to the intelmqctl command[2]
And be careful with develop - it may break.
Sebastian
[0]: https://intelmq.readthedocs.io/en/latest/dev/guide.html#id5 [1]: https://github.com/certtools/intelmq-manager/tree/develop/ [2]: https://intelmq.readthedocs.io/en/latest/user/intelmq-api.html#configuring-i...
Hi Sebastian,
Thanks for the pointers. Seems I have a little too many things to follow :-)
Cheers, Mika
----- Original Message ----- From: "Sebastian Wagner" wagner@cert.at To: "Mika Silander" mika.silander@csc.fi, "intelmq-dev" intelmq-dev@lists.cert.at Cc: "Bernhard Reiter" bernhard@intevation.de Sent: Tuesday, 2 March, 2021 12:32:39 Subject: Re: [IntelMQ-dev] Installing new bots from devel host to production host
Hi Mika,
On 3/2/21 10:57 AM, Mika Silander wrote:
Ok, thanks for confirming the idea is feasible. Where can I find the newer intelmq-manager that is compatible with the development version (2.3.x)?
In the unstable deb/rpm repositories[0] and in the git-repository of the intelmq-manager in the develop branch[1].
If you install intelmq-manager from the (unstable) repository - and therefore the required API as well - you probably need to adapt the API's configuration in regards to the intelmqctl command[2]
And be careful with develop - it may break.
Sebastian
[0]: https://intelmq.readthedocs.io/en/latest/dev/guide.html#id5 [1]: https://github.com/certtools/intelmq-manager/tree/develop/ [2]: https://intelmq.readthedocs.io/en/latest/user/intelmq-api.html#configuring-i...
On 3/1/21 10:43 AM, Mika Silander wrote:
I'd like to install from the package management the following to the production host:
intelmq intelmq-manager
If you get them from the default (stable) repository, make sure that your local repository with the changes is either based on the master or maintenance branch. Otherwise you may see breakages.
From the development machine I'd like to copy+install the bots, runtime.conf, pipeline.conf and the BOTS conf file to the production machine so as to be able to manage them there using intelmq-manager. Intelmq-manager would be an easier tool for the other admins to adopt than intelmqctl.
Apart from what Bernhard mentioned (installing all the packages normally - to /usr/lib/python... - and then installing your fork with pip - to /usr/local/lib/python... overriding the other installation) you can also have a look at the intelmq-tools by Jean-Paul: https://github.com/jhemp/intelmq-tools (but I don't know about the compatibility with recent IntelMQ developments)
With IntelMQ 3.0 (and the preview in the develop branch) such cases will get easier as we get rid of the BOTS file. There is still more room for improvement - looking for contributors - as always :)
Sebastian