Regarding Parallel package [1], I have a suggestion. On my understanding
(from applications on a single machine), Parallel starts Octave sub-engines. When the whole job is finished, the sub-engines do not die. This can create problems in cases where a kind of re-initialization is needed. Consider this case. I am using Parallel along with the expansion of the Genetic Algorithm package for GNU Octave that supports parallelization and bounds [2]. After finishing the first optimization, I want to change the loss-function. In this case, it seems that the sub-engines are not informed about the change. They keep using the version of the loss-function they first encountered. I would suggest adding a function to Parallel package that would perform some kind of re-initialization of the created sub-engines, or at least kill them. [1] https://octave.sourceforge.io/parallel/index.html [2] https://github.com/stevenwatterson/GA -- Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html |
On Sat, Feb 06, 2021 at 12:46:40PM -0600, vrozos wrote:
> Regarding Parallel package [1], I have a suggestion. On my understanding > (from applications on a single machine), Parallel starts Octave sub-engines. > When the whole job is finished, the sub-engines do not die. This can create > problems in cases where a kind of re-initialization is needed. > > Consider this case. I am using Parallel along with the expansion of the > Genetic Algorithm package for GNU Octave that supports parallelization and > bounds [2]. After finishing the first optimization, I want to change the > loss-function. In this case, it seems that the sub-engines are not informed > about the change. They keep using the version of the loss-function they > first encountered. > > I would suggest adding a function to Parallel package that would perform > some kind of re-initialization of the created sub-engines, or at least kill > them. https://octave.sourceforge.io/parallel/package_doc/parcellfun_005fset_005fnproc.html#XREFparcellfun_005fset_005fnproc Olaf -- public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net |
Thank you for your reply.
Maybe a wrapper named "reset_background_processes" calling parcellfun_set_nproc with nproc_arg=0 would help the next unsuspected guy. Maybe not. Parallel computing is a madness. I don't know. V. -- Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html |
In reply to this post by Olaf Till-2
On Wed, Feb 10, 2021 at 07:53:36PM +0100, Olaf Till wrote:
> On Sat, Feb 06, 2021 at 12:46:40PM -0600, vrozos wrote: > > Regarding Parallel package [1], I have a suggestion. On my understanding > > (from applications on a single machine), Parallel starts Octave sub-engines. > > When the whole job is finished, the sub-engines do not die. This can create > > problems in cases where a kind of re-initialization is needed. > > > > Consider this case. I am using Parallel along with the expansion of the > > Genetic Algorithm package for GNU Octave that supports parallelization and > > bounds [2]. After finishing the first optimization, I want to change the > > loss-function. In this case, it seems that the sub-engines are not informed > > about the change. They keep using the version of the loss-function they > > first encountered. a change, just as the main Octave session should see it, without re-initialization. You could test if the main Octave session sees the change. Olaf > > I would suggest adding a function to Parallel package that would perform > > some kind of re-initialization of the created sub-engines, or at least kill > > them. > > But there is such a function: > > https://octave.sourceforge.io/parallel/package_doc/parcellfun_005fset_005fnproc.html#XREFparcellfun_005fset_005fnproc > > Olaf > > -- > public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net -- public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net |
To my knowledge, for the sake of execution speed, Octave does not check
before each execution whether a function file has changed. However, eventually, the main Octave sees the change (after a "clear -f", or after multiple calls). I can confirm this because the change in loss-function (reduce the sampling size) results in faster execution of loss-function (measured with tic-toc). However, the total optimization does not conclude faster, unless I restart Octave (or kill sub-engines). -- Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html |
Free forum by Nabble | Edit this page |