Graphics in forked subprocess

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Graphics in forked subprocess

phofman
Hi,

I am trying to write a control script for my set of octave scripts which
must run simultaneously. I am trying to use fork/exec.

testGui.m:

figure();
while true
   pause(1);
   printf('Paused\n');
endwhile


testFork.m:

ctrlPid = ( @() fork() )();
if ctrlPid == 0
   source('testGui.m');
else
   waitpid(-1);
endif


When running testGui directly from terminal, it shows the figure correctly:

octave testGui.m


But when running testFork, the figure never appears while Paused get
printed out - the process testGui.m is clearly running.

octave testFork.m


This problem is under Linux/Ubuntu, older octave 4.2.2. Is perhaps this
issue solved by https://savannah.gnu.org/bugs/?53034 in Octave 5.1?

Thanks a lot.

With regards,

Pavel.


Reply | Threaded
Open this post in threaded view
|

Re: Graphics in forked subprocess

Mike Miller-4
On Sun, May 19, 2019 at 21:04:50 +0200, Pavel Hofman wrote:

> Hi,
>
> I am trying to write a control script for my set of octave scripts which
> must run simultaneously. I am trying to use fork/exec.
>
> testGui.m:
>
> figure();
> while true
>   pause(1);
>   printf('Paused\n');
> endwhile
>
>
> testFork.m:
>
> ctrlPid = ( @() fork() )();
> if ctrlPid == 0
>   source('testGui.m');
> else
>   waitpid(-1);
> endif
>
>
> When running testGui directly from terminal, it shows the figure correctly:
>
> octave testGui.m
>
>
> But when running testFork, the figure never appears while Paused get printed
> out - the process testGui.m is clearly running.
>
> octave testFork.m
>
>
> This problem is under Linux/Ubuntu, older octave 4.2.2. Is perhaps this
> issue solved by https://savannah.gnu.org/bugs/?53034 in Octave 5.1?
No, the issue is that your script is using 'fork' without 'exec'. You
mention 'exec', but it doesn't appear in the above.

It's a well known problem that "fork without exec" does not work with
large libraries and frameworks linked with a program, including Python,
Qt, probably Java. In fact, it's considered deprecated on macOS now to
'fork' a process and not 'exec' a brand new executable image.

Calling 'fork' by itself for creating worker child processes should only
be used carefully in small programs that don't link with large
frameworks, such as Octave.

Can you rework your scripts so that 'fork' is followed immediately by
'exec' to start a new separate Octave process for example?

    ctrl_pid = fork ();
    if (ctrl_pid == 0)
      exec ('octave', 'testGui.m');
    else
      waitpid (ctrl_pid);
    endif

--
mike



signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Graphics in forked subprocess

phofman
Hi Mike,

>
> No, the issue is that your script is using 'fork' without 'exec'. You
> mention 'exec', but it doesn't appear in the above.
>
> It's a well known problem that "fork without exec" does not work with
> large libraries and frameworks linked with a program, including Python,
> Qt, probably Java. In fact, it's considered deprecated on macOS now to
> 'fork' a process and not 'exec' a brand new executable image.
>
> Calling 'fork' by itself for creating worker child processes should only
> be used carefully in small programs that don't link with large
> frameworks, such as Octave.
>
> Can you rework your scripts so that 'fork' is followed immediately by
> 'exec' to start a new separate Octave process for example?
>
>      ctrl_pid = fork ();
>      if (ctrl_pid == 0)
>        exec ('octave', 'testGui.m');
>      else
>        waitpid (ctrl_pid);
>      endif

Thanks a lot for your help. I did use exec. But exec creates two child
processes:  A) octave which runs B)
/usr/lib/x86_64-linux-gnu/octave/4.2.2/exec/x86_64-pc-linux-gnu/octave-gui.
B) runs the actual script while PID of A) is returned by exec(). I need
to have a way to stop the child process from my control script. I wanted
to stop the child process by sending kill(pid) - for that I need pid of
B) - sending kill to A) does not stop B). I have to be able to check B)
has finished/died to restart if needed

That is why I skipped the exec call and sourced octave code directly
after forking.

Please what is the best practice for this kind of control? I know I can
use a side channel, but I want to be able to kill the child reliably
(kill -15 ?) even if it  gets stuck.

I could write the control script in shell (bash) but I would love to
write a single code for linux and windows, if actually possible. I see
Octave for windows uses vbs scripts for startup... perhaps I will have
to resort to that too, as the last resort...

Thanks a lot,

Pavel.




Reply | Threaded
Open this post in threaded view
|

Re: Graphics in forked subprocess

Mike Miller-4
On Sun, May 19, 2019 at 21:52:39 +0200, Pavel Hofman wrote:
> Thanks a lot for your help. I did use exec. But exec creates two child
> processes:  A) octave which runs B)
> /usr/lib/x86_64-linux-gnu/octave/4.2.2/exec/x86_64-pc-linux-gnu/octave-gui.
> B) runs the actual script while PID of A) is returned by exec().

Are you sure the process IDs of A and B are different? In my testing
they are the same. The intent is for the process ID to remain the same.

> I need to
> have a way to stop the child process from my control script. I wanted to
> stop the child process by sending kill(pid) - for that I need pid of B) -
> sending kill to A) does not stop B). I have to be able to check B) has
> finished/died to restart if needed

Sure.

> That is why I skipped the exec call and sourced octave code directly after
> forking.
>
> Please what is the best practice for this kind of control? I know I can use
> a side channel, but I want to be able to kill the child reliably (kill -15
> ?) even if it  gets stuck.

That sounds like a fine approach. Keep in mind there is at least one
known bug with ending an Octave process with SIGTERM in Octave 4.4 and
later versions.

  https://savannah.gnu.org/bugs/?54444)

Feel free to ask for help or report bugs if you run into other problems
starting or controlling Octave subprocesses.

--
mike



signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Graphics in forked subprocess

phofman
Hi Mike,

Dne 19. 05. 19 v 22:53 Mike Miller napsal(a):
> On Sun, May 19, 2019 at 21:52:39 +0200, Pavel Hofman wrote:
>> Thanks a lot for your help. I did use exec. But exec creates two
>> child processes:  A) octave which runs B)
>> /usr/lib/x86_64-linux-gnu/octave/4.2.2/exec/x86_64-pc-linux-gnu/octave-gui.
>>
>>
B) runs the actual script while PID of A) is returned by exec().
>
> Are you sure the process IDs of A and B are different? In my testing
> they are the same. The intent is for the process ID to remain the
> same.

There are two processes for each octave created:

4868 pts/4    S+     0:00 octave testExec.m
  4869 ?        Ssl    0:00
/usr/lib/x86_64-linux-gnu/octave/4.2.2/exec/x86_64-pc-linux-gnu/octave-gui
testExec.m

  4885 ?        S      0:00 octave testGui.m
  4886 ?        Ssl    0:00
/usr/lib/x86_64-linux-gnu/octave/4.2.2/exec/x86_64-pc-linux-gnu/octave-gui
testGui.m

The fork call returns PID 4885. If testExec does not use waitpid() and
4885 gets killed (kill 4885), the child 4886 exits correctly, but 4885
stays running as zombie process and checking with kill(4885, 0) keeps
returning 0 - the process is still running.

I need to start 3 children. The waitpid(-1) should be waiting for any
child and return the child PID which finished. Still my controlling
thread cannot block, it must be able to react to user requests. My idea
was to check for all child PIDs existence with a loop using kill(PID,
0), without blocking with waitpid().

I could have one  thread blocked with waitpid, restarting the respective
children if needed, and another sending the kills if requested by the user.

But GUI (pushbutton callback) and waitpid do not mix as the GUI callback
requires pause/waitfor while waitpid blocks, never reaching that code...

>
>> That is why I skipped the exec call and sourced octave code
>> directly after forking.
>>
>> Please what is the best practice for this kind of control? I know I
>> can use a side channel, but I want to be able to kill the child
>> reliably (kill -15 ?) even if it  gets stuck.
>
> That sounds like a fine approach.

Well, I really do not want to use the side channel as it is not reliable.

> Keep in mind there is at least one
> known bug with ending an Octave process with SIGTERM in Octave 4.4
> and later versions.
>
> https://savannah.gnu.org/bugs/?54444)

Interesting, please any chance of that fixed? I do not know if that
would affect my case but probably yes - I wanted to be sending sigterm
signals.

Please does this fork/exec/kill functionality work in windows octave
build too? If not, no reason to fight this in octave and I whould
directly use bash for linux and vbs for windows somehow.


Thanks a lot,

Pavel.


Reply | Threaded
Open this post in threaded view
|

Re: Re: Graphics in forked subprocess

tmacchant
In reply to this post by phofman
--- pavel.hofman

> Hi Mike,
>
> Dne 19. 05. 19 v 22:53 Mike Miller napsal(a):
> > On Sun, May 19, 2019 at 21:52:39 +0200, Pavel Hofman wrote:
> >> Thanks a lot for your help. I did use exec. But exec creates two
> >> child processes:  A) octave which runs B) /usr/lib/x86_64-linux-gnu/octave/4.2.2/exec/x86_64-pc-linux-gnu/octave-gui.
> >>
> >>
> B) runs the actual script while PID of A) is returned by exec().
> >
> > Are you sure the process IDs of A and B are different? In my testing they are the same. The intent is for the process ID to remain the
> > same.
>
> There are two processes for each octave created:
>
> 4868 pts/4    S+     0:00 octave testExec.m
>  4869 ?        Ssl    0:00
> /usr/lib/x86_64-linux-gnu/octave/4.2.2/exec/x86_64-pc-linux-gnu/octave-gui
> testExec.m
>
>  4885 ?        S      0:00 octave testGui.m
>  4886 ?        Ssl    0:00
> /usr/lib/x86_64-linux-gnu/octave/4.2.2/exec/x86_64-pc-linux-gnu/octave-gui
> testGui.m
>
> The fork call returns PID 4885. If testExec does not use waitpid() and
> 4885 gets killed (kill 4885), the child 4886 exits correctly, but 4885
> stays running as zombie process and checking with kill(4885, 0) keeps
> returning 0 - the process is still running.
>
> I need to start 3 children. The waitpid(-1) should be waiting for any child and return the child PID which finished. Still my controlling thread cannot block, it must be able to react to user requests. My idea was to check for all child PIDs existence with a loop using kill(PID, 0), without blocking with waitpid().
>
> I could have one  thread blocked with waitpid, restarting the respective children if needed, and another sending the kills if requested by the user.
>
> But GUI (pushbutton callback) and waitpid do not mix as the GUI callback requires pause/waitfor while waitpid blocks, never reaching that code...
>
> >
> >> That is why I skipped the exec call and sourced octave code
> >> directly after forking.
> >>
> >> Please what is the best practice for this kind of control? I know I
> >> can use a side channel, but I want to be able to kill the child
> >> reliably (kill -15 ?) even if it  gets stuck.
> >
> > That sounds like a fine approach.
>
> Well, I really do not want to use the side channel as it is not reliable.
>
> > Keep in mind there is at least one known bug with ending an Octave process with SIGTERM in Octave 4.4
> > and later versions.
> >
> > https://savannah.gnu.org/bugs/?54444)
>
> Interesting, please any chance of that fixed? I do not know if that would affect my case but probably yes - I wanted to be sending sigterm signals.
>
> Please does this fork/exec/kill functionality work in windows octave build too? If not, no reason to fight this in octave and I whould directly use bash for linux and vbs for windows somehow.
>
>
> Thanks a lot,
>
> Pavel.
>
>
windows itself does not have fork and octave for native windows does not have fork.

If you want fork on windows, try cygwin or wsl.

Tatsuro


Reply | Threaded
Open this post in threaded view
|

Re: Graphics in forked subprocess

phofman
Hi Tatsuro,

Dne 20. 05. 19 v 12:36 Tatsuro MATSUOKA napsal(a):
>>
> windows itself does not have fork and octave for native windows does not have fork.
>

Thanks a lot for your key info. It means I will not fight octave's
process functions and write separate control scripts for linux and windows.

Pavel.