Passing variables up to the GUI

classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Passing variables up to the GUI

John Donoghue-3
Just a general question/observation/what will it take since I havent dug
much into the depth of the octave engine.

Currently, in GUI land, the workspace-model is receiving notification of
variables in scope via the set_workspace slot.
The variables are in string/stringlist format by this point, and are
pretty much just input as is to the model.

If we want to do things like the ability of specifying additional
columns  to display (min, max, avg etc??), the lower level will have to
provide additional strings that are always being sent up to the GUI with
the values as strings, unless we change how it is currently doing it

So how hard would it be (or whats the best/safest way) of being able to
provide the variables up to the GUI where a amount of manipulation can
be done?

I think it will be required if we want to be able to edit/display the
content of variables, change how the GUI displays (without having to
modify the engine each time).


Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

Daniel Sebald
On 04/12/2013 07:43 PM, John Donoghue wrote:

> Just a general question/observation/what will it take since I havent dug
> much into the depth of the octave engine.
>
> Currently, in GUI land, the workspace-model is receiving notification of
> variables in scope via the set_workspace slot.
> The variables are in string/stringlist format by this point, and are
> pretty much just input as is to the model.
>
> If we want to do things like the ability of specifying additional
> columns to display (min, max, avg etc??), the lower level will have to
> provide additional strings that are always being sent up to the GUI with
> the values as strings, unless we change how it is currently doing it
>
> So how hard would it be (or whats the best/safest way) of being able to
> provide the variables up to the GUI where a amount of manipulation can
> be done?
>
> I think it will be required if we want to be able to edit/display the
> content of variables, change how the GUI displays (without having to
> modify the engine each time).

This is related to the discussion from week ago.  One of the things
about octave is that it has all sorts of commands already providing
information.  If there were some way to provide a command to octave and
get the result, then the GUI could be inquiring such information.

For example, I see John just added a new feature for renaming variables
in the workspace from the GUI workspace window.  That is a nice feature.
  However, it's done via callback, when it could be done with a command

"z = x; clear x;"

for example.

Dan
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

John W. Eaton
Administrator
In reply to this post by John Donoghue-3
On 04/12/2013 08:43 PM, John Donoghue wrote:

> Just a general question/observation/what will it take since I havent dug
> much into the depth of the octave engine.
>
> Currently, in GUI land, the workspace-model is receiving notification of
> variables in scope via the set_workspace slot.
> The variables are in string/stringlist format by this point, and are
> pretty much just input as is to the model.
>
> If we want to do things like the ability of specifying additional
> columns to display (min, max, avg etc??), the lower level will have to
> provide additional strings that are always being sent up to the GUI with
> the values as strings, unless we change how it is currently doing it
>
> So how hard would it be (or whats the best/safest way) of being able to
> provide the variables up to the GUI where a amount of manipulation can
> be done?

I initially tried to pass a reference to a symbol_record object, or a
list of them, but couldn't make it work because including symtab.h in
the octave-qt-link.h file caused trouble.

jwe

Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

John Donoghue-3
On 04/12/2013 09:30 PM, John W. Eaton wrote:

> On 04/12/2013 08:43 PM, John Donoghue wrote:
>> Just a general question/observation/what will it take since I havent dug
>> much into the depth of the octave engine.
>>
>> Currently, in GUI land, the workspace-model is receiving notification of
>> variables in scope via the set_workspace slot.
>> The variables are in string/stringlist format by this point, and are
>> pretty much just input as is to the model.
>>
>> If we want to do things like the ability of specifying additional
>> columns to display (min, max, avg etc??), the lower level will have to
>> provide additional strings that are always being sent up to the GUI with
>> the values as strings, unless we change how it is currently doing it
>>
>> So how hard would it be (or whats the best/safest way) of being able to
>> provide the variables up to the GUI where a amount of manipulation can
>> be done?
>
> I initially tried to pass a reference to a symbol_record object, or a
> list of them, but couldn't make it work because including symtab.h in
> the octave-qt-link.h file caused trouble.
>
> jwe
>
Trouble in compiling, or trouble with using the objects?
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

John W. Eaton
Administrator
In reply to this post by Daniel Sebald
On 04/12/2013 09:00 PM, Daniel J Sebald wrote:

> For example, I see John just added a new feature for renaming variables
> in the workspace from the GUI workspace window. That is a nice feature.
> However, it's done via callback, when it could be done with a command
>
> "z = x; clear x;"
>
> for example.

If you send commands like this back to the interpreter, I think
Octave's behavior will seem unreliable and confusing to many users.
For example, what happens for the above command sequence if there is a
a variable called "clear" in the top-level workspace?  Users will
expect the buttons on the GUI to always work.  Telling them, "if you
want the GUI to work correctly, then don't define variables with the
same names as the Octave commands that the GUI uses" doesn't seem very
good to me.

jwe
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

John W. Eaton
Administrator
In reply to this post by John Donoghue-3
On 04/12/2013 09:34 PM, John Donoghue wrote:

> Trouble in compiling, or trouble with using the objects?

Compiling, because we generate a moc file for octave-qt-link.  The moc
files do not include config.h, but many Octave header files need
to have config.h included before they are.

There may be ways to work around this problem, but for now it seemed
easier to do what I did.

jwe
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

Daniel Sebald
In reply to this post by John W. Eaton
On 04/12/2013 08:40 PM, John W. Eaton wrote:

> On 04/12/2013 09:00 PM, Daniel J Sebald wrote:
>
>> For example, I see John just added a new feature for renaming variables
>> in the workspace from the GUI workspace window. That is a nice feature.
>> However, it's done via callback, when it could be done with a command
>>
>> "z = x; clear x;"
>>
>> for example.
>
> If you send commands like this back to the interpreter, I think
> Octave's behavior will seem unreliable and confusing to many users.
> For example, what happens for the above command sequence if there is a
> a variable called "clear" in the top-level workspace? Users will
> expect the buttons on the GUI to always work. Telling them, "if you
> want the GUI to work correctly, then don't define variables with the
> same names as the Octave commands that the GUI uses" doesn't seem very
> good to me.

There is no requirement that such a function behave exactly as the
interpreter that the user has.  As with the "corefeval" I proposed the
other day, it is that underlying builtin code that is what the GUI has
interest in.  So, the commands that go from the GUI's internal access
bypass that first level of indirection that otherwise would be M-script
overrides, variable overrides, etc.  Neither would these commands appear
in the history list.  The best way to describe "corefeval" is that
builtins cannot be overriden.

Dan
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

John W. Eaton
Administrator
On 04/12/2013 10:31 PM, Daniel J Sebald wrote:

> There is no requirement that such a function behave exactly as the
> interpreter that the user has. As with the "corefeval" I proposed the
> other day, it is that underlying builtin code that is what the GUI has
> interest in. So, the commands that go from the GUI's internal access
> bypass that first level of indirection that otherwise would be M-script
> overrides, variable overrides, etc. Neither would these commands appear
> in the history list. The best way to describe "corefeval" is that
> builtins cannot be overriden.

Maybe we can consider something like that later but I don't have time
now to think about that kind of change or try to make it work.

jwe

Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

John W. Eaton
Administrator
On 04/12/2013 10:39 PM, John W. Eaton wrote:

> On 04/12/2013 10:31 PM, Daniel J Sebald wrote:
>
>> There is no requirement that such a function behave exactly as the
>> interpreter that the user has. As with the "corefeval" I proposed the
>> other day, it is that underlying builtin code that is what the GUI has
>> interest in. So, the commands that go from the GUI's internal access
>> bypass that first level of indirection that otherwise would be M-script
>> overrides, variable overrides, etc. Neither would these commands appear
>> in the history list. The best way to describe "corefeval" is that
>> builtins cannot be overriden.
>
> Maybe we can consider something like that later but I don't have time
> now to think about that kind of change or try to make it work.

But, one quick question about what you've proposed.  If you send a
command like

   clear x

to the interpreter, how does it know to bypass variables for the symbol
"clear" but not "x"?  Perhaps you can describe something that will work
for a simple case like this.  Will it generalize to any arbitrarily
complex list of statements?

Since you are calling your function "corefeval", do you see this as
something that is only intended to call functions, or is it supposed to
evaluate arbitrary expressions?

jwe

Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

Daniel Sebald
In reply to this post by John W. Eaton
On 04/12/2013 09:39 PM, John W. Eaton wrote:

> Maybe we can consider something like that later but I don't have time
> now to think about that kind of change or try to make it work.

I understand.  I'm just suggesting that rather than try to come up with
some callback to send data, that down the road such features as editing
data can be gotten in a more general way.  I suppose there are some
inefficiency in sending data back and forth via command.  But if what is
being modified is some big matrix, and feature that displays that will
be equally intensive, such as loading a Qt model and viewtable.

Dan
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

Daniel Sebald
In reply to this post by John W. Eaton
On 04/12/2013 09:45 PM, John W. Eaton wrote:

> On 04/12/2013 10:39 PM, John W. Eaton wrote:
>> On 04/12/2013 10:31 PM, Daniel J Sebald wrote:
>>
>>> There is no requirement that such a function behave exactly as the
>>> interpreter that the user has. As with the "corefeval" I proposed the
>>> other day, it is that underlying builtin code that is what the GUI has
>>> interest in. So, the commands that go from the GUI's internal access
>>> bypass that first level of indirection that otherwise would be M-script
>>> overrides, variable overrides, etc. Neither would these commands appear
>>> in the history list. The best way to describe "corefeval" is that
>>> builtins cannot be overriden.
>>
>> Maybe we can consider something like that later but I don't have time
>> now to think about that kind of change or try to make it work.
>
> But, one quick question about what you've proposed. If you send a
> command like
>
> clear x
>
> to the interpreter, how does it know to bypass variables for the symbol
> "clear" but not "x"? Perhaps you can describe something that will work
> for a simple case like this. Will it generalize to any arbitrarily
> complex list of statements?
>
> Since you are calling your function "corefeval", do you see this as
> something that is only intended to call functions, or is it supposed to
> evaluate arbitrary expressions?

It would be a C++ only function (user can't access), similar to the
"feval" builtin.  But where feval builtin might be

octave_value& feval (command, ...)
   1) Look up where the source of command is.  I'm not
      certain how that works, but I assume there is some
      table that directs the code execution to some builtin
      code, or variable evaluation, or compiled script code--
      however the users sets up the environment.
   2) Evaluate the code and put it into an octave_value
      format.
   3) Handle that code based on how the user has set up the
      output variables.  If no output arguments, it gets
      formated, printed, sent to pager.  If output arguments,
      it gets saved in variables.

octave_value& corefeval (command, ...)
   1) Direct execution directly to core builtins.
   2) Evaluate the code and put it into an octave_value
      format.
   3) Make that octave_value data available to the GUI.

So, there would be that part on Octave core side.  Admittedly, I'm not
sure how difficult it is and what issues would arise.  But, it is sort
of simpler than the work that feval needs to do.  If right now the function
address of the builtin is lost once the user overrides, then that's a
problem.

On the GUI side and interface, there is a bit more work.  I suppose
there would need to be two queues, maybe one for the user input
commands, and one for the "callback" commands.  The GUI needs to queue
up the command and also what object wants the data.  The object that
wants the data has to know what it asked for and how to interpret it.
Maybe it asks for "x".  Or maybe it asks for "size(x)".  Or maybe
"x(1:10,21:30)".

Dan
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

Daniel Sebald
On 04/12/2013 10:14 PM, Daniel J Sebald wrote:

> 2) Evaluate the code and put it into an octave_value
> format.

Writing to fast.  "Evaluate the code and put *the data* into..."

> 3) Handle that code based on how the user has set up the

"Handle that *data* based..."

Dan
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

Daniel Sebald
In reply to this post by Daniel Sebald
On 04/12/2013 10:14 PM, Daniel J Sebald wrote:

> On the GUI side and interface, there is a bit more work. I suppose there
> would need to be two queues, maybe one for the user input commands, and
> one for the "callback" commands. The GUI needs to queue up the command
> and also what object wants the data. The object that wants the data has
> to know what it asked for and how to interpret it. Maybe it asks for
> "x". Or maybe it asks for "size(x)". Or maybe "x(1:10,21:30)".

Followup thought.  Maybe the thing to do is instead of attempting the
"corefeval" function first, is to just create some octave_value object
on the core side, say a small matrix, and see if that can be sent over
by reference using

emit octave_result (qobject, ovres);

to some hunk of code on the GUI side that will just toss up a dialog
showing the contents of the object.  I'll just place it in one of the
existing callback functions.  It will create some annoying, odd dialogs
randomly appearing, but it would just be proof of concept.  I'll try
that late next week sometime.

Dan
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

John W. Eaton
Administrator
On 04/12/2013 11:32 PM, Daniel J Sebald wrote:

> On 04/12/2013 10:14 PM, Daniel J Sebald wrote:
>
>> On the GUI side and interface, there is a bit more work. I suppose there
>> would need to be two queues, maybe one for the user input commands, and
>> one for the "callback" commands. The GUI needs to queue up the command
>> and also what object wants the data. The object that wants the data has
>> to know what it asked for and how to interpret it. Maybe it asks for
>> "x". Or maybe it asks for "size(x)". Or maybe "x(1:10,21:30)".
>
> Followup thought. Maybe the thing to do is instead of attempting the
> "corefeval" function first, is to just create some octave_value object
> on the core side, say a small matrix, and see if that can be sent over
> by reference using
>
> emit octave_result (qobject, ovres);
>
> to some hunk of code on the GUI side that will just toss up a dialog
> showing the contents of the object.

I predict crashes because of reference counting operations that are not
thread safe.

jwe

Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

Daniel Sebald
On 04/12/2013 10:37 PM, John W. Eaton wrote:

> On 04/12/2013 11:32 PM, Daniel J Sebald wrote:
>> On 04/12/2013 10:14 PM, Daniel J Sebald wrote:
>>
>>> On the GUI side and interface, there is a bit more work. I suppose there
>>> would need to be two queues, maybe one for the user input commands, and
>>> one for the "callback" commands. The GUI needs to queue up the command
>>> and also what object wants the data. The object that wants the data has
>>> to know what it asked for and how to interpret it. Maybe it asks for
>>> "x". Or maybe it asks for "size(x)". Or maybe "x(1:10,21:30)".
>>
>> Followup thought. Maybe the thing to do is instead of attempting the
>> "corefeval" function first, is to just create some octave_value object
>> on the core side, say a small matrix, and see if that can be sent over
>> by reference using
>>
>> emit octave_result (qobject, ovres);
>>
>> to some hunk of code on the GUI side that will just toss up a dialog
>> showing the contents of the object.
>
> I predict crashes because of reference counting operations that are not
> thread safe.

Eh, we'll see.  Octave is running in a QThread, so Qt's scheme is the
one influencing garbage collection.  Documentation says that the slots
are all processed immediately upon a signal so once that is all done,
the reference can be discarded.

Dan
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

Michael Goffioul
On Fri, Apr 12, 2013 at 11:55 PM, Daniel J Sebald <[hidden email]> wrote:
On 04/12/2013 10:37 PM, John W. Eaton wrote:
On 04/12/2013 11:32 PM, Daniel J Sebald wrote:
On 04/12/2013 10:14 PM, Daniel J Sebald wrote:

On the GUI side and interface, there is a bit more work. I suppose there
would need to be two queues, maybe one for the user input commands, and
one for the "callback" commands. The GUI needs to queue up the command
and also what object wants the data. The object that wants the data has
to know what it asked for and how to interpret it. Maybe it asks for
"x". Or maybe it asks for "size(x)". Or maybe "x(1:10,21:30)".

Followup thought. Maybe the thing to do is instead of attempting the
"corefeval" function first, is to just create some octave_value object
on the core side, say a small matrix, and see if that can be sent over
by reference using

emit octave_result (qobject, ovres);

to some hunk of code on the GUI side that will just toss up a dialog
showing the contents of the object.

I predict crashes because of reference counting operations that are not
thread safe.

Eh, we'll see.

Trust me, it'll crash. I've tried that in QtHandles and that's the reason I implemented atomic refcount in octave.
 
 Octave is running in a QThread, so Qt's scheme is the one influencing garbage collection.

I'm not sure what you refer to with "garbage collection", we're talking about C++ here. And Qt is completely unaware of the octave-internal reference counting.
 
 Documentation says that the slots are all processed immediately upon a signal so once that is all done, the reference can be discarded.

That is, if the QObject receiver lives in the same thread that fires the signal. Otherwise, the default is to queue the signal in the eventloop of the thread owning the QObject (and for that to work, the target thread must have a running eventloop). You can change that behavior and make the signal emission blocking, but then you've the problem of deadlocking your threads (something I also experienced in QtHandles).

Michael.

Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

Daniel Sebald
On 04/13/2013 10:52 AM, Michael Goffioul wrote:

> On Fri, Apr 12, 2013 at 11:55 PM, Daniel J Sebald
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     On 04/12/2013 10:37 PM, John W. Eaton wrote:
>
>         On 04/12/2013 11:32 PM, Daniel J Sebald wrote:
>
>             On 04/12/2013 10:14 PM, Daniel J Sebald wrote:
>
>                 On the GUI side and interface, there is a bit more work.
>                 I suppose there
>                 would need to be two queues, maybe one for the user
>                 input commands, and
>                 one for the "callback" commands. The GUI needs to queue
>                 up the command
>                 and also what object wants the data. The object that
>                 wants the data has
>                 to know what it asked for and how to interpret it. Maybe
>                 it asks for
>                 "x". Or maybe it asks for "size(x)". Or maybe
>                 "x(1:10,21:30)".
>
>
>             Followup thought. Maybe the thing to do is instead of
>             attempting the
>             "corefeval" function first, is to just create some
>             octave_value object
>             on the core side, say a small matrix, and see if that can be
>             sent over
>             by reference using
>
>             emit octave_result (qobject, ovres);
>
>             to some hunk of code on the GUI side that will just toss up
>             a dialog
>             showing the contents of the object.
>
>
>         I predict crashes because of reference counting operations that
>         are not
>         thread safe.
>
>
>     Eh, we'll see.
>
>
> Trust me, it'll crash. I've tried that in QtHandles and that's the
> reason I implemented atomic refcount in octave.
>
>       Octave is running in a QThread, so Qt's scheme is the one
>     influencing garbage collection.
>
>
> I'm not sure what you refer to with "garbage collection", we're talking
> about C++ here. And Qt is completely unaware of the octave-internal
> reference counting.
>
>       Documentation says that the slots are all processed immediately
>     upon a signal so once that is all done, the reference can be discarded.
>
>
> That is, if the QObject receiver lives in the same thread that fires the
> signal. Otherwise, the default is to queue the signal in the eventloop
> of the thread owning the QObject (and for that to work, the target
> thread must have a running eventloop). You can change that behavior and
> make the signal emission blocking, but then you've the problem of
> deadlocking your threads (something I also experienced in QtHandles).

I put a really rough prototype on the patch list last night.  Seems to
function fine.

The documentation on connections is here:

"
http://qt-project.org/doc/qt-4.8/threads-qobject.html#signals-and-slots-across-threads

The connection type can be specified by passing an additional argument
to connect(). Be aware that using direct connections when the sender and
receiver live in different threads is unsafe if an event loop is running
in the receiver's thread, for the same reason that calling any function
on an object living in another thread is unsafe.
"

OK, I see now that no argument for the connection type results in
*either* a direct connection (e.g., when within the same event loop) or
a queued connection (e.g., across threads).  If one wants a direct
connection, a value of Qt::DirectConnection can be given.

Well, let's think about this.  How is this going to crash from a too
deep reference count?  Would it be the case that if the worker thread
(Octave core) keeps emitting signals while the GUI thread doesn't
respond to them?  Is that the bad scenario you are describing?  Or is it
that the worker thread emits a signal, then that queued signal creates a
"knot in the thread" so that when it continues onward it can only build
up references in its computations and can't do garbage collection properly?

This is one reason I've tried to eliminate any sort of ->exec () inside
of the GUI thread.  My feeling about GUI is it should never be waiting
on anything.  If from time to time it does something that is momentarily
slow (e.g., opening a file), that's fine.  But to toss up a dialog box
and halt the GUI waiting for user input is bad.  That's why
DeleteOnClose is such a nice thing.

The situation we are running into with the GUI is that the worker thread
is the one with delay.  I tried this the other night with John's
workspace-variable-name-change feature.  I set Octave to running a
two-minute command, then altered the name of the variable.  The variable
name did not change upon hitting return.  Upon Octave core finishing,
then the name of the variable mysteriously changed (i.e., my short term
memory doesn't work more beyond a minute).  I don't think that is
unreasonable.  Octave commands have to remain sequential; to do
otherwise would really get to be tricky programming.

There are things that could be done on the GUI side to prevent dangerous
building up of queued signals.  For example, don't either queue an
Octave command or signal an Octave command (effectively the same as
queueing) if it has already been queued.  For example, if the user asks
to change the name of a variable, set some flag in the object.  When the
slot is called, clear that flag.  If the flag hasn't been cleared, don't
send another Octave command, maybe issue a Critical message dialog box
like "Busy Processing".  But I doubt a user is going to keep changing
variable names if they don't seem to change immediately, so I wouldn't
go about doing such a thing for no compelling reason.

There is this option Qt::BlockingQueuedConnection that we could use.
This runs the risk of, as you said, deadlocking the threads.  But that
might be fine in this case so long as there is no ->exec() in the GUI
event loop.  If the deadlocking is placed at moments where the GUI just
accepts the data and stores it somewhere, builds a table, etc., I don't
think that is too critical.  If just the connections that are requesting
octave_value_list data are BlockingQueued, then these are the GUI
routines that are just wanting some data and not much else.  I'm not
sure using BlockingQueued is a good idea though in a complex system; not
without some thought anyway.  There is too much risk of some other
programmer putting in an ->exec () into the code not realizing that is bad.

In summary, yeah there are concerns, but it seems to me that Qt has
worked out a lot of details so that there seems to be some kind of
solution if just staying within the design.  The other approach is to
keep building function callbacks, which is something Qt documentation
suggests not doing.

Dan
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

John W. Eaton
Administrator
On 04/13/2013 03:29 PM, Daniel J Sebald wrote:

> The situation we are running into with the GUI is that the worker thread
> is the one with delay. I tried this the other night with John's
> workspace-variable-name-change feature. I set Octave to running a
> two-minute command, then altered the name of the variable. The variable
> name did not change upon hitting return. Upon Octave core finishing,
> then the name of the variable mysteriously changed (i.e., my short term
> memory doesn't work more beyond a minute). I don't think that is
> unreasonable. Octave commands have to remain sequential; to do otherwise
> would really get to be tricky programming.

Yes, but what commands are "safe" to execute in this fashion?

Let's say you are running some long script and you attempt to change the
name of a variable with the GUI workspace browser.  That appears to work
when the request is queued, but what happens if the code you are running
deletes the variable before your request is processed?

jwe
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

Daniel Sebald
On 04/13/2013 03:43 PM, John W. Eaton wrote:

> On 04/13/2013 03:29 PM, Daniel J Sebald wrote:
>
>> The situation we are running into with the GUI is that the worker thread
>> is the one with delay. I tried this the other night with John's
>> workspace-variable-name-change feature. I set Octave to running a
>> two-minute command, then altered the name of the variable. The variable
>> name did not change upon hitting return. Upon Octave core finishing,
>> then the name of the variable mysteriously changed (i.e., my short term
>> memory doesn't work more beyond a minute). I don't think that is
>> unreasonable. Octave commands have to remain sequential; to do otherwise
>> would really get to be tricky programming.
>
> Yes, but what commands are "safe" to execute in this fashion?
>
> Let's say you are running some long script and you attempt to change the
> name of a variable with the GUI workspace browser. That appears to work
> when the request is queued, but what happens if the code you are running
> deletes the variable before your request is processed?

I don't believe that is a problem.  In the prototype:

    retval = eval_string (first_command->toStdString (), false,
parse_status, first_nout);
    if (!parse_status && !error_state)
      {
        emit octave_command_result (first_qobject, retval);
      }

If the command fails, nothing is emitted back to the GUI, nor is any
internal workspace variable modified.  Or it could send back the result
with the first argument being success or fail.  The responsibility is on
the GUI programmer to understand a bit about what is going on, but I
think that is fine.

For example, if the user clicks on a variable meant to display its data
while Octave core is processing and unbeknown to the GUI has been
deleted.  It queues, in this case,

size(x);
x;

Since the Octave commands are handled sequentially, there won't be a
situation of size(x) passing and x failing.  So the slots of the
requesting object can be like a tiny state machine.  If the Octave
core-side code doesn't emit that sequence of octave_value_list, then the
slots for displaying data won't be called.  If another such variable is
requested, the little state machine in the requesting object is reset.

Now, let's say it gets to the point where the GUI has acquired the data
and want's to modify something.  It queues command

x(3,8) = <user input>;

Well, before changing that value in its display, it needs to get a
signal that was successfully set.  Otherwise, the value remains
unchanged.  Or maybe the signal could come back with the boolean result
of unsuccess in which case the GUI can display "Variable has disappeared
from workspace".

Dan
Reply | Threaded
Open this post in threaded view
|

Re: Passing variables up to the GUI

Michael Goffioul
In reply to this post by Daniel Sebald
On Sat, Apr 13, 2013 at 3:29 PM, Daniel J Sebald <[hidden email]> wrote:
Well, let's think about this.  How is this going to crash from a too deep reference count?  Would it be the case that if the worker thread (Octave core) keeps emitting signals while the GUI thread doesn't respond to them?  Is that the bad scenario you are describing?

The problem is very basic: 2 (or more) threads trying to access/modify the reference count of an object concurrently. The copy constructor of octave_value does:

    rep->count++;

The destructor of octave_value does:

    if (--rep->count == 0)
        delete rep;

Those operations are *not* atomic, this means that while one thread is incrementing the counter, another one is decrementing it. The result is undefined. The best case scenario, the counter never reaches 0 and you get a memory leak. Worst case scenario, you try to access an object that has been deleted and you get a crash.

There's not much you can do to prevent this, except making the operation (increment/decrement) on the counter atomic at the CPU level. This is typically what shared_ptr<> implementation does.

Another ad-hoc solution would be to have the emitter thread to keep a reference of the passed object, until it is sure that the receiver thread has finished with it (for instance by receiving a signal back). But even then, you might get into trouble, as you don't know where and when all existing references will be deleted: a queued signal must keep a reference somewhere to the passed objects, which get discarded at some point, but is it in the emitter thread or the receiver thread?
 
There is this option Qt::BlockingQueuedConnection that we could use. This runs the risk of, as you said, deadlocking the threads.  But that might be fine in this case so long as there is no ->exec() in the GUI event loop.  If the deadlocking is placed at moments where the GUI just accepts the data and stores it somewhere, builds a table, etc., I don't think that is too critical.  If just the connections that are requesting octave_value_list data are BlockingQueued, then these are the GUI routines that are just wanting some data and not much else.  I'm not sure using BlockingQueued is a good idea though in a complex system; not without some thought anyway.  There is too much risk of some other programmer putting in an ->exec () into the code not realizing that is bad.

Blocking calls in threads is probably a bad practice imo.

Michael.

12