Re: GUI terminal widget

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

Rik-4
On 05/29/2019 09:00 AM, [hidden email] wrote:
Subject:
Thoughts about a different approach to the GUI terminal widget
From:
"John W. Eaton" [hidden email]
Date:
05/28/2019 08:06 PM
To:
Octave Maintainers List [hidden email]
CC:
[hidden email]
List-Post:
[hidden email]
Content-Transfer-Encoding:
7bit
Precedence:
list
MIME-Version:
1.0
Message-ID:
[hidden email]
Content-Type:
text/plain; charset=utf-8; format=flowed
Message:
1

Way back in January 2012, I posted the following message about changing the terminal widget in the GUI to handle input differently.


http://lists.gnu.org/archive/html/octave-maintainers/2012-01/msg00416.html

I included a simple example written with gtkmm to illustrate the idea of having the terminal widget in the GUI be in control of input and output and feed lines of text to the interpreter.  To properly handle input that spans multiple lines, a push parser is used instead of the current pull parser.  It worked as an example, but was not much help for Octave since we are using Qt.  So now I've reworked it using Qt and you can find my sources here:

  http://hg.octave.org/jwe-qt-gui-with-push-parser

See the NOTES file for build instructions.  That file also contains the following list of open questions that will need to be resolved if we are going to attempt a switch.

  * With this arrangement, would the interpreter have to run in a separate thread?  As the example shows, it's not absolutely necessary. It could offer some advantages, but only if it is possible for the GUI to do useful things while the interpreter is busy.

  * If the interpreter is not running in a separate thread but but the graphics engine is, then what happens to graphics callbacks when the parser is in the middle of parsing an expression?  Or is this not an issue because separate parsers can be used even if there is only one evaluator?  Could it ultimately be possible to have the evaluator running in multiple threads?

  * If the interpreter does run in a separate thread, we still must wait for it to calculate and return a result so we can synchronize input and output.  Otherwise, we may print the next prompt before the output from the previous expression is evaluated and printed.  You'll see this behavior if you build the example program with CALC_USE_INTERPRETER_THREAD defined.

  * The example parser currently also performs evaluations and computes results immediately so it doesn't properly handle expression lists that are split across multiple lines.  Octave wouldn't have this problem because we already build a parse tree then execute it once it is complete.

  * Do we need text position markers to keep track of the prompt position (beginning of current line) when inserting or clearing text? This doesn't seem necessary in the current example, but it doesn't have functions that can clear the screen or otherwise redraw prior output that would cause the position of the cursor in the window to change.

  * The example program doesn't attempt handle multi-line prompts or prompts with invisible characters (color specifications, for example). Fixing that will make the redisplay function significantly more complex.  See, for example, how complicated the default rl_redisplay function is in the readline library.  Unless we actually write a terminal emulator (like the current terminal widgets) then it is not possible to use readline's rl_redisplay function directly.

  * We'll need to implement a pager ourselves, since "less" won't work in this simplified terminal widget.

  * The system function may need to be modified so that external programs that expect to be running in a terminal will continue to work properly.  On Unixy systems, this job can be done with ptys.  I guess Windows systems can use a hidden console?  But if these things are required, are we more or less back to were we were before since we used a pty and hidden console to implement the terminal widgets?  I believe the Emacs start-process function must do similar things, so we might be able to reuse that code.

  * If readline runs in the terminal widget, who owns the command-line history?  Either way, if the GUI is in control of keyboard input, it will need access to the history list and Octave will also need access for the history functions.

Now that I have an almost working (if quite simplistic) example in Qt, I will attempt to modify Octave to use this approach.  From that, I expect many more questions to come up.

jwe

jwe,

I didn't see any comments at https://wiki.octave.org/GUI_terminal_widget so I thought I would bring the conversation back to the mailing list.

From a strategic level, this is a big change.  What do we get from doing this (pros) and what do we lose (cons)?  Are there sufficient benefits to undertake this task?

Although the terminal often causes problems, it also allows for a lot of code re-use.  We use Readline for history and input processing.  We get 'less' and other utilities for free.  In replacing this, do we essentially need to re-invent the wheel?  That's a *lot* of coding effort for something that isn't even particularly Octave related.  Since your time is finite, if you are working on this then there will be other Octave bugs that aren't addressed.  Is the balance worthwhile?  You understand core Octave better than anyone so it may be that your time is better spent resolving 50 bugs there, rather than writing more generic terminal emulation code that other programmers could do.

Just some ideas on what to answer before going forward.

--Rik

Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

Richard Crozier


On 03/06/2019 17:34, Rik wrote:

> On 05/29/2019 09:00 AM, [hidden email] wrote:
>> Subject:
>> Thoughts about a different approach to the GUI terminal widget
>> From:
>> "John W. Eaton" <[hidden email]>
>> Date:
>> 05/28/2019 08:06 PM
>>
>> To:
>> Octave Maintainers List <[hidden email]>
>> CC:
>> [hidden email]
>>
>> List-Post:
>> <mailto:[hidden email]>
>> Content-Transfer-Encoding:
>> 7bit
>> Precedence:
>> list
>> MIME-Version:
>> 1.0
>> Message-ID:
>> <[hidden email]>
>> Content-Type:
>> text/plain; charset=utf-8; format=flowed
>> Message:
>> 1
>>
>>
>> Way back in January 2012, I posted the following message about
>> changing the terminal widget in the GUI to handle input differently.
>>
>>
>> http://lists.gnu.org/archive/html/octave-maintainers/2012-01/msg00416.html
>>
>>
>> I included a simple example written with gtkmm to illustrate the idea
>> of having the terminal widget in the GUI be in control of input and
>> output and feed lines of text to the interpreter.  To properly handle
>> input that spans multiple lines, a push parser is used instead of the
>> current pull parser.  It worked as an example, but was not much help
>> for Octave since we are using Qt.  So now I've reworked it using Qt
>> and you can find my sources here:
>>
>> http://hg.octave.org/jwe-qt-gui-with-push-parser
>>
>> See the NOTES file for build instructions.  That file also contains
>> the following list of open questions that will need to be resolved if
>> we are going to attempt a switch.
>>
>>   * With this arrangement, would the interpreter have to run in a
>> separate thread?  As the example shows, it's not absolutely necessary.
>> It could offer some advantages, but only if it is possible for the GUI
>> to do useful things while the interpreter is busy.
>>
>>   * If the interpreter is not running in a separate thread but but the
>> graphics engine is, then what happens to graphics callbacks when the
>> parser is in the middle of parsing an expression?  Or is this not an
>> issue because separate parsers can be used even if there is only one
>> evaluator?  Could it ultimately be possible to have the evaluator
>> running in multiple threads?
>>
>>   * If the interpreter does run in a separate thread, we still must
>> wait for it to calculate and return a result so we can synchronize
>> input and output.  Otherwise, we may print the next prompt before the
>> output from the previous expression is evaluated and printed.  You'll
>> see this behavior if you build the example program with
>> CALC_USE_INTERPRETER_THREAD defined.
>>
>>   * The example parser currently also performs evaluations and
>> computes results immediately so it doesn't properly handle expression
>> lists that are split across multiple lines.  Octave wouldn't have this
>> problem because we already build a parse tree then execute it once it
>> is complete.
>>
>>   * Do we need text position markers to keep track of the prompt
>> position (beginning of current line) when inserting or clearing text?
>> This doesn't seem necessary in the current example, but it doesn't
>> have functions that can clear the screen or otherwise redraw prior
>> output that would cause the position of the cursor in the window to
>> change.
>>
>>   * The example program doesn't attempt handle multi-line prompts or
>> prompts with invisible characters (color specifications, for example).
>> Fixing that will make the redisplay function significantly more
>> complex.  See, for example, how complicated the default rl_redisplay
>> function is in the readline library.  Unless we actually write a
>> terminal emulator (like the current terminal widgets) then it is not
>> possible to use readline's rl_redisplay function directly.
>>
>>   * We'll need to implement a pager ourselves, since "less" won't work
>> in this simplified terminal widget.
>>
>>   * The system function may need to be modified so that external
>> programs that expect to be running in a terminal will continue to work
>> properly.  On Unixy systems, this job can be done with ptys.  I guess
>> Windows systems can use a hidden console?  But if these things are
>> required, are we more or less back to were we were before since we
>> used a pty and hidden console to implement the terminal widgets?  I
>> believe the Emacs start-process function must do similar things, so we
>> might be able to reuse that code.
>>
>>   * If readline runs in the terminal widget, who owns the command-line
>> history?  Either way, if the GUI is in control of keyboard input, it
>> will need access to the history list and Octave will also need access
>> for the history functions.
>>
>> Now that I have an almost working (if quite simplistic) example in Qt,
>> I will attempt to modify Octave to use this approach. From that, I
>> expect many more questions to come up.
>>
>> jwe
>
> jwe,
>
> I didn't see any comments at https://wiki.octave.org/GUI_terminal_widget
> so I thought I would bring the conversation back to the mailing list.
>
>  From a strategic level, this is a big change.  What do we get from
> doing this (pros) and what do we lose (cons)?  Are there sufficient
> benefits to undertake this task?
>
> Although the terminal often causes problems, it also allows for a lot of
> code re-use.  We use Readline for history and input processing.  We get
> 'less' and other utilities for free.  In replacing this, do we
> essentially need to re-invent the wheel? That's a *lot* of coding effort
> for something that isn't even particularly Octave related.  Since your
> time is finite, if you are working on this then there will be other
> Octave bugs that aren't addressed.  Is the balance worthwhile?  You
> understand core Octave better than anyone so it may be that your time is
> better spent resolving 50 bugs there, rather than writing more generic
> terminal emulation code that other programmers could do.
>
> Just some ideas on what to answer before going forward.
>
> --Rik
>


The terminal code in Octave is a mess, I have gone in to try and fix
bugs, but it is so complex (for example there is, or at least was,
completely different terminal code for Windows and Linux), that fixing
simple bugs had a huge overhead in understanding the terminal code.

Given that no-one has done anything to improve this for a long time
perhaps JWE's help is needed to get the terminal to a state where others
can help pick things up, especially if he has done some of the work already.

I also get the impression he's enjoying getting this working. Enjoyment
is important for a project maintainer!

Incidentally, do you really need a pager in the gui? You have scrollbars
on the windows. All you really need is a keyboard shortcut to scroll up
don't you? Like SHIFT+PageUp

I also wonder if there is a lot which can be lifted/translated from the
Jupyter Qt console:

https://github.com/jupyter/qtconsole

It has a history and a pager.

Regards,

Richard







The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

siko1056
On 6/4/19 11:35 AM, CROZIER Richard wrote:

>
> On 03/06/2019 17:34, Rik wrote:
>>
>> I didn't see any comments at https://wiki.octave.org/GUI_terminal_widget
>> so I thought I would bring the conversation back to the mailing list.
>>
>>  From a strategic level, this is a big change.  What do we get from
>> doing this (pros) and what do we lose (cons)?  Are there sufficient
>> benefits to undertake this task?
>>
>> Although the terminal often causes problems, it also allows for a lot of
>> code re-use.  We use Readline for history and input processing.  We get
>> 'less' and other utilities for free.  In replacing this, do we
>> essentially need to re-invent the wheel? That's a *lot* of coding effort
>> for something that isn't even particularly Octave related.  Since your
>> time is finite, if you are working on this then there will be other
>> Octave bugs that aren't addressed.  Is the balance worthwhile?  You
>> understand core Octave better than anyone so it may be that your time is
>> better spent resolving 50 bugs there, rather than writing more generic
>> terminal emulation code that other programmers could do.
>>
>> Just some ideas on what to answer before going forward.
>>
>> --Rik
>>
>
>
> The terminal code in Octave is a mess, I have gone in to try and fix
> bugs, but it is so complex (for example there is, or at least was,
> completely different terminal code for Windows and Linux), that fixing
> simple bugs had a huge overhead in understanding the terminal code.
>
> Given that no-one has done anything to improve this for a long time
> perhaps JWE's help is needed to get the terminal to a state where others
> can help pick things up, especially if he has done some of the work already.
>
> I also get the impression he's enjoying getting this working. Enjoyment
> is important for a project maintainer!
>
> Incidentally, do you really need a pager in the gui? You have scrollbars
> on the windows. All you really need is a keyboard shortcut to scroll up
> don't you? Like SHIFT+PageUp
>
> I also wonder if there is a lot which can be lifted/translated from the
> Jupyter Qt console:
>
> https://github.com/jupyter/qtconsole
>
> It has a history and a pager.
>
> Regards,
>
> Richard
>

Looks cool and is something I regularly dream of in Octave, but
qtconsole is written in Python.  Unless Octave moves in this direction,
I think this is difficult to realize.  Another promising C++ candidate
is qtermwidet. Both listed here:

   https://wiki.octave.org/GUI_terminal_widget#Alternatives

Additionally, I raised a question about Unicode support of this new
solution, which occasionally results in bug reports.


https://wiki.octave.org/GUI_terminal_widget#International_Characters_Support

Best regards,

Kai

Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

Richard Crozier


On 04/06/2019 13:10, Kai Torben Ohlhus wrote:

> On 6/4/19 11:35 AM, CROZIER Richard wrote:
>>
>> On 03/06/2019 17:34, Rik wrote:
>>>
>>> I didn't see any comments at https://wiki.octave.org/GUI_terminal_widget
>>> so I thought I would bring the conversation back to the mailing list.
>>>
>>>   From a strategic level, this is a big change.  What do we get from
>>> doing this (pros) and what do we lose (cons)?  Are there sufficient
>>> benefits to undertake this task?
>>>
>>> Although the terminal often causes problems, it also allows for a lot of
>>> code re-use.  We use Readline for history and input processing.  We get
>>> 'less' and other utilities for free.  In replacing this, do we
>>> essentially need to re-invent the wheel? That's a *lot* of coding effort
>>> for something that isn't even particularly Octave related.  Since your
>>> time is finite, if you are working on this then there will be other
>>> Octave bugs that aren't addressed.  Is the balance worthwhile?  You
>>> understand core Octave better than anyone so it may be that your time is
>>> better spent resolving 50 bugs there, rather than writing more generic
>>> terminal emulation code that other programmers could do.
>>>
>>> Just some ideas on what to answer before going forward.
>>>
>>> --Rik
>>>
>>
>>
>> The terminal code in Octave is a mess, I have gone in to try and fix
>> bugs, but it is so complex (for example there is, or at least was,
>> completely different terminal code for Windows and Linux), that fixing
>> simple bugs had a huge overhead in understanding the terminal code.
>>
>> Given that no-one has done anything to improve this for a long time
>> perhaps JWE's help is needed to get the terminal to a state where others
>> can help pick things up, especially if he has done some of the work already.
>>
>> I also get the impression he's enjoying getting this working. Enjoyment
>> is important for a project maintainer!
>>
>> Incidentally, do you really need a pager in the gui? You have scrollbars
>> on the windows. All you really need is a keyboard shortcut to scroll up
>> don't you? Like SHIFT+PageUp
>>
>> I also wonder if there is a lot which can be lifted/translated from the
>> Jupyter Qt console:
>>
>> https://github.com/jupyter/qtconsole
>>
>> It has a history and a pager.
>>
>> Regards,
>>
>> Richard
>>
>
> Looks cool and is something I regularly dream of in Octave, but
> qtconsole is written in Python.  Unless Octave moves in this direction,
> I think this is difficult to realize.  Another promising C++ candidate
> is qtermwidet. Both listed here:
>
>     https://wiki.octave.org/GUI_terminal_widget#Alternatives
>
> Additionally, I raised a question about Unicode support of this new
> solution, which occasionally results in bug reports.
>
>
> https://wiki.octave.org/GUI_terminal_widget#International_Characters_Support
>
> Best regards,
>
> Kai
>

Although it is written in python, the python methods and functions
should map quite well to the underlying Qt C++ methods and functions. I
was thinking a translation might not be too bad, or at least easier than
starting from scratch. However, your qtermwidget also looks like a good
starting point.

If only Octave had qt bindings ... :-)


The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

Rik-4
In reply to this post by Richard Crozier
On 06/04/2019 02:35 AM, CROZIER Richard wrote:

>
>> jwe,
>>
>> I didn't see any comments at https://wiki.octave.org/GUI_terminal_widget
>> so I thought I would bring the conversation back to the mailing list.
>>
>>  From a strategic level, this is a big change.  What do we get from
>> doing this (pros) and what do we lose (cons)?  Are there sufficient
>> benefits to undertake this task?
>>
>> Although the terminal often causes problems, it also allows for a lot of
>> code re-use.  We use Readline for history and input processing.  We get
>> 'less' and other utilities for free.  In replacing this, do we
>> essentially need to re-invent the wheel? That's a *lot* of coding effort
>> for something that isn't even particularly Octave related.  Since your
>> time is finite, if you are working on this then there will be other
>> Octave bugs that aren't addressed.  Is the balance worthwhile?  You
>> understand core Octave better than anyone so it may be that your time is
>> better spent resolving 50 bugs there, rather than writing more generic
>> terminal emulation code that other programmers could do.
>>
>> Just some ideas on what to answer before going forward.
>>
>> --Rik
>>
>
> The terminal code in Octave is a mess, I have gone in to try and fix
> bugs, but it is so complex (for example there is, or at least was,
> completely different terminal code for Windows and Linux), that fixing
> simple bugs had a huge overhead in understanding the terminal code.
Yeah, it is bad.  But are we leaping out of the frying pan and in to the
fire?  I.e., it might take a long time to change over to a new way of doing
things, and then we discover that there are other things, like threading,
which make it even more painful then supporting the terminal.
>
> Given that no-one has done anything to improve this for a long time
> perhaps JWE's help is needed to get the terminal to a state where others
> can help pick things up, especially if he has done some of the work already.
>
> I also get the impression he's enjoying getting this working. Enjoyment
> is important for a project maintainer!
That's fair enough, and it should be entered in the "pro" column if true.
>
> Incidentally, do you really need a pager in the gui? You have scrollbars
> on the windows. All you really need is a keyboard shortcut to scroll up
> don't you? Like SHIFT+PageUp
We would need it if we wanted to maintain compatibility with Matlab.  But
you're right that pagers are much less necessary when the data that scrolls
off the screen is not gone forever.

--Rik

>
> I also wonder if there is a lot which can be lifted/translated from the
> Jupyter Qt console:
>
> https://github.com/jupyter/qtconsole
>
> It has a history and a pager.
>
> Regards,
>
> Richard
>
>
>
>
>
>
>
> The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

Torsten Lilge
In reply to this post by siko1056
On Tue, 2019-06-04 at 14:10 +0200, Kai Torben Ohlhus wrote:

> On 6/4/19 11:35 AM, CROZIER Richard wrote:
> > On 03/06/2019 17:34, Rik wrote:
> > > I didn't see any comments at
> > > https://wiki.octave.org/GUI_terminal_widget
> > > so I thought I would bring the conversation back to the mailing
> > > list.
> > >
> > >  From a strategic level, this is a big change.  What do we get
> > > from
> > > doing this (pros) and what do we lose (cons)?  Are there
> > > sufficient
> > > benefits to undertake this task?
> > >
> > > Although the terminal often causes problems, it also allows for a
> > > lot of
> > > code re-use.  We use Readline for history and input
> > > processing.  We get
> > > 'less' and other utilities for free.  In replacing this, do we
> > > essentially need to re-invent the wheel? That's a *lot* of coding
> > > effort
> > > for something that isn't even particularly Octave related.  Since
> > > your
> > > time is finite, if you are working on this then there will be
> > > other
> > > Octave bugs that aren't addressed.  Is the balance
> > > worthwhile?  You
> > > understand core Octave better than anyone so it may be that your
> > > time is
> > > better spent resolving 50 bugs there, rather than writing more
> > > generic
> > > terminal emulation code that other programmers could do.
> > >
> > > Just some ideas on what to answer before going forward.
> > >
> > > --Rik
> > >
> >
> > The terminal code in Octave is a mess, I have gone in to try and fix
> > bugs, but it is so complex (for example there is, or at least was,
> > completely different terminal code for Windows and Linux), that
> > fixing
> > simple bugs had a huge overhead in understanding the terminal code.
> >
> > Given that no-one has done anything to improve this for a long time
> > perhaps JWE's help is needed to get the terminal to a state where
> > others
> > can help pick things up, especially if he has done some of the work
> > already.
> >
> > I also get the impression he's enjoying getting this working.
> > Enjoyment
> > is important for a project maintainer!
> >
> > Incidentally, do you really need a pager in the gui? You have
> > scrollbars
> > on the windows. All you really need is a keyboard shortcut to scroll
> > up
> > don't you? Like SHIFT+PageUp
> >
> > I also wonder if there is a lot which can be lifted/translated from
> > the
> > Jupyter Qt console:
> >
> > https://github.com/jupyter/qtconsole
> >
> > It has a history and a pager.
> >
> > Regards,
> >
> > Richard
> >
>
> Looks cool and is something I regularly dream of in Octave, but
> qtconsole is written in Python.  Unless Octave moves in this
> direction,
> I think this is difficult to realize.  Another promising C++ candidate
> is qtermwidet. Both listed here:
>
>    https://wiki.octave.org/GUI_terminal_widget#Alternatives
>
> Additionally, I raised a question about Unicode support of this new
> solution, which occasionally results in bug reports.
>
>
> https://wiki.octave.org/GUI_terminal_widget#International_Characters_Support
>
> Best regards,
>
> Kai

As mentioned in the wiki , qtermwidget is not available for windows. I
think that's why it is not an option since, we still would have to
maintain two different terminals, which is really hard when trying to
implement new features or fixing bugs.

Torsten






Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

Mike Miller-4
In reply to this post by Rik-4
On Mon, Jun 03, 2019 at 09:34:12 -0700, Rik wrote:
> From a strategic level, this is a big change.  What do we get from doing
> this (pros) and what do we lose (cons)?  Are there sufficient benefits to
> undertake this task?

It might also help to define what are the actual goals of the command
window replacement widget. What do we want the command window to do and
how do we want it to behave?

> Although the terminal often causes problems, it also allows for a lot of
> code re-use.  We use Readline for history and input processing.  We get
> 'less' and other utilities for free.  In replacing this, do we essentially
> need to re-invent the wheel?  That's a *lot* of coding effort for something
> that isn't even particularly Octave related.  Since your time is finite, if
> you are working on this then there will be other Octave bugs that aren't
> addressed.  Is the balance worthwhile?  You understand core Octave better
> than anyone so it may be that your time is better spent resolving 50 bugs
> there, rather than writing more generic terminal emulation code that other
> programmers could do.
I was under the impression that a replacement would be a more flexible
text rendering input area, for example being able to render inline
styled text with HTML or other markup, being able to move the cursor and
select text with the mouse. But I was also under the impression that it
would not be a full terminal emulator. That's the kind of command window
replacement I would find more useful and I think many users want.

If we do still want a full terminal emulator, then maybe we should push
for reusing something like qtermwidget, and helping to make it work on
all the systems we want to target?

--
mike

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

John W. Eaton
Administrator
In reply to this post by Torsten Lilge
On 6/4/19 2:39 PM, Torsten Lilge wrote:

> As mentioned in the wiki , qtermwidget is not available for windows. I
> think that's why it is not an option since, we still would have to
> maintain two different terminals, which is really hard when trying to
> implement new features or fixing bugs.

My memory may be a little fuzzy, but as I recall, the qterminal code
that we forked only supported running a separate process in the terminal
and we wanted to run Octave in a separate thread inside the terminal,
not as a separate process.  Running Octave as a separate process brings
up a whole other set of issues (parsing output and creating a way to
send and receive info to the process apart from the displayed
input/output stream).

We could attempt to go back to the qtermwidget code and see if we could
modify it to do what we need, and this time, get our changes adopted
upstream.  The changes would be

   * Allow direct connection to a pty instead of using a subprocess.

   * Make it work on Windows so we don't have two completely separate
widgets.  But we would still have almost completely separate
implementations internally because the Unix code uses ptys and the
Windows code uses a Windows Console object.

jwe



Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

John W. Eaton
Administrator
In reply to this post by Rik-4
On 6/3/19 12:34 PM, Rik wrote:

> I didn't see any comments at https://wiki.octave.org/GUI_terminal_widget 
> so I thought I would bring the conversation back to the mailing list.
>
>  From a strategic level, this is a big change.  What do we get from
> doing this (pros) and what do we lose (cons)?  Are there sufficient
> benefits to undertake this task?

Yes, it would be a fairly big change.  That's why I wanted to create a
small example that would be easy to experiment with.  I'd also make any
use of the new widget optional in Octave, at least until we decide
whether the new approach is really better.

> Although the terminal often causes problems, it also allows for a lot of
> code re-use.  We use Readline for history and input processing.

We can still use readline.  My toy program uses it.  But we need a lot
less of readline.  None of the messy code that deals with the actual
terminal is needed.  All you really need is the code for handling
editing and history.

> We get 'less' and other utilities for free.  In replacing this, do we
> essentially need to re-invent the wheel? That's a *lot* of coding effort
> for something that isn't even particularly Octave related.  Since your
> time is finite, if you are working on this then there will be other
> Octave bugs that aren't addressed.  Is the balance worthwhile?  You
> understand core Octave better than anyone so it may be that your time is
> better spent resolving 50 bugs there, rather than writing more generic
> terminal emulation code that other programmers could do.

Yeah, I'm not sure about any of this either.  It's definitely worth the
discussion before going to far.

jwe


Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

John W. Eaton
Administrator
In reply to this post by Rik-4
On 6/4/19 12:06 PM, Rik wrote:
> On 06/04/2019 02:35 AM, CROZIER Richard wrote:

>> Incidentally, do you really need a pager in the gui? You have scrollbars
>> on the windows. All you really need is a keyboard shortcut to scroll up
>> don't you? Like SHIFT+PageUp
> We would need it if we wanted to maintain compatibility with Matlab.  But
> you're right that pagers are much less necessary when the data that scrolls
> off the screen is not gone forever.

What does Matlab do for the pager inside the GUI?  Does it run a
separate process or provide a simple built-in pager in a GUI window?

I recall asking previously whether Matlab's terminal window provides a
full terminal emulation when running on Windows or Linux systems, just
to know what Matlab users might expect to be able to do and I think the
answer was that it did not provide full emulation.  Maybe someone could
verify that with a current version?  What happens if you try to execute
something like

   system ('vi')

or

   system ('emacs -nw')

at the Matlab prompt?

Octave's current terminal window on Unix can run some programs like less
or more, but not all.  Emacs and vi don't work for me, for example.

Also, to properly run less on Unixy systems, we have to use that
annoying wrapper program that forks, calls setsid to give up the
controlling terminal (so less can access /dev/tty to get keyboard input,
I think) and then execs the real Octave GUI program.  That caused all
sorts of trouble and wasted effort in the past.  And it still makes
debugging the GUI startup code much more difficult than it should be.
It might have been simpler and saved a lot of time if, instead of
writing that wrapper program to get less working properly, I had just
known how to write a simpler unified terminal widget.

jwe


Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

John W. Eaton
Administrator
In reply to this post by Mike Miller-4
On 6/4/19 3:01 PM, Mike Miller wrote:

> It might also help to define what are the actual goals of the command
> window replacement widget. What do we want the command window to do and
> how do we want it to behave?

Yes, I'd like input from others about that.

> I was under the impression that a replacement would be a more flexible
> text rendering input area, for example being able to render inline
> styled text with HTML or other markup, being able to move the cursor and
> select text with the mouse. But I was also under the impression that it
> would not be a full terminal emulator. That's the kind of command window
> replacement I would find more useful and I think many users want.

Yes, that's what I see it being as well.

jwe

Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

nrjank
In reply to this post by John W. Eaton
>
>    system ('vi')
>
> or
>
>    system ('emacs -nw')
>
> at the Matlab prompt?
>

what would a windows equivalent be?   cmd and powershell?   they
start, but not sure what functionality you're testing:

in matlab 2019a:

>> system ('vi')
'vi' is not recognized as an internal or external command,
operable program or batch file.
ans =
     1
>> system ('emacs -nw')
'emacs' is not recognized as an internal or external command,
operable program or batch file.
ans =
     1
>> system ('cmd')
Microsoft Windows [Version 10.0.16299.1146]
(c) 2017 Microsoft Corporation. All rights reserved.

C:\Users\nicholas.jankowski\Documents\MATLAB>exit
exit
ans =
        9009
>> system ('powershell')
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Attempting to perform the InitializeDefaultDrives operation on the
'FileSystem' provider failed.
PS C:\Users\nicholas.jankowski\Documents\MATLAB> exit
exit
ans =
     0

and in Octave 5.1.0

>> system ('cmd')
Microsoft Windows [Version 10.0.16299.1146]
(c) 2017 Microsoft Corporation. All rights reserved.
C:\Users\nicholas.jankowski\Desktop>exit
ans = 0
>> system ('powershell')
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
Attempting to perform the InitializeDefaultDrives operation on the
'FileSystem' provider failed.
PS C:\Users\nicholas.jankowski\Desktop> exit
ans = 0

note that powershell under Octave gave me colored text.

Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

Pantxo
In reply to this post by John W. Eaton
John W. Eaton wrote

> On 6/4/19 12:06 PM, Rik wrote:
>> On 06/04/2019 02:35 AM, CROZIER Richard wrote:
>
>>> Incidentally, do you really need a pager in the gui? You have scrollbars
>>> on the windows. All you really need is a keyboard shortcut to scroll up
>>> don't you? Like SHIFT+PageUp
>> We would need it if we wanted to maintain compatibility with Matlab.  But
>> you're right that pagers are much less necessary when the data that
>> scrolls
>> off the screen is not gone forever.
>
> What does Matlab do for the pager inside the GUI?  Does it run a
> separate process or provide a simple built-in pager in a GUI window?
>
> I recall asking previously whether Matlab's terminal window provides a
> full terminal emulation when running on Windows or Linux systems, just
> to know what Matlab users might expect to be able to do and I think the
> answer was that it did not provide full emulation.  Maybe someone could
> verify that with a current version?  What happens if you try to execute
> something like
>
>    system ('vi')
>
> or
>
>    system ('emacs -nw')
>
> at the Matlab prompt?
>
>
> jwe

On a linux machine, with Matlab 2016a:

>> system ('emacs -nw')
emacs: Terminal type "dumb" is not powerful enough to run Emacs.
It lacks the ability to position the cursor.
If that is not the actual type of terminal you have,
use the Bourne shell command `TERM=... export TERM' (C-shell:
`setenv TERM ...') to specify the correct type.  It may be necessary
to do `unset TERMINFO' (C-shell: `unsetenv TERMINFO') as well.
ans =

     1
>> system ('export TERM=xterm; emacs -nw')
[?1049h[?1049h[?12;25h[?1h=[?25l-=--:----F1
*scratch*      All L1    
(Fundamental)------------
....

The terminal then becomes unusable (e.g. can't "C-x C-c") then I have kill
Matlab.

As for the pager, I don't know how to determine if it is a separate process
since my ML session is run on a separate node from the one I launch it from
(i.e. "ps -u $USER" in the launching terminal won't even list Matlab). All I
can tell is that the pager is pretty basic: "space" scrolls one page
forward, any other key scrolls one line forward, "q" quits the pager. Looks
like its no more than "more", much less capable than "less".
   
Pantxo



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

John W. Eaton
Administrator
On 6/5/19 10:19 AM, Pantxo wrote:

> On a linux machine, with Matlab 2016a:
>
>>> system ('emacs -nw')
> emacs: Terminal type "dumb" is not powerful enough to run Emacs.
> It lacks the ability to position the cursor.
> If that is not the actual type of terminal you have,
> use the Bourne shell command `TERM=... export TERM' (C-shell:
> `setenv TERM ...') to specify the correct type.  It may be necessary
> to do `unset TERMINFO' (C-shell: `unsetenv TERMINFO') as well.
> ans =
>
>       1
>>> system ('export TERM=xterm; emacs -nw')
> [?1049h[?1049h[?12;25h[?1h=[?25l-=--:----F1
> *scratch*      All L1
> (Fundamental)------------
> ....
>
> The terminal then becomes unusable (e.g. can't "C-x C-c") then I have kill
> Matlab.
>
> As for the pager, I don't know how to determine if it is a separate process
> since my ML session is run on a separate node from the one I launch it from
> (i.e. "ps -u $USER" in the launching terminal won't even list Matlab). All I
> can tell is that the pager is pretty basic: "space" scrolls one page
> forward, any other key scrolls one line forward, "q" quits the pager. Looks
> like its no more than "more", much less capable than "less".

OK, thanks.

We might be able to come up with some tests but I don't think it really
matters.  The point is that Matlab users who begin using Octave won't
expect to be able to run something like Emacs in the command window.

Having the capabilities of less would be nice, but I don't see it as
necessary.  We could also just provide good ways to search in the
command window without having output paged and that would probably
satisfy most users.  We've already disabled the pager by default anyway
because having it enabled seems to confuse new users.  So I don't think
most GUI users will miss it much even if we provide nothing but scrollbars.

jwe


Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

Ian McCallion
On Wed, 5 Jun 2019 at 16:49, John W. Eaton <[hidden email]> wrote:

> Having the capabilities of less would be nice, but I don't see it as
> necessary.  We could also just provide good ways to search in the
> command window without having output paged and that would probably
> satisfy most users.  We've already disabled the pager by default anyway
> because having it enabled seems to confuse new users.  So I don't think
> most GUI users will miss it much even if we provide nothing but scrollbars.

I would personally downvote a pager which seems an archaic way of
getting around text and deeply rooted in UNIX history. So scrollbars,
yes, search, maybe, but working pageup and pagedown keys, yes, and
ability to mark and copy more than a window's worth of text by
automatic scrolling, yes.

(Apologies if this outsider view is out of place on octave-maintainers)

Cheers... Ian

Reply | Threaded
Open this post in threaded view
|

Re: GUI terminal widget

PhilipNienhuis
In reply to this post by John W. Eaton
John W. Eaton wrote

> On 6/5/19 10:19 AM, Pantxo wrote:
>
>> On a linux machine, with Matlab 2016a:
>>
>>>> system ('emacs -nw')
>> emacs: Terminal type "dumb" is not powerful enough to run Emacs.
>> It lacks the ability to position the cursor.
>> If that is not the actual type of terminal you have,
>> use the Bourne shell command `TERM=... export TERM' (C-shell:
>> `setenv TERM ...') to specify the correct type.  It may be necessary
>> to do `unset TERMINFO' (C-shell: `unsetenv TERMINFO') as well.
>> ans =
>>
>>       1
>>>> system ('export TERM=xterm; emacs -nw')
>> [?1049h[?1049h[?12;25h[?1h=[?25l-=--:----F1
>> *scratch*      All L1
>> (Fundamental)------------
>> ....
>>
>> The terminal then becomes unusable (e.g. can't "C-x C-c") then I have
>> kill
>> Matlab.
>>
>> As for the pager, I don't know how to determine if it is a separate
>> process
>> since my ML session is run on a separate node from the one I launch it
>> from
>> (i.e. "ps -u $USER" in the launching terminal won't even list Matlab).
>> All I
>> can tell is that the pager is pretty basic: "space" scrolls one page
>> forward, any other key scrolls one line forward, "q" quits the pager.
>> Looks
>> like its no more than "more", much less capable than "less".
>
> OK, thanks.
>
> We might be able to come up with some tests but I don't think it really
> matters.  The point is that Matlab users who begin using Octave won't
> expect to be able to run something like Emacs in the command window.
>
> Having the capabilities of less would be nice, but I don't see it as
> necessary.  We could also just provide good ways to search in the
> command window without having output paged and that would probably
> satisfy most users.  We've already disabled the pager by default anyway
> because having it enabled seems to confuse new users.  So I don't think
> most GUI users will miss it much even if we provide nothing but
> scrollbars.

GUI or CLI, I find the less pager one of the attractive features of Octave.
I'm always annoyed by the primitive behavior of Matlab's command window.

Seeing output stopped automatically after a pageful is way less troublesome
than first seeing a sometimes very long listing flashing by and then having
to scroll back in search of the desired output lines.
AFAIK the vertical scrollbar refers to the complete output of a session, not
just output from one command. In the terminal buffer is large and filled up,
specific lines of output can only be brought into view by very careful
grabbing and manipulation of a possibly very thin scrollbar handle, where
just a few pixels of movement of it can translate to many pages of output
scrolling by.

IMO a GUI terminal should have:
- pager
- colors (possibly ANSI)
- maybe ANSI cursor positioning
- copy /paste capabilities
- scrolling using the mouse wheel
- scroll bars (yes they still can be useful)
- automatic line wrapping at the right border
and that's about it. But OK, maybe "it" is asking too much.

Just my opinion...

Philip




--
Sent from: http://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html