Encoding in Octave 5.0

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Encoding in Octave 5.0

Chris Zheng
Dear maintainers,

I use Octave within Emacs under Windows 8.1 using Inferior Octave
mode. With version 4.4.1, the following code

title('é');
print n.svg

results in garbage content in the Figure window, but the right content
in the SVG file. However with version 5.0.90 downloaded from
ftp://alpha.gnu.org/gnu/octave/, the above code results in right
content on the screen but wrong content in the SVG file (the wrong
content is encoded in GBK which is my locale encoding). The code sends
to the Octave process is indeed encoded in UTF-8. I wonder what is
changed between 4.4.1 and 5.0, and what can I do to fix the problem.

Thank you for your work,

Chris

Reply | Threaded
Open this post in threaded view
|

Re: Encoding in Octave 5.0

apjanke-floss


On 1/29/19 12:02 PM, Chris Zheng wrote:

> Dear maintainers,
>
> I use Octave within Emacs under Windows 8.1 using Inferior Octave
> mode. With version 4.4.1, the following code
>
> title('é');
> print n.svg
>
> results in garbage content in the Figure window, but the right content
> in the SVG file. However with version 5.0.90 downloaded from
> ftp://alpha.gnu.org/gnu/octave/, the above code results in right
> content on the screen but wrong content in the SVG file (the wrong
> content is encoded in GBK which is my locale encoding). The code sends
> to the Octave process is indeed encoded in UTF-8. I wonder what is
> changed between 4.4.1 and 5.0, and what can I do to fix the problem.
>
> Thank you for your work,
>
> Chris

Can you tell us exactly how you got this code into Octave?

title('é');
print n.svg

Is this in a *.m file you are editing in Emacs? Or is it something you
pasted into the inferior Octave buffer? If it's a *.m file, did you
copy-paste that é in, or use a Unicode-entry keyboard sequence? The
input mechanics might matter here.

If it's in a *.m file, could you attach a copy of that .m file for us to
examine?

And how did you install your Emacs? I'll try to reproduce if you can
tell me what setup you have.

Cheers,
Andrew

Reply | Threaded
Open this post in threaded view
|

Aw: Encoding in Octave 5.0

mmuetzel
In reply to this post by Chris Zheng
Am 29. Januar 2019 um 18:02 Uhr schrieb "Chris Zheng":

> Dear maintainers,
>
> I use Octave within Emacs under Windows 8.1 using Inferior Octave
> mode. With version 4.4.1, the following code
>
> title('é');
> print n.svg
>
> results in garbage content in the Figure window, but the right content
> in the SVG file. However with version 5.0.90 downloaded from
> ftp://alpha.gnu.org/gnu/octave/, the above code results in right
> content on the screen but wrong content in the SVG file (the wrong
> content is encoded in GBK which is my locale encoding). The code sends
> to the Octave process is indeed encoded in UTF-8. I wonder what is
> changed between 4.4.1 and 5.0, and what can I do to fix the problem.

I can confirm what you describe with the 5.0.90 build on Windows 10: The strings in the svg should be UTF-8 encoded. But they seem to be encoded in Latin-1 (which is my locale encoding).

Can you please open a bug report on savannah with your example?
https://savannah.gnu.org/bugs/?group=octave

Thanks.

Markus

Reply | Threaded
Open this post in threaded view
|

Re: Encoding in Octave 5.0

mmuetzel
Am 30. Januar 2019 um 08:30 Uhr schrieb "Markus Mützel":

> Am 29. Januar 2019 um 18:02 Uhr schrieb "Chris Zheng":
> > Dear maintainers,
> >
> > I use Octave within Emacs under Windows 8.1 using Inferior Octave
> > mode. With version 4.4.1, the following code
> >
> > title('é');
> > print n.svg
> >
> > results in garbage content in the Figure window, but the right content
> > in the SVG file. However with version 5.0.90 downloaded from
> > ftp://alpha.gnu.org/gnu/octave/, the above code results in right
> > content on the screen but wrong content in the SVG file (the wrong
> > content is encoded in GBK which is my locale encoding). The code sends
> > to the Octave process is indeed encoded in UTF-8. I wonder what is
> > changed between 4.4.1 and 5.0, and what can I do to fix the problem.
>
> I can confirm what you describe with the 5.0.90 build on Windows 10: The strings in the svg should be UTF-8 encoded. But they seem to be encoded in Latin-1 (which is my locale encoding).
>
> Can you please open a bug report on savannah with your example?
> https://savannah.gnu.org/bugs/?group=octave
>

Sorry. Please wait before opening a bug report.
I thought that .svg file creation no longer went through .ps (which has problems with non-ASCII characters). But it looks like it still does.
@Pantxo: I think you know the printing system better. Do you know which options would make this work for svg? Thanks.

Markus

Reply | Threaded
Open this post in threaded view
|

Re: Encoding in Octave 5.0

Pantxo
mmuetzel wrote

> Am 30. Januar 2019 um 08:30 Uhr schrieb "Markus Mützel":
>> Am 29. Januar 2019 um 18:02 Uhr schrieb "Chris Zheng":
>> > Dear maintainers,
>> >
>> > I use Octave within Emacs under Windows 8.1 using Inferior Octave
>> > mode. With version 4.4.1, the following code
>> >
>> > title('é');
>> > print n.svg
>> >
>> > results in garbage content in the Figure window, but the right content
>> > in the SVG file. However with version 5.0.90 downloaded from
>> > ftp://alpha.gnu.org/gnu/octave/, the above code results in right
>> > content on the screen but wrong content in the SVG file (the wrong
>> > content is encoded in GBK which is my locale encoding). The code sends
>> > to the Octave process is indeed encoded in UTF-8. I wonder what is
>> > changed between 4.4.1 and 5.0, and what can I do to fix the problem.
>>
>> I can confirm what you describe with the 5.0.90 build on Windows 10: The
>> strings in the svg should be UTF-8 encoded. But they seem to be encoded
>> in Latin-1 (which is my locale encoding).
>>
>> Can you please open a bug report on savannah with your example?
>> https://savannah.gnu.org/bugs/?group=octave
>>
>
> Sorry. Please wait before opening a bug report.
> I thought that .svg file creation no longer went through .ps (which has
> problems with non-ASCII characters). But it looks like it still does.
> @Pantxo: I think you know the printing system better. Do you know which
> options would make this work for svg? Thanks.
>
> Markus

The only difference with how SVG was handled before, is that we try to
reconstruct polygons (if any) through the use of a new utility program
(octave-svgconvert). Since the original SVG file (straight out of gl2ps) is
read and rewritten, it is probable that the final write operation is
performed (by Qt) using the users locale encoding. I'll take a look and see
if I can find something.

Pantxo  



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

Re: Encoding in Octave 5.0

Chris Zheng
In reply to this post by apjanke-floss
Hello,

> Can you tell us exactly how you got this code into Octave?
>
> title('é');
> print n.svg
>
> Is this in a *.m file you are editing in Emacs? Or is it something you
> pasted into the inferior Octave buffer? If it's a *.m file, did you
> copy-paste that é in, or use a Unicode-entry keyboard sequence? The
> input mechanics might matter here.

The above code was run in the inferior Octave buffer. I did try the
code in a .m file which is in UTF-8. With versions 5.0.90 and 4.4.1,
the result is garbage on the screen and correct in the SVG file. The
difference is that with version 5.0.90, the garbage is 0x8305. While
with 4.4.1, the garbage is “é” (in hex is 0xC3 0xA9). The “é” is
copied form the web page and Emacs tells the codepoint is 0xE9.

> If it's in a *.m file, could you attach a copy of that .m file for us to
> examine?
>
> And how did you install your Emacs? I'll try to reproduce if you can
> tell me what setup you have.

The Emacs is built from the current master branch.

> Cheers,
> Andrew

Thank you,

Chris

Reply | Threaded
Open this post in threaded view
|

Re: Encoding in Octave 5.0

Pantxo
In reply to this post by mmuetzel
Le 30/01/2019 à 08:37, "Markus Mützel" a écrit :

> Am 30. Januar 2019 um 08:30 Uhr schrieb "Markus Mützel":
>> Am 29. Januar 2019 um 18:02 Uhr schrieb "Chris Zheng":
>>> Dear maintainers,
>>>
>>> I use Octave within Emacs under Windows 8.1 using Inferior Octave
>>> mode. With version 4.4.1, the following code
>>>
>>> title('é');
>>> print n.svg
>>>
>>> results in garbage content in the Figure window, but the right content
>>> in the SVG file. However with version 5.0.90 downloaded from
>>> ftp://alpha.gnu.org/gnu/octave/, the above code results in right
>>> content on the screen but wrong content in the SVG file (the wrong
>>> content is encoded in GBK which is my locale encoding). The code sends
>>> to the Octave process is indeed encoded in UTF-8. I wonder what is
>>> changed between 4.4.1 and 5.0, and what can I do to fix the problem.
>> I can confirm what you describe with the 5.0.90 build on Windows 10: The strings in the svg should be UTF-8 encoded. But they seem to be encoded in Latin-1 (which is my locale encoding).
>>
>> Can you please open a bug report on savannah with your example?
>> https://savannah.gnu.org/bugs/?group=octave
>>
> Sorry. Please wait before opening a bug report.
> I thought that .svg file creation no longer went through .ps (which has problems with non-ASCII characters). But it looks like it still does.
> @Pantxo: I think you know the printing system better. Do you know which options would make this work for svg? Thanks.
>
> Markus

This is definitely a regression. Could you file a bug report so that we
can discuss this issue there?


Pantxo


Reply | Threaded
Open this post in threaded view
|

Re: Encoding in Octave 5.0

mmuetzel
Am 30. Januar 2019 um 23:19 Uhr schrieb "Pantxo Diribarne":

> Le 30/01/2019 à 08:37, "Markus Mützel" a écrit :
> > Am 30. Januar 2019 um 08:30 Uhr schrieb "Markus Mützel":
> >> Am 29. Januar 2019 um 18:02 Uhr schrieb "Chris Zheng":
> >>> Dear maintainers,
> >>>
> >>> I use Octave within Emacs under Windows 8.1 using Inferior Octave
> >>> mode. With version 4.4.1, the following code
> >>>
> >>> title('é');
> >>> print n.svg
> >>>
> >>> results in garbage content in the Figure window, but the right content
> >>> in the SVG file. However with version 5.0.90 downloaded from
> >>> ftp://alpha.gnu.org/gnu/octave/, the above code results in right
> >>> content on the screen but wrong content in the SVG file (the wrong
> >>> content is encoded in GBK which is my locale encoding). The code sends
> >>> to the Octave process is indeed encoded in UTF-8. I wonder what is
> >>> changed between 4.4.1 and 5.0, and what can I do to fix the problem.
> >> I can confirm what you describe with the 5.0.90 build on Windows 10: The strings in the svg should be UTF-8 encoded. But they seem to be encoded in Latin-1 (which is my locale encoding).
> >>
> >> Can you please open a bug report on savannah with your example?
> >> https://savannah.gnu.org/bugs/?group=octave
> >>
> > Sorry. Please wait before opening a bug report.
> > I thought that .svg file creation no longer went through .ps (which has problems with non-ASCII characters). But it looks like it still does.
> > @Pantxo: I think you know the printing system better. Do you know which options would make this work for svg? Thanks.
> >
> > Markus
>
> This is definitely a regression. Could you file a bug report so that we
> can discuss this issue there?

I opened a bug #55612 here:
https://savannah.gnu.org/bugs/index.php?55612

Markus

Reply | Threaded
Open this post in threaded view
|

Re: Encoding in Octave 5.0

Chris Zheng
On Thu, Jan 31, 2019 at 08:33:05AM +0100, "Markus Mützel" wrote:
>
>I opened a bug #55612 here:
>https://savannah.gnu.org/bugs/index.php?55612
>
>Markus

Thank you, Markus and Pantxo. I see it is fixed on the bug report.