JSON Test suite status

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

JSON Test suite status

Abdallah Elshamy
Greetings,

The testing suite [1] for jsonencode [2] and jsondecode [3] is almost complete (Any suggestions for something to add are surely very welcomed) except for checking if errors is raised correctly This will wait after coding the functions (during writing the BIST) so that the error messages is determined.

Also in MATLAB, they don't support encoding for complex numbers. Is it better to follow this approach or should we support encoding for it? Maybe encoding it to a JSON object with two keys: "real" and "imaginary".

Lastly in decoding json objects to structs MATLAB uses makeValidName [4] in converting the keys of the objects to a valid field in the struct (as it doesn't allow the fields of a struct to start with a space or a number ... etc.) As this is valid in Octave, I don't think that this will be necessary. What do you think?

Best,
Abdallah

Reply | Threaded
Open this post in threaded view
|

Re: [GSoC] JSON Test suite status

siko1056
On 6/22/20 5:19 AM, Abdallah Elshamy wrote:
> [...]
> Also in MATLAB, they don't support encoding for complex numbers. Is it
> better to follow this approach or should we support encoding for it?
> Maybe encoding it to a JSON object with two keys: "real" and "imaginary".
> [...]

I cherish your enthusiasm.  For the start it is more than sufficient to
support what Matlab supports and to do this job perfectly.  If users
demand the complex encoding feature that much, it can be added at any
time later.

> Lastly in decoding json objects to structs MATLAB uses makeValidName [4]
> in converting the keys of the objects to a valid field in the struct (as
> it doesn't allow the fields of a struct to start with a space or a
> number ... etc.) As this is valid in Octave, I don't think that this
> will be necessary. What do you think?

What do you mean by "this is valid in Octave"?  Such calls are neither
allowed in Octave too

   my_struct.2bad_fieldname = 4

Btw. Octave implements that function too [2].

Kai


[1] https://www.mathworks.com/help/matlab/ref/matlab.lang.makevalidname.html
[2]
https://www.octave.org/doc/v5.2.0/XREFmatlab_005flang_005fmakeValidName.html

Reply | Threaded
Open this post in threaded view
|

Re: [GSoC] JSON Test suite status

Abdallah Elshamy
On Mon, Jun 22, 2020 at 10:53 AM Kai Torben Ohlhus <[hidden email]> wrote:
On 6/22/20 5:19 AM, Abdallah Elshamy wrote:
> [...]
> Also in MATLAB, they don't support encoding for complex numbers. Is it
> better to follow this approach or should we support encoding for it?
> Maybe encoding it to a JSON object with two keys: "real" and "imaginary".
> [...]

I cherish your enthusiasm.  For the start it is more than sufficient to
support what Matlab supports and to do this job perfectly.  If users
demand the complex encoding feature that much, it can be added at any
time later.

> Lastly in decoding json objects to structs MATLAB uses makeValidName [4]
> in converting the keys of the objects to a valid field in the struct (as
> it doesn't allow the fields of a struct to start with a space or a
> number ... etc.) As this is valid in Octave, I don't think that this
> will be necessary. What do you think?

What do you mean by "this is valid in Octave"?  Such calls are neither
allowed in Octave too

   my_struct.2bad_fieldname = 4

What I meant is that if you tried to create the following struct

a = struct("123foo", 2, "foo bar", 5)

in MATLAB (the online version), you will get an error that the field name is invalid. But in Octave, This struct will be created successfully and it can be accessed using:

a.("123foo")    or     a.("foo bar")

I think this is not a common way to access structs so it may be better to use makeValidName despite it is not necessary to call it to create structs in Octave unlike MATLAB. What do you think?
 
Btw. Octave implements that function too [2].
 
I have seen it in Octave. It has some differences (for example "_id" is not a valid name in MATLAB and becomes "x_id" but it is valid in Octave) that will require some modifications to the tests when writing BIST.  

Best,
Abdallah

Reply | Threaded
Open this post in threaded view
|

Re: [GSoC] JSON Test suite status

Andreas Weber-6
Am 22.06.20 um 11:30 schrieb Abdallah Elshamy:
> a.("123foo")    or     a.("foo bar")

you also create it this way. Kais example:
octave:3> my_struct.("2bad_fieldname") = 4
my_struct =
  scalar structure containing the fields:
    2bad_fieldname = 4

> I think this is not a common way to access structs so it may be better
> to use makeValidName despite it is not necessary to call it to create
> structs in Octave unlike MATLAB. What do you think?

I think the goal for most users will be, that the code written for the
other software also works on GNU Octave. So I would suggest to use the
same "renaming" conventions.

-- Andy

Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

Andreas Weber-6
In reply to this post by Abdallah Elshamy
Am 21.06.20 um 22:19 schrieb Abdallah Elshamy:
> The testing suite [1] for jsonencode [2] and jsondecode [3] is almost
> complete (Any suggestions for something to add are surely very welcomed)

I had a look at jsonencodetest.m and wonder I you are really trying to
create the exact same JSON output (byte identical for example same
whitespace and so on) as the other software does or "identical parsed
JSON" where whitespace doesn't matter.

-- Andy

Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

Abdallah Elshamy
On Mon, Jun 22, 2020 at 2:24 PM Andreas Weber <[hidden email]> wrote:
Am 21.06.20 um 22:19 schrieb Abdallah Elshamy:
> The testing suite [1] for jsonencode [2] and jsondecode [3] is almost
> complete (Any suggestions for something to add are surely very welcomed)

I had a look at jsonencodetest.m and wonder I you are really trying to
create the exact same JSON output (byte identical for example same
whitespace and so on) as the other software does or "identical parsed
JSON" where whitespace doesn't matter.

Thanks for providing feedback. This script is intended to run on MATLAB to test compatibility so producing the same output in it is a core goal of it.

Abdallah
Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

Andreas Weber-6
Am 22.06.20 um 19:47 schrieb Abdallah Elshamy:

> On Mon, Jun 22, 2020 at 2:24 PM Andreas Weber <[hidden email]
> <mailto:[hidden email]>> wrote:
>     I had a look at jsonencodetest.m and wonder I you are really trying to
>     create the exact same JSON output (byte identical for example same
>     whitespace and so on) as the other software does or "identical parsed
>     JSON" where whitespace doesn't matter.
>
> Thanks for providing feedback. This script is intended to run on MATLAB
> to test compatibility so producing the same output in it is a core goal
> of it.

Just to be clear: You want to create byte-identical JSON output which
possibly means patching/modifying rapidjson?

-- Andy

Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

Andrew Janke-2


On 6/23/20 1:25 AM, Andreas Weber wrote:

> Am 22.06.20 um 19:47 schrieb Abdallah Elshamy:
>> On Mon, Jun 22, 2020 at 2:24 PM Andreas Weber <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>     I had a look at jsonencodetest.m and wonder I you are really trying to
>>     create the exact same JSON output (byte identical for example same
>>     whitespace and so on) as the other software does or "identical parsed
>>     JSON" where whitespace doesn't matter.
>>
>> Thanks for providing feedback. This script is intended to run on MATLAB
>> to test compatibility so producing the same output in it is a core goal
>> of it.
>
> Just to be clear: You want to create byte-identical JSON output which
> possibly means patching/modifying rapidjson?
>
> -- Andy
>

"Patching/modifying rapidjson" sounds a little extreme here: the compact
"minified" form that Matlab's jsonencode produces is probably achievable
without such extreme measures. It's just all whitespace elided. (Whether
this can be practically achieved without violating the Matlab license is
another question.)

Given that the JSON spec is kind of loose and there are a lot of
extensions to it, but the base spec is pretty simple, IMHO
byte-identical JSON output is a worthy and maybe achievable goal here.
(Identical decoding behavior on arbitrary inputs is a whole nother ball
game, and I wouldn't suggest pursuing that.)

Cheers,
Andrew

Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

Andreas Weber-6
Am 23.06.20 um 07:43 schrieb Andrew Janke:

> On 6/23/20 1:25 AM, Andreas Weber wrote:
>> Am 22.06.20 um 19:47 schrieb Abdallah Elshamy:
>>> Thanks for providing feedback. This script is intended to run on MATLAB
>>> to test compatibility so producing the same output in it is a core goal
>>> of it.
>>
>> Just to be clear: You want to create byte-identical JSON output which
>> possibly means patching/modifying rapidjson?
>>
> "Patching/modifying rapidjson" sounds a little extreme here: the compact
> "minified" form that Matlab's jsonencode produces is probably achievable
> without such extreme measures. It's just all whitespace elided. (Whether
> this can be practically achieved without violating the Matlab license is
> another question.)

Yes, you are right. I should have written: "modifying rapidjsons output
to byte-match MATLABs JSON output".

Btw, does the other software product has a "pretty" output like rapidjson?

-- Andy

Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

siko1056
In reply to this post by Andrew Janke-2
On 6/23/20 2:43 PM, Andrew Janke wrote:

> On 6/23/20 1:25 AM, Andreas Weber wrote:
>> Am 22.06.20 um 19:47 schrieb Abdallah Elshamy:
>>> On Mon, Jun 22, 2020 at 2:24 PM Andreas Weber <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>     I had a look at jsonencodetest.m and wonder I you are really trying to
>>>     create the exact same JSON output (byte identical for example same
>>>     whitespace and so on) as the other software does or "identical parsed
>>>     JSON" where whitespace doesn't matter.
>>>
>>> Thanks for providing feedback. This script is intended to run on MATLAB
>>> to test compatibility so producing the same output in it is a core goal
>>> of it.
>>
>> Just to be clear: You want to create byte-identical JSON output which
>> possibly means patching/modifying rapidjson?
>>
>> -- Andy
>>
>
> "Patching/modifying rapidjson" sounds a little extreme here: the compact
> "minified" form that Matlab's jsonencode produces is probably achievable
> without such extreme measures. It's just all whitespace elided. (Whether
> this can be practically achieved without violating the Matlab license is
> another question.)
>
> Given that the JSON spec is kind of loose and there are a lot of
> extensions to it, but the base spec is pretty simple, IMHO
> byte-identical JSON output is a worthy and maybe achievable goal here.
> (Identical decoding behavior on arbitrary inputs is a whole nother ball
> game, and I wouldn't suggest pursuing that.)
>
> Cheers,
> Andrew
>

@Andy: thanks for the pointer to whitespace issues.  Did you experience
differences while implementing octave-rapidjson regarding whitespace?  I
think Abdallah would be interested to hear about your experience.

Regarding the JSON test suite of Abdallah, I think it is a first good
step have unit tests for genuine Matlab input and output in first place
(it might also change with the next Matlab release ...).

If in the second GSoC block (own implementation) we notice whitespace
differences, Abdallah can judge how to handle them, e.g. strtrim(), etc.
 In general I think white space is not that much an issue with JSON
itself.  Like Andrew says, patching RapidJSON for byte-identical JSON
seems a high price to pay at the start of the implementation.  I vote
for getting things started first, before thinking about fixing
nonexistent issues yet ;-)

Kai

Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

Andrew Janke-2
In reply to this post by Andreas Weber-6


On 6/23/20 2:02 AM, Andreas Weber wrote:

> Am 23.06.20 um 07:43 schrieb Andrew Janke:
>> On 6/23/20 1:25 AM, Andreas Weber wrote:
>>> Am 22.06.20 um 19:47 schrieb Abdallah Elshamy:
>>>> Thanks for providing feedback. This script is intended to run on MATLAB
>>>> to test compatibility so producing the same output in it is a core goal
>>>> of it.
>>>
>>> Just to be clear: You want to create byte-identical JSON output which
>>> possibly means patching/modifying rapidjson?
>>>
>> "Patching/modifying rapidjson" sounds a little extreme here: the compact
>> "minified" form that Matlab's jsonencode produces is probably achievable
>> without such extreme measures. It's just all whitespace elided. (Whether
>> this can be practically achieved without violating the Matlab license is
>> another question.)
>
> Yes, you are right. I should have written: "modifying rapidjsons output
> to byte-match MATLABs JSON output".
>
> Btw, does the other software product has a "pretty" output like rapidjson?
>
> -- Andy

No. To the other product's users' consternation, it does not. So its
output is not suitable for things like human-editable config files and
the like.

I would certainly encourage this product's developers to include
pretty-printing as an extension, since it's a nice thing to have IMHO.

Cheers,
Andrew

Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

Andrew Janke-2
In reply to this post by siko1056


On 6/23/20 2:03 AM, Kai Torben Ohlhus wrote:

> On 6/23/20 2:43 PM, Andrew Janke wrote:
>> On 6/23/20 1:25 AM, Andreas Weber wrote:
>>> Am 22.06.20 um 19:47 schrieb Abdallah Elshamy:
>>>> On Mon, Jun 22, 2020 at 2:24 PM Andreas Weber <[hidden email]
>>>> <mailto:[hidden email]>> wrote:
>>>>     I had a look at jsonencodetest.m and wonder I you are really trying to
>>>>     create the exact same JSON output (byte identical for example same
>>>>     whitespace and so on) as the other software does or "identical parsed
>>>>     JSON" where whitespace doesn't matter.
>>>>
>>>> Thanks for providing feedback. This script is intended to run on MATLAB
>>>> to test compatibility so producing the same output in it is a core goal
>>>> of it.
>>>
>>> Just to be clear: You want to create byte-identical JSON output which
>>> possibly means patching/modifying rapidjson?
>>>
>>> -- Andy
>>>
>>
>> "Patching/modifying rapidjson" sounds a little extreme here: the compact
>> "minified" form that Matlab's jsonencode produces is probably achievable
>> without such extreme measures. It's just all whitespace elided. (Whether
>> this can be practically achieved without violating the Matlab license is
>> another question.)
>>
>> Given that the JSON spec is kind of loose and there are a lot of
>> extensions to it, but the base spec is pretty simple, IMHO
>> byte-identical JSON output is a worthy and maybe achievable goal here.
>> (Identical decoding behavior on arbitrary inputs is a whole nother ball
>> game, and I wouldn't suggest pursuing that.)
>>
>> Cheers,
>> Andrew
>>
>
> @Andy: thanks for the pointer to whitespace issues.  Did you experience
> differences while implementing octave-rapidjson regarding whitespace?  I
> think Abdallah would be interested to hear about your experience.

When implementing octave-jsonstuff [1] (not octave-rapidjson! [2] That's
a different project by a different Andy!), I did not care about
whitespace. :) All I cared about is whether the outputs resulted in the
same data model when parsed back in by a "conformant" JSON parser
(whatever that is). In fact, one of my priorities in implementing
jsonstuff was to add support for pretty-printing, because I find
"compact" JSON to be unreadable and hard to debug.

So I guess I'm suggesting that this project have higher standards than I
did. :)

> Regarding the JSON test suite of Abdallah, I think it is a first good
> step have unit tests for genuine Matlab input and output in first place
> (it might also change with the next Matlab release ...).
>
> If in the second GSoC block (own implementation) we notice whitespace
> differences, Abdallah can judge how to handle them, e.g. strtrim(), etc.
>  In general I think white space is not that much an issue with JSON
> itself.  Like Andrew says, patching RapidJSON for byte-identical JSON
> seems a high price to pay at the start of the implementation.  I vote
> for getting things started first, before thinking about fixing
> nonexistent issues yet ;-)
>
> Kai

I vote with Kai here. Perfect is the enemy of good.

Cheers,
Andrew

[1] https://github.com/apjanke/octave-jsonstuff
[2] https://github.com/Andy1978/octave-rapidjson

Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

siko1056
On 6/23/20 3:11 PM, Andrew Janke wrote:

> [...]
>> @Andy: thanks for the pointer to whitespace issues.
> [...]
> When implementing octave-jsonstuff [1] (not octave-rapidjson! [2] That's
> a different project by a different Andy!)
> [...]
>
> Cheers,
> Andrew
>
> [1] https://github.com/apjanke/octave-jsonstuff
> [2] https://github.com/Andy1978/octave-rapidjson
>


Sorry for the confusion.  In my previous mail, I identified "Andreas"
with his signature "Andy" and you by your signature "Andrew".

Kai

Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

Andrew Janke-2


On 6/23/20 2:17 AM, Kai Torben Ohlhus wrote:

> On 6/23/20 3:11 PM, Andrew Janke wrote:
>> [...]
>>> @Andy: thanks for the pointer to whitespace issues.
>> [...]
>> When implementing octave-jsonstuff [1] (not octave-rapidjson! [2] That's
>> a different project by a different Andy!)
>> [...]
>>
>> Cheers,
>> Andrew
>>
>> [1] https://github.com/apjanke/octave-jsonstuff
>> [2] https://github.com/Andy1978/octave-rapidjson
>>
>
>
> Sorry for the confusion.  In my previous mail, I identified "Andreas"
> with his signature "Andy" and you by your signature "Andrew".
>
> Kai
>

LOL. No worries, sometimes I get confused myself. There's a lot of
hackin' Andr*s out there.

Cheers,
Andrew

Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

Andreas Weber-6
In reply to this post by Andrew Janke-2
Am 23.06.20 um 08:11 schrieb Andrew Janke:
> When implementing octave-jsonstuff [1] (not octave-rapidjson! [2] That's
> a different project by a different Andy!),

The octave-rapidjson guy is me, Andy thus Kai was writing @Andy
-- Andy

Reply | Threaded
Open this post in threaded view
|

Re: JSON Test suite status

Andrew Janke-2

On 6/23/20 2:31 AM, Andreas Weber wrote:
> Am 23.06.20 um 08:11 schrieb Andrew Janke:
>> When implementing octave-jsonstuff [1] (not octave-rapidjson! [2] That's
>> a different project by a different Andy!),
>
> The octave-rapidjson guy is me, Andy thus Kai was writing @Andy
> -- Andy

Oh, sorry! I need to learn to read better.

Cheers,
Andrew