io.tst failures with lto

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

io.tst failures with lto

Dmitri A. Sergatskov
it is not only int32 that is wrong, the remaining (uint32, int64 , and uint64).
test io.tst bails out on the first failure.
here is the outputs (in bit format). Hopefully it gives some clues.


octave:35> s32
s32 =

  11011001101010010011101110110101  00000000000000000000000000000000
  11000101100111111011000110100100  00000000000000000000000000000000

octave:36> s32t
s32t =

  11011001101010010011101110110101  01110101100110011111110000101100
  11000101100111111011000110100100  01100110010000110100110100010010

octave:37> u32
u32 =

  01111101110100100011100010001100  00000000000000000000000000000000
  01101110001010010011001010000101  00000000000000000000000000000000

octave:38> u32t
u32t =

  01111101110100100011100010001100  00000000000000000000000000000000
  01101110001010010011001010000101  01010111010110010101100100101001

octave:39> s64
s64 =

  0010111010011111101011110001110101000010001110001100100000000000  0000000000000000000000000000000000000000000000000000000000000000
  0000000000000000000000000000000000000000000000000000000000000000  0000000000000000000000000000000000000000000000000000000000000000

octave:40> s64t
s64t =

  0010111010011111101011110001110101000010001110001100100000000000  0100110010110010101100011000100101111001100001000111100000000000
  1000010011110111110110001111110001000000110100101011010000000000  0001001101110100101111011111001010101101000010111010100000000000

octave:41> u64
u64 =

  0010111000101000100110001001100001111111010100110000000000000000  0000000000000000000000000000000000000000000000000000000000000000
  0000000000000000000000000000000000000000000000000000000000000000  0000000000000000000000000000000000000000000000000000000000000000

octave:42> u64t
u64t =

  0010111000101000100110001001100001111111010100110000000000000000  0001000101000011111110000010001110011011000010110001100000000000
  0011001100101100000100111000100000101111100010001110000000000000  0101101110101110011011010010001010110110100111110001000000000000


Dmitri.

Reply | Threaded
Open this post in threaded view
|

Re: io.tst failures with lto

Dmitri A. Sergatskov
On Sat, Apr 22, 2017 at 11:58 PM, Dmitri A. Sergatskov <[hidden email]> wrote:
it is not only int32 that is wrong, the remaining (uint32, int64 , and uint64).
test io.tst bails out on the first failure.
here is the outputs (in bit format). Hopefully it gives some clues.


octave:35> s32
s32 =

  11011001101010010011101110110101  00000000000000000000000000000000
  11000101100111111011000110100100  00000000000000000000000000000000

octave:36> s32t
s32t =

  11011001101010010011101110110101  01110101100110011111110000101100
  11000101100111111011000110100100  01100110010000110100110100010010

octave:37> u32
u32 =

  01111101110100100011100010001100  00000000000000000000000000000000
  01101110001010010011001010000101  00000000000000000000000000000000

octave:38> u32t
u32t =

  01111101110100100011100010001100  00000000000000000000000000000000
  01101110001010010011001010000101  01010111010110010101100100101001

octave:39> s64
s64 =

  0010111010011111101011110001110101000010001110001100100000000000  0000000000000000000000000000000000000000000000000000000000000000
  0000000000000000000000000000000000000000000000000000000000000000  0000000000000000000000000000000000000000000000000000000000000000

octave:40> s64t
s64t =

  0010111010011111101011110001110101000010001110001100100000000000  0100110010110010101100011000100101111001100001000111100000000000
  1000010011110111110110001111110001000000110100101011010000000000  0001001101110100101111011111001010101101000010111010100000000000

octave:41> u64
u64 =

  0010111000101000100110001001100001111111010100110000000000000000  0000000000000000000000000000000000000000000000000000000000000000
  0000000000000000000000000000000000000000000000000000000000000000  0000000000000000000000000000000000000000000000000000000000000000

octave:42> u64t
u64t =

  0010111000101000100110001001100001111111010100110000000000000000  0001000101000011111110000010001110011011000010110001100000000000
  0011001100101100000100111000100000101111100010001110000000000000  0101101110101110011011010010001010110110100111110001000000000000


Dmitri.


​Here is the test file. Note,  if you read it with non-lto binary the values will be different again.​

​Dmitri.


t1.hdf5 (22K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: io.tst failures with lto

Dmitri A. Sergatskov
​using h5dump utility I see with the file saved with lto binary that all integers datatypes are no set to
H5T_STD_I16LE
E.g.:


GROUP "u8" {
      ATTRIBUTE "OCTAVE_NEW_FORMAT" {
         DATATYPE  H5T_STD_U8LE
         DATASPACE  SCALAR
         DATA {
         (0): 1
         }
      }
      DATASET "type" {
         DATATYPE  H5T_STRING {
            STRSIZE 13;
            STRPAD H5T_STR_NULLTERM;
            CSET H5T_CSET_ASCII;
            CTYPE H5T_C_S1;
         }
         DATASPACE  SCALAR
         DATA {
         (0): "uint8 matrix"
         }
      }
      DATASET "value" {
         DATATYPE  H5T_STD_I16LE
         DATASPACE  SIMPLE { ( 2, 2 ) / ( 2, 2 ) }
         DATA {
         (0,0): 0, 21504,
         (1,0): 32547, 0
         }
      }
   }

and

GROUP "u64" {
      ATTRIBUTE "OCTAVE_NEW_FORMAT" {
         DATATYPE  H5T_STD_U8LE
         DATASPACE  SCALAR
         DATA {
         (0): 1
         }
      }
      DATASET "type" {
         DATATYPE  H5T_STRING {
            STRSIZE 14;
            STRPAD H5T_STR_NULLTERM;
            CSET H5T_CSET_ASCII;
            CTYPE H5T_C_S1;
         }
         DATASPACE  SCALAR
         DATA {
         (0): "uint64 matrix"
         }
      }
      DATASET "value" {
         DATATYPE  H5T_STD_I16LE
         DATASPACE  SIMPLE { ( 2, 2 ) / ( 2, 2 ) }
         DATA {
         (0,0): 0, 32595,
         (1,0): -26472, 11816
         }
      }
   }

But the file saved with octave compiled with default optimization I get the appropriate types:


GROUP "u8" {
      ATTRIBUTE "OCTAVE_NEW_FORMAT" {
         DATATYPE  H5T_STD_U8LE
         DATASPACE  SCALAR
         DATA {
         (0): 1
         }
      }
      DATASET "type" {
         DATATYPE  H5T_STRING {
            STRSIZE 13;
            STRPAD H5T_STR_NULLTERM;
            CSET H5T_CSET_ASCII;
            CTYPE H5T_C_S1;
         }
         DATASPACE  SCALAR
         DATA {
         (0): "uint8 matrix"
         }
      }
      DATASET "value" {
         DATATYPE  H5T_STD_U8LE
         DATASPACE  SIMPLE { ( 2, 2 ) / ( 2, 2 ) }
         DATA {
         (0,0): 120, 0,
         (1,0): 120, 67
         }
      }
   }
}

  GROUP "u64" {
      ATTRIBUTE "OCTAVE_NEW_FORMAT" {
         DATATYPE  H5T_STD_U8LE
         DATASPACE  SCALAR
         DATA {
         (0): 1
         }
      }
      DATASET "type" {
         DATATYPE  H5T_STRING {
            STRSIZE 14;
            STRPAD H5T_STR_NULLTERM;
            CSET H5T_CSET_ASCII;
            CTYPE H5T_C_S1;
         }
         DATASPACE  SCALAR
         DATA {
         (0): "uint64 matrix"
         }
      }
      DATASET "value" {
         DATATYPE  H5T_STD_U64LE
         DATASPACE  SIMPLE { ( 2, 2 ) / ( 2, 2 ) }
         DATA {
         (0,0): 0, 2021257291668197376,
         (1,0): 682960284105943040, 0
         }
      }
   }

Dmitri.



Reply | Threaded
Open this post in threaded view
|

Re: io.tst failures with lto

John W. Eaton
Administrator
On 04/23/2017 02:50 AM, Dmitri A. Sergatskov wrote:
> ​using h5dump utility I see with the file saved with lto binary that all
> integers datatypes are no set to
> H5T_STD_I16LE

Thanks for looking at this.

I guess I would start by examining how "value" is written and whether
the DATATYPE tag is correct at the time the HDF5 function that writes it
is called.

Also, it might help use the code from Octave to create a stand-alone
program that writes this same data and see whether that also shows the
problem.  At least that might be easier to debug.  If that doesn't show
the problem, then maybe we are calling HDF5 functions incorrectly?

jwe


Reply | Threaded
Open this post in threaded view
|

Re: io.tst failures with lto

Dmitri A. Sergatskov
On Tue, Apr 25, 2017 at 2:54 PM, John W. Eaton <[hidden email]> wrote:
On 04/23/2017 02:50 AM, Dmitri A. Sergatskov wrote:
​using h5dump utility I see with the file saved with lto binary that all
integers datatypes are no set to
H5T_STD_I16LE

Thanks for looking at this.

I guess I would start by examining how "value" is written and whether the DATATYPE tag is correct at the time the HDF5 function that writes it is called.

Also, it might help use the code from Octave to create a stand-alone program that writes this same data and see whether that also shows the problem.  At least that might be easier to debug.  If that doesn't show the problem, then maybe we are calling HDF5 functions incorrectly?

jwe


​I noticed that we are not linking to hdf5_cpp (C++ interface to hdf5 library); perhaps this is why we needed all this
trickery with extern "C" and ​coct-hdf5-types.c ?

I tried sample programs from HDF5 webpage and could not reproduce the problem.

Dmitri.
--