Re: Save produces large files

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Save produces large files

David Bateman-3
Mike Miller a écrit :

> On Tue, 14 Jun 2005, David Bateman wrote:
>
>> Søren Hauberg a écrit :
>>
>>> a = zeros(3); a(5) = NaN;
>>> octave:25> save -float-binary test.data a
>>>
>>> which gives me the following warning:
>>>
>>> warning: save: some values too large to save as floats --
>>> warning: save: saving as doubles instead
>>>
>>> Btw. I'm using 2.1.71 on linux
>>
>>
>> That's pretty stupid behaviour... Perhaps the test for whether data
>> can be saved as float needs to be updated to allow that NaN can be
>> saved as float.....
>
>
>
> How is NaN encoded?  The "too large to save as floats" seems to imply
> that NaN is encoded as a large value.  Is that the case?
>
> Mike
No it implies that the > and < operators with NaN on the left-hand side
always return true... Consider the attached patch (John do I commit it?)

D.


2005-06-14  David Bateman  <[hidden email]>

    * dMatrix.cc (too_large_for_float): Special case NaN and NA values
    * CMatrix.cc (too_large_for_float): Ditto


*** liboctave/CMatrix.cc~ 2005-04-28 23:42:46.000000000 +0200
--- liboctave/CMatrix.cc 2005-06-14 16:51:07.532501906 +0200
***************
*** 2607,2616 ****
  double r_val = std::real (val);
  double i_val = std::imag (val);
 
! if (r_val > FLT_MAX
!    || i_val > FLT_MAX
!    || r_val < FLT_MIN
!    || i_val < FLT_MIN)
   return true;
        }
 
--- 2607,2616 ----
  double r_val = std::real (val);
  double i_val = std::imag (val);
 
! if (((! octave_is_NaN_or_NA (r_val)) &&
!     (r_val > FLT_MAX || r_val < FLT_MIN)) ||
!    ((! octave_is_NaN_or_NA (i_val)) &&
!     (i_val > FLT_MAX || i_val < FLT_MIN)))    
   return true;
        }
 
*** liboctave/dMatrix.cc~ 2005-04-26 21:28:16.000000000 +0200
--- liboctave/dMatrix.cc 2005-06-14 16:47:20.121487592 +0200
***************
*** 2114,2120 ****
      {
        double val = elem (i);
 
!       if (val > FLT_MAX || val < FLT_MIN)
  return true;
      }
 
--- 2114,2120 ----
      {
        double val = elem (i);
 
!       if ((! octave_is_NaN_or_NA (val)) && (val > FLT_MAX || val < FLT_MIN))
  return true;
      }
 
Reply | Threaded
Open this post in threaded view
|

Re: Save produces large files

John W. Eaton-6
On 14-Jun-2005, David Bateman wrote:

| No it implies that the > and < operators with NaN on the left-hand side
| always return true... Consider the attached patch (John do I commit it?)

I applied the patch.

Thanks,

jwe

Reply | Threaded
Open this post in threaded view
|

Re: Save produces large files

John W. Eaton-6
On 14-Jun-2005, John W. Eaton wrote:

| On 14-Jun-2005, David Bateman wrote:
|
| | No it implies that the > and < operators with NaN on the left-hand side
| | always return true... Consider the attached patch (John do I commit it?)
|
| I applied the patch.

I also added a check for Inf values and made similar changes to the
N-d array classes so that it would work for the octave_value matrix
objects (which use N-d array objects internally).

jwe

Reply | Threaded
Open this post in threaded view
|

Re: Save produces large files

David Bateman-3
John W. Eaton a écrit :

>On 14-Jun-2005, John W. Eaton wrote:
>
>| On 14-Jun-2005, David Bateman wrote:
>|
>| | No it implies that the > and < operators with NaN on the left-hand side
>| | always return true... Consider the attached patch (John do I commit it?)
>|
>| I applied the patch.
>
>I also added a check for Inf values and made similar changes to the
>N-d array classes so that it would work for the octave_value matrix
>objects (which use N-d array objects internally).
>
>jwe
>  
>
You also need similar changes in ls-mat5.cc as it has independent code
to calculate whether float will be used...

D.

Reply | Threaded
Open this post in threaded view
|

Re: Save produces large files

John W. Eaton-6
On 14-Jun-2005, David Bateman wrote:

| You also need similar changes in ls-mat5.cc as it has independent code
| to calculate whether float will be used...

Fixed.

BTW, the test is now

      if (! (octave_is_NaN_or_NA (val) || xisinf (val))
          && fabs (val) > FLT_MAX)
        return true;

so NaN, NA, and Inf values are allowed, and we only check if the
absolute value is out of range.  Small (absolute) values (i.e., <
FLT_MIN, or approximately 1e-38) will be silently converted to zero.
Is that OK?  I haven't checked the Matlab docs to see whether this is
compatible behavior.

Also, NA is not preserved.  On writing, it is converted to an ordinary
float NaN, without the special value in the mantissa.  To preserve NA
when saving in single precision, we will need special code to convert
a double NA to a float NA when saving, and also to convert a float NA
to a double NA when reading.

jwe