fillmissing

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

fillmissing

Claude Marinier
Allô,

While taking an online MATLAB course and working on an exercise, I needed to use the fillmissing function. It is not included in the Octave version installed on Ubuntu 18.04 : GNU Octave, version 4.2.2

I searched for it or a replacement and found nothing useful. There seems to be something on the Math Exchange, but we are not allowed to use those. :-(

I created a quick MEX file to do this. The function takes a vector and replaces all NaN values with an interpolated value (linear). The code seems to work for the test cases I created.

Three things.

1) In the process, I found that line 83 in mex.h has a typo. I changed
#define mexGetNaN mxGetNan
to
#define mexGetNaN mxGetNaN
This fixed a compilation error.

2) Should there be functions equivalent to mexIsNaN and mexGetNaN for NA (Not Available)? Those would make a good addition to fillmissing; it seems like the expected use of NA - missing values.

3) Is there interest in this function? I can provide the C code and the test script.

Merci.

--
ClaudeM


Reply | Threaded
Open this post in threaded view
|

Re: fillmissing

apjanke-floss


On 6/10/20 9:37 PM, Claude Marinier wrote:

> Allô,
>
> While taking an online MATLAB course and working on an exercise, I
> needed to use the fillmissing function. It is not included in the Octave
> version installed on Ubuntu 18.04 : GNU Octave, version 4.2.2
>
> I searched for it or a replacement and found nothing useful. There seems
> to be something on the Math Exchange, but we are not allowed to use
> those. :-(
>
> I created a quick MEX file to do this. The function takes a vector and
> replaces all NaN values with an interpolated value (linear). The code
> seems to work for the test cases I created.
>
> Three things.
>
> 1) In the process, I found that line 83 in mex.h has a typo. I changed
> #define mexGetNaN mxGetNan
> to
> #define mexGetNaN mxGetNaN
> This fixed a compilation error.
>
> 2) Should there be functions equivalent to mexIsNaN and mexGetNaN for NA
> (Not Available)? Those would make a good addition to fillmissing; it
> seems like the expected use of NA - missing values.
>
> 3) Is there interest in this function? I can provide the C code and the
> test script.
>
> Merci.
>
> --
> ClaudeM
>
>

I have also made an attempt to implement fillmissing() and the rest of
the "missing" semantics as part my my Tablicious package; please have a
look at that, too.

https://github.com/apjanke/octave-tablicious
https://github.com/apjanke/octave-tablicious/blob/master/inst/fillmissing.m

1) You're talking about libinterp/corefcn/mex.h, right? Yup, that sure
looks like a typo bug to me. I see this, in my checkout of the current
Octave repo at default:

  grep -ri mexgetnan *
libinterp/corefcn/mex.h:#define mexGetNaN mxGetNan
💓  ~/l/r/octave 
  grep -ri mxgetnan *
libinterp/corefcn/mexproto.h:extern OCTINTERP_API double mxGetNaN (void);
libinterp/corefcn/mex.h:#define mexGetNaN mxGetNan
libinterp/corefcn/mex.cc:mxGetNaN (void)
🐙  ~/l/r/octave 
 

Are you up for reporting this as a bug against Octave in the main Octave
bug tracker? https://savannah.gnu.org/bugs/?group=octave

I'm surprised nobody else has run in to this???

2) I think "no": recently, Octave has been focused on Matlab
compatibility, and Matlab does not define a NA value. Some other
programming languages do – in that case, NA, is a distict subset of the
NaN values defined by IEEE 754. But that is language-specific, and
neither IEEE 754 itself nor Matlab define NA, so I'm guessing Octave
would not be interested in implementing that.

3) Certainly. Octave is, generally speaking, interested in implementing
everything that Matlab does. And, personally, I think that fillmissing
and its friends are useful. But this needs to be part of a larger
discussion about how to implement the whole of "missing" semantics in
Octave, not an isolated function.

Thanks for bringing this up!

And regardless of whether it's adopted by core Octave, I think we'd be
interested in seeing your code. Any chance it's on GitHub or somewhere
public?

Cheers,
Andrew


Reply | Threaded
Open this post in threaded view
|

Re: fillmissing

Claude Marinier

On Wed, 10 Jun 2020 at 22:25, Andrew Janke <[hidden email]> wrote:
On 6/10/20 9:37 PM, Claude Marinier wrote:
> While taking an online MATLAB course and working on an exercise, I
> needed to use the fillmissing function. It is not included in the Octave
> version installed on Ubuntu 18.04 : GNU Octave, version 4.2.2
>
> I searched for it or a replacement and found nothing useful. There seems
> to be something on the Math Exchange, but we are not allowed to use
> those. :-(
>
> I created a quick MEX file to do this. The function takes a vector and
> replaces all NaN values with an interpolated value (linear). The code
> seems to work for the test cases I created.
>
> Three things.
>
> 1) In the process, I found that line 83 in mex.h has a typo. I changed
> #define mexGetNaN mxGetNan
> to
> #define mexGetNaN mxGetNaN
> This fixed a compilation error.
>
> 2) Should there be functions equivalent to mexIsNaN and mexGetNaN for NA
> (Not Available)? Those would make a good addition to fillmissing; it
> seems like the expected use of NA - missing values.
>
> 3) Is there interest in this function? I can provide the C code and the
> test script.
>
> Merci.

I have also made an attempt to implement fillmissing() and the rest of
the "missing" semantics as part my my Tablicious package; please have a
look at that, too.

https://github.com/apjanke/octave-tablicious
https://github.com/apjanke/octave-tablicious/blob/master/inst/fillmissing.m

I will have a look. It may be farther along than my quick simple function. Thanks.

1) You're talking about libinterp/corefcn/mex.h, right? Yup, that sure
looks like a typo bug to me. I see this, in my checkout of the current
Octave repo at default:

grep -ri mexgetnan *
libinterp/corefcn/mex.h:#define mexGetNaN mxGetNan
grep -ri mxgetnan *
libinterp/corefcn/mexproto.h:extern OCTINTERP_API double mxGetNaN (void);
libinterp/corefcn/mex.h:#define mexGetNaN mxGetNan
libinterp/corefcn/mex.cc:mxGetNaN (void) 

The file is here : /usr/include/octave-4.2.2/octave/mex.h
 
Are you up for reporting this as a bug against Octave in the main Octave
bug tracker? https://savannah.gnu.org/bugs/?group=octave
 
Done.

2) I think "no": recently, Octave has been focused on Matlab
compatibility, and Matlab does not define a NA value. Some other
programming languages do – in that case, NA, is a distict subset of the
NaN values defined by IEEE 754. But that is language-specific, and
neither IEEE 754 itself nor Matlab define NA, so I'm guessing Octave
would not be interested in implementing that.

Agreed.

3) Certainly. Octave is, generally speaking, interested in implementing
everything that Matlab does. And, personally, I think that fillmissing
and its friends are useful. But this needs to be part of a larger
discussion about how to implement the whole of "missing" semantics in
Octave, not an isolated function.

Thanks for bringing this up!

And regardless of whether it's adopted by core Octave, I think we'd be
interested in seeing your code. Any chance it's on GitHub or somewhere
public?


--
Claude Marinier


Reply | Threaded
Open this post in threaded view
|

Re: fillmissing

nrjank
Administrator

Are you up for reporting this as a bug against Octave in the main Octave
bug tracker? https://savannah.gnu.org/bugs/?group=octave
 
Done.

just for traceability and closure, the bug report is at 



Reply | Threaded
Open this post in threaded view
|

Re: fillmissing

Juan Pablo Carbajal-2
In reply to this post by apjanke-floss
> 2) I think "no": recently, Octave has been focused on Matlab
> compatibility, and Matlab does not define a NA value. Some other
> programming languages do – in that case, NA, is a distict subset of the
> NaN values defined by IEEE 754. But that is language-specific, and
> neither IEEE 754 itself nor Matlab define NA, so I'm guessing Octave
> would not be interested in implementing that.

I am not sure I understand the answer but Octave has NA values, and
NaN values,  since long. Check
> NA(3,2)
ans =

    NA    NA
    NA    NA
    NA    NA

and the logic of NA values is consistent with missing data
https://octave.org/doc/interpreter/Missing-Data.html#Missing-Data


Reply | Threaded
Open this post in threaded view
|

Re: fillmissing

Juan Pablo Carbajal-2
On Sat, Jun 13, 2020 at 11:35 PM Juan Pablo Carbajal
<[hidden email]> wrote:

>
> > 2) I think "no": recently, Octave has been focused on Matlab
> > compatibility, and Matlab does not define a NA value. Some other
> > programming languages do – in that case, NA, is a distict subset of the
> > NaN values defined by IEEE 754. But that is language-specific, and
> > neither IEEE 754 itself nor Matlab define NA, so I'm guessing Octave
> > would not be interested in implementing that.
>
> I am not sure I understand the answer but Octave has NA values, and
> NaN values,  since long. Check
> > NA(3,2)
> ans =
>
>     NA    NA
>     NA    NA
>     NA    NA
>
> and the logic of NA values is consistent with missing data
> https://octave.org/doc/interpreter/Missing-Data.html#Missing-Data

BTW, pandas has implemented NA since version 1.0
https://pandas.pydata.org/pandas-docs/stable/user_guide/missing_data.html


Reply | Threaded
Open this post in threaded view
|

Re: fillmissing

apjanke-floss


On 6/13/20 5:36 PM, Juan Pablo Carbajal wrote:

> On Sat, Jun 13, 2020 at 11:35 PM Juan Pablo Carbajal
> <[hidden email]> wrote:
>>
>>> 2) I think "no": recently, Octave has been focused on Matlab
>>> compatibility, and Matlab does not define a NA value. Some other
>>> programming languages do – in that case, NA, is a distict subset of the
>>> NaN values defined by IEEE 754. But that is language-specific, and
>>> neither IEEE 754 itself nor Matlab define NA, so I'm guessing Octave
>>> would not be interested in implementing that.
>>
>> I am not sure I understand the answer but Octave has NA values, and
>> NaN values,  since long. Check
>>> NA(3,2)
>> ans =
>>
>>      NA    NA
>>      NA    NA
>>      NA    NA
>>
>> and the logic of NA values is consistent with missing data
>> https://octave.org/doc/interpreter/Missing-Data.html#Missing-Data
>
> BTW, pandas has implemented NA since version 1.0
> https://pandas.pydata.org/pandas-docs/stable/user_guide/missing_data.html

Ha! This is news to me. I guess Octave has joined the family of
languages that defines a NA. Guess I was wrong about what the Octave
developers' answer would be here.

Cheers,
Andrew