# averaging/smoothing data

7 messages
Open this post in threaded view
|

## averaging/smoothing data

 [sorry for repost: maybe my original post gave an impression that I am not asking seriously, but I think it's an important and frequent use case, so here it is again] Often, I need to smooth/average a data vector. I usually do a running average using filter(), e.g. for 5-value average:     filter(ones(1:5)/5,eye(5,1),x) I've been doing this for years, so I got used to it even though I have to think a little every time I need to do again, but maybe there's a better way? What do y'all use? By the way, sometimes I'd use a Savitzky-Golay peak/moment-preserving filter coefficients instead of a simple rectangular averaging window: # Copyright (C) 1993 Przemek Klosowski ; licensed under GPL # # You should have received a copy of the GNU General Public License # along with Octave; see the file COPYING.  If not, write to the Free # Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA. function retval = savgol (m,nl,nr) # usage: savgol (m,n) or (m,nl,nr)  # m - order -nl..nr - smoothing window # # Return the Savitzky-Golay filter coeffs for 2n window, order m.  # Ref.: Num.Rec. (2nd edition) p. 650 # Try plot([savgol(2,8);savgol(4,8);savgol(6,8)]'); sometime # filter coeffs are in first row of (A'*A)^-1 * A'   if (nargin == 2)     nr = nl; nl = nl;   elseif (nargin!=3)     error ("usage: savgol (n,m)");   endif   nmax = length(nl) * length(nr) * length(m);   if (nmax == 1)    if (nl<0 || nr<0 )      error (" n must be positive in  savgol()");    endif    ii = -nl:nr;    jj = 0:m;    A  = repmat(ii',1,m+1).^repmat(jj,nl+nr+1,1) ;    A=(A'*A)^-1 * A';    retval=A(1,:);   else     error ("savgol: expecting scalar arguments, found something else");   endif endfunction _______________________________________________ Help-octave mailing list [hidden email] https://lists.gnu.org/mailman/listinfo/help-octave
Open this post in threaded view
|

## Re: averaging/smoothing data

 On Wed, Jul 13, 2016 at 11:58 AM, Przemek Klosowski wrote: [sorry for repost: maybe my original post gave an impression that I am not asking seriously, but I think it's an important and frequent use case, so here it is again] Often, I need to smooth/average a data vector. I usually do a running average using filter(), e.g. for 5-value average:     filter(ones(1:5)/5,eye(5,1),x) I've been doing this for years, so I got used to it even though I have to think a little every time I need to do again, but maybe there's a better way? What do y'all use? By the way, sometimes I'd use a Savitzky-Golay peak/moment-preserving filter coefficients instead of a simple rectangular averaging window: # Copyright (C) 1993 Przemek Klosowski ; licensed under GPL # # You should have received a copy of the GNU General Public License # along with Octave; see the file COPYING.  If not, write to the Free # Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA. function retval = savgol (m,nl,nr)​​There are also sgolay and agolayfilt in signal package.​ Dmi​tri.-- _______________________________________________ Help-octave mailing list [hidden email] https://lists.gnu.org/mailman/listinfo/help-octave
Open this post in threaded view
|

## Re: averaging/smoothing data

 In reply to this post by Przemek Klosowski-7 On 13/07/16 09:58, Przemek Klosowski wrote: > Often, I need to smooth/average a data vector. I usually do a running > average using filter(), e.g. for 5-value average: > >     filter(ones(1:5)/5,eye(5,1),x) Did you mean: filter(ones(1,5)/5, eye(5,1), x) I don't understand what your example does, and it doesn't seem to do anything nice to a random 1D x. (I don't have a better way, but maybe that would explain why you didn't much response). Colin _______________________________________________ Help-octave mailing list [hidden email] https://lists.gnu.org/mailman/listinfo/help-octave
Open this post in threaded view
|

## Re: averaging/smoothing data

 In reply to this post by Przemek Klosowski-7 On Wed, 13 Jul 2016 12:58:29 -0400 Przemek Klosowski <[hidden email]> wrote: > Often, I need to smooth/average a data vector. I have had the good fortune of seeing wide distributions and lots of outliers over the years.  If there is a significant probability of picking up data from some other distribution (such as blunders in GPS), I like to use a double window, median filter.  I use a wider window to find what data to ignore, and then I find the median within the inner window. Another use case associated with GPS, is when the combine harvestor (farming) has finished cutting grain in the current path, and is traversing ground that has already been cut and processed, the yield monitor data is not useful.  This is when processing data as a time series, and not processing it in conjunction with location. Gord _______________________________________________ Help-octave mailing list [hidden email] https://lists.gnu.org/mailman/listinfo/help-octave
Open this post in threaded view
|

## RE: averaging/smoothing data

 In reply to this post by Przemek Klosowski-7 Przemek, From: Help-octave [mailto:help-octave-bounces+allen.windhorn=[hidden email]] On Behalf Of Przemek Klosowski > Often, I need to smooth/average a data vector. I usually do a running > average using filter(), e.g. for 5-value average: >     filter(ones(1:5)/5,eye(5,1),x) > I've been doing this for years, so I got used to it even though I have to > think a little every time I need to do again, but maybe there's a better > way? What do y'all use? If you have "spiky" data it really helps to run it through a median filter (signals package) before you do any linear filtering. Yfilt = medfilt1(Y, n); This gets rid of noise that only affects one reading, but doesn't affect the shape of the signal very much. Regards, Allen _______________________________________________ Help-octave mailing list [hidden email] https://lists.gnu.org/mailman/listinfo/help-octave