Problem with regdatasmooth (part of data-smoothing package)
Not sure if this is the correct forum for asking a question about a package, but here goes anyways. I asked another question about a week about (see Calling Octave from a Fortran Program), but decided to write a post-processing program instead. I want to use regdatasmooth to smooth out a large set of data (a million points). I tested it with 2000 points and the default 'd' value (2) and I specified a lambda value of 1e-6, which gave me a result in a few seconds. Then I tried 3000 points and it hung for 30 minutes before being killed by admin (I'm running it on an HPC cluster). I tried 2500, 2300, 2200 points etc. and had the same problem. But then I tried 2000 points again and it completed in a few seconds. So I'm not sure whats happening...
So I tried using a first-order penalty (i.e. a 'd' value of one) with the same lambda value and it took only a few seconds to complete 3000 data points. So I went ~ an order of magnitude higher and tried to smooth 50 000 data points, and I got the following error:
"error: memory exhausted or requested size too large for range of Octave's index type -- trying to return to prompt"
...which by googling I determined probably means I need to compile a 64 bit version of Octave rather than the default 32 bit, but I'm not 100% sure on this.
So if I could fix either one of these problems I'd be ok. I'd rather have it working with a 'd' value of 2, as that gives me better results. Or maybe the data-smoothing package isn't meant for such large data sets?