Quantcast

memory leak in regexprep ? (windows/3.7.2+/VS)

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

memory leak in regexprep ? (windows/3.7.2+/VS)

Christoph Ellenberger-2
I am experiencing some strange behavior and wanted to check if it is  a bug or me....
It happens when I try to read in and manipulating some ascii data files. It only happens with the VS version of octave. (tested with several gcc versions on windows and didn't showed up).
Ok here is some example code to reproduce:
First generating an ascii file:
s="0123456789";
for i=1:21, s=cstrcat(s,s);,endfor
save TestData.dat s
 
Then readin and manipulate:
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
fclose(fid);
s = regexprep(s,'0','2');
 
Clear all and repeat:
clear all
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
fclose(fid);
s = regexprep(s,'0','2');
 
Depending on datasize and how many replacement were performed it either fails on the new readin or regexprep with a: "error: out of memory or dimension too large for Octave's index type"
Any help is appreciated
Christoph
 
 

_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: memory leak in regexprep ? (windows/3.7.2+/VS)

tmacchant
--- On Tue, 2013/3/26, Christoph Ellenberger  wrote:

> I am experiencing some strange behavior and wanted to check if it is  a bug or me....
> It happens when I try to read in and manipulating some ascii data files. It only happens with the VS version of octave. (tested with several gcc versions on windows and didn't showed up).
> Ok here is some example code to reproduce:
> First generating an ascii file:
> s="0123456789";
> for i=1:21, s=cstrcat(s,s);,endfor
> save TestData.dat s
>  
> Then readin and manipulate:
> fid = fopen("TestData.dat","r");
> s = fscanf(fid,'%c');
> fclose(fid);
> s = regexprep(s,'0','2');
>  
> Clear all and repeat:
> clear all
>
> fid = fopen("TestData.dat","r");
> s = fscanf(fid,'%c');
> fclose(fid);
> s = regexprep(s,'0','2');
>  
> Depending on datasize and how many replacement were performed it either fails on the new readin or regexprep with a: "error: out of memory or dimension too large for Octave's index type"
> Any help is appreciated
> Christoph

I have tested on octave-3.6.2 VS, 3.6.2 MinGW, 3.7.2+ VS and 3.6.2 cygwin
(win 7 64 bit Home premium, 4GB memory)


1. octave-3.6.2 VS and 3.6.2 MinGW*****************************
>> fid = fopen("TestData.dat","r");
>> s = fscanf(fid,'%c');
warning: range error for conversion to character value
>> fclose(fid);
>> s = regexprep(s,'0','2');
>> clear all
>> fid = fopen("TestData.dat","r");
>> s = fscanf(fid,'%c');
error: memory exhausted or requested size too large for range of Octave's index type -- trying to return to prompt
********************************************************************


2. octave-3.7.2+ VS*****************************
>> fid = fopen("TestData.dat","r");
>> s = fscanf(fid,'%c');
warning: range error for conversion to character value
>> fclose(fid);
>> s = regexprep(s,'0','2');
>> clear all
>> fid = fopen("TestData.dat","r");
>> s = fscanf(fid,'%c');
error: out of memory or dimension too large for Octave's index type
********************************************************************

3.Octave-3.6.2 Cygwin**********************************************
>> fid = fopen("TestData.dat","r");
>> s = fscanf(fid,'%c');
>> fclose(fid);
>> s = regexprep(s,'0','2');
>> clear all
>> fid = fopen("TestData.dat","r");
>> s = fscanf(fid,'%c');
>> fclose(fid);
>> s = regexprep(s,'0','2');
********************************************************************

For me, both VS and MinGW version caused problem. Cygwin version seemed to work correctly.

Regards

Tatsuro

_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: memory leak in regexprep ? (windows/3.7.2+/VS)

tmacchant
--- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:

> --- On Tue, 2013/3/26, Christoph Ellenberger  wrote:
> > I am experiencing some strange behavior and wanted to check if it is  a bug or me....
> > It happens when I try to read in and manipulating some ascii data files. It only happens with the VS version of octave. (tested with several gcc versions on windows and didn't showed up).
> > Ok here is some example code to reproduce:
> > First generating an ascii file:
> > s="0123456789";
> > for i=1:21, s=cstrcat(s,s);,endfor
> > save TestData.dat s
> >  
> > Then readin and manipulate:
> > fid = fopen("TestData.dat","r");
> > s = fscanf(fid,'%c');
> > fclose(fid);
> > s = regexprep(s,'0','2');
> >  
> > Clear all and repeat:
> > clear all
> >
> > fid = fopen("TestData.dat","r");
> > s = fscanf(fid,'%c');
> > fclose(fid);
> > s = regexprep(s,'0','2');
> >  
> > Depending on datasize and how many replacement were performed it either fails on the new readin or regexprep with a: "error: out of memory or dimension too large for Octave's index type"
> > Any help is appreciated
> > Christoph
>
> I have tested on octave-3.6.2 VS, 3.6.2 MinGW, 3.7.2+ VS and 3.6.2 cygwin
> (win 7 64 bit Home premium, 4GB memory)
>
>
> 1. octave-3.6.2 VS and 3.6.2 MinGW*****************************
> >> fid = fopen("TestData.dat","r");
> >> s = fscanf(fid,'%c');
> warning: range error for conversion to character value
> >> fclose(fid);
> >> s = regexprep(s,'0','2');
> >> clear all
> >> fid = fopen("TestData.dat","r");
> >> s = fscanf(fid,'%c');
> error: memory exhausted or requested size too large for range of Octave's index type -- trying to return to prompt
> ********************************************************************
>
>
> 2. octave-3.7.2+ VS*****************************
> >> fid = fopen("TestData.dat","r");
> >> s = fscanf(fid,'%c');
> warning: range error for conversion to character value
> >> fclose(fid);
> >> s = regexprep(s,'0','2');
> >> clear all
> >> fid = fopen("TestData.dat","r");
> >> s = fscanf(fid,'%c');
> error: out of memory or dimension too large for Octave's index type
> ********************************************************************
>
> 3.Octave-3.6.2 Cygwin**********************************************
> >> fid = fopen("TestData.dat","r");
> >> s = fscanf(fid,'%c');
> >> fclose(fid);
> >> s = regexprep(s,'0','2');
> >> clear all
> >> fid = fopen("TestData.dat","r");
> >> s = fscanf(fid,'%c');
> >> fclose(fid);
> >> s = regexprep(s,'0','2');
> ********************************************************************
>
> For me, both VS and MinGW version caused problem. Cygwin version seemed to work correctly.
>
> Regards
>
> Tatsuro


I have also tested on
octave-3.2.4 and 3.4.3 MinGW version.

4. octave-3.2.4 and 3.4.3 MinGW *****************************
>> fid = fopen("TestData.dat","r");
>> s = fscanf(fid,'%c');
warning: range error for conversion to character value
>> fclose(fid);
>> s = regexprep(s,'0','2');
>> clear all
>> fid = fopen("TestData.dat","r");
>> s = fscanf(fid,'%c');
warning: range error for conversion to character value
>> fclose(fid);
>> s = regexprep(s,'0','2');
**************************************************

The problem seemed to be happened on octave-3.6 or later on native windows.

Regards

Tatsuro
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: memory leak in regexprep ? (windows/3.7.2+/VS)

tmacchant
--- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:

> --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
> > --- On Tue, 2013/3/26, Christoph Ellenberger  wrote:
> > > I am experiencing some strange behavior and wanted to check if it is  a bug or me....
> > > It happens when I try to read in and manipulating some ascii data files. It only happens with the VS version of octave. (tested with several gcc versions on windows and didn't showed up).
> > > Ok here is some example code to reproduce:
> > > First generating an ascii file:
> > > s="0123456789";
> > > for i=1:21, s=cstrcat(s,s);,endfor
> > > save TestData.dat s
> > >  
> > > Then readin and manipulate:
> > > fid = fopen("TestData.dat","r");
> > > s = fscanf(fid,'%c');
> > > fclose(fid);
> > > s = regexprep(s,'0','2');
> > >  
> > > Clear all and repeat:
> > > clear all
> > >
> > > fid = fopen("TestData.dat","r");
> > > s = fscanf(fid,'%c');
> > > fclose(fid);
> > > s = regexprep(s,'0','2');
> > >  
> > > Depending on datasize and how many replacement were performed it either fails on the new readin or regexprep with a: "error: out of memory or dimension too large for Octave's index type"
> > > Any help is appreciated
> > > Christoph
> >
> > I have tested on octave-3.6.2 VS, 3.6.2 MinGW, 3.7.2+ VS and 3.6.2 cygwin
> > (win 7 64 bit Home premium, 4GB memory)
> >
> >
> > 1. octave-3.6.2 VS and 3.6.2 MinGW*****************************
> > >> fid = fopen("TestData.dat","r");
> > >> s = fscanf(fid,'%c');
> > warning: range error for conversion to character value
> > >> fclose(fid);
> > >> s = regexprep(s,'0','2');
> > >> clear all
> > >> fid = fopen("TestData.dat","r");
> > >> s = fscanf(fid,'%c');
> > error: memory exhausted or requested size too large for range of Octave's index type -- trying to return to prompt
> > ********************************************************************
> >
> >
> > 2. octave-3.7.2+ VS*****************************
> > >> fid = fopen("TestData.dat","r");
> > >> s = fscanf(fid,'%c');
> > warning: range error for conversion to character value
> > >> fclose(fid);
> > >> s = regexprep(s,'0','2');
> > >> clear all
> > >> fid = fopen("TestData.dat","r");
> > >> s = fscanf(fid,'%c');
> > error: out of memory or dimension too large for Octave's index type
> > ********************************************************************
> >
> > 3.Octave-3.6.2 Cygwin**********************************************
> > >> fid = fopen("TestData.dat","r");
> > >> s = fscanf(fid,'%c');
> > >> fclose(fid);
> > >> s = regexprep(s,'0','2');
> > >> clear all
> > >> fid = fopen("TestData.dat","r");
> > >> s = fscanf(fid,'%c');
> > >> fclose(fid);
> > >> s = regexprep(s,'0','2');
> > ********************************************************************
> >
> > For me, both VS and MinGW version caused problem. Cygwin version seemed to work correctly.
> >
> > Regards
> >
> > Tatsuro
>
>
> I have also tested on
> octave-3.2.4 and 3.4.3 MinGW version.
>
> 4. octave-3.2.4 and 3.4.3 MinGW *****************************
> >> fid = fopen("TestData.dat","r");
> >> s = fscanf(fid,'%c');
> warning: range error for conversion to character value
> >> fclose(fid);
> >> s = regexprep(s,'0','2');
> >> clear all
> >> fid = fopen("TestData.dat","r");
> >> s = fscanf(fid,'%c');
> warning: range error for conversion to character value
> >> fclose(fid);
> >> s = regexprep(s,'0','2');
> **************************************************
>
> The problem seemed to be happened on octave-3.6 or later on native windows.
>
> Regards
>
> Tatsuro

Sorry for successive post.

I have tested the following  :

%*********************************
s="0123456789";
for i=1:21, s=cstrcat(s,s);,endfor
save TestData.dat s
load TestData.dat
s = regexprep(s,'0','2');
clear all
load TestData.dat
s = regexprep(s,'0','2');
%*********************************

The above worked correctly on octave-3.2.4 (MinGW), 3.4.3 (MinGW), 3.6.2 (MinGW,VS, Cygwin) and 3.7.2+ (VS).

I think that the problem does not rely on regexprep but on fscanf for large data.

Regards

Tatsuro

_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Aw: Re: memory leak in regexprep ? (windows/3.7.2+/VS)

Christoph Ellenberger-2
> --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
> > --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
> > > --- On Tue, 2013/3/26, Christoph Ellenberger  wrote:
> > > > I am experiencing some strange behavior and wanted to check if it is  a bug or me....
> > > > It happens when I try to read in and manipulating some ascii data files. It only happens with the VS version of octave. (tested with several gcc versions on windows and didn't showed up).
> > > > Ok here is some example code to reproduce:
> > > > First generating an ascii file:
> > > > s="0123456789";
> > > > for i=1:21, s=cstrcat(s,s);,endfor
> > > > save TestData.dat s
> > > >  
> > > > Then readin and manipulate:
> > > > fid = fopen("TestData.dat","r");
> > > > s = fscanf(fid,'%c');
> > > > fclose(fid);
> > > > s = regexprep(s,'0','2');
> > > >  
> > > > Clear all and repeat:
> > > > clear all
> > > >
> > > > fid = fopen("TestData.dat","r");
> > > > s = fscanf(fid,'%c');
> > > > fclose(fid);
> > > > s = regexprep(s,'0','2');
> > > >  
> > > > Depending on datasize and how many replacement were performed it either fails on the new readin or regexprep with a: "error: out of memory or dimension too large for Octave's index type"
> > > > Any help is appreciated
> > > > Christoph
> > >
> > > I have tested on octave-3.6.2 VS, 3.6.2 MinGW, 3.7.2+ VS and 3.6.2 cygwin
> > > (win 7 64 bit Home premium, 4GB memory)
> > >
> > >
> > > 1. octave-3.6.2 VS and 3.6.2 MinGW*****************************
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > warning: range error for conversion to character value
> > > >> fclose(fid);
> > > >> s = regexprep(s,'0','2');
> > > >> clear all
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > error: memory exhausted or requested size too large for range of Octave's index type -- trying to return to prompt
> > > ********************************************************************
> > >
> > >
> > > 2. octave-3.7.2+ VS*****************************
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > warning: range error for conversion to character value
> > > >> fclose(fid);
> > > >> s = regexprep(s,'0','2');
> > > >> clear all
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > error: out of memory or dimension too large for Octave's index type
> > > ********************************************************************
> > >
> > > 3.Octave-3.6.2 Cygwin**********************************************
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > >> fclose(fid);
> > > >> s = regexprep(s,'0','2');
> > > >> clear all
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > >> fclose(fid);
> > > >> s = regexprep(s,'0','2');
> > > ********************************************************************
> > >
> > > For me, both VS and MinGW version caused problem. Cygwin version seemed to work correctly.
> > >
> > > Regards
> > >
> > > Tatsuro
> >
> >
> > I have also tested on
> > octave-3.2.4 and 3.4.3 MinGW version.
> >
> > 4. octave-3.2.4 and 3.4.3 MinGW *****************************
> > >> fid = fopen("TestData.dat","r");
> > >> s = fscanf(fid,'%c');
> > warning: range error for conversion to character value
> > >> fclose(fid);
> > >> s = regexprep(s,'0','2');
> > >> clear all
> > >> fid = fopen("TestData.dat","r");
> > >> s = fscanf(fid,'%c');
> > warning: range error for conversion to character value
> > >> fclose(fid);
> > >> s = regexprep(s,'0','2');
> > **************************************************
> >
> > The problem seemed to be happened on octave-3.6 or later on native windows.
> >
> > Regards
> >
> > Tatsuro
>
> Sorry for successive post.
>
> I have tested the following  :
>
> %*********************************
> s="0123456789";
> for i=1:21, s=cstrcat(s,s);,endfor
> save TestData.dat s
> load TestData.dat
> s = regexprep(s,'0','2');
> clear all
> load TestData.dat
> s = regexprep(s,'0','2');
> %*********************************
>
> The above worked correctly on octave-3.2.4 (MinGW), 3.4.3 (MinGW), 3.6.2 (MinGW,VS, Cygwin) and 3.7.2+ (VS).
>
> I think that the problem does not rely on regexprep but on fscanf for large data.
>
> Regards
>
> Tatsuro
>
Thank you very much for the fast confirmation of the same problem. Actually it must be a strange combination of both I tested:

octave-3.7.2+ VS*****************************
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
clear all
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
clear all
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
clear all
********************************************************************

So it can't be fscanf only. On the other hand I tested as well (because your last example the second regexprep does nothing, as all 0 were already exchanged)

octave-3.7.2+ VS*****************************
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
s = regexprep(s,'0','2');
s = regexprep(s,'1','2');
s = regexprep(s,'2','3');
error: out of memory or dimension too large for Octave's index type
********************************************************************

So it clearly also depends on how many replacements are performed within the regexprep. (same fails as well in 3.6.0_gcc4.6.2 and 3.2.4_gcc4.4.0)
As something like this always works:

octave-3.7.2+ VS*****************************
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
s = ctsrcat(s,"aaaaaaaaa", s);
s = regexprep(s,'a','2');
clear all
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
clear all
********************************************************************

Kind regards
Christoph
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Aw: Re: memory leak in regexprep ? (windows/3.7.2+/VS)

tmacchant
--- On Wed, 2013/3/27, Christoph Ellenberger wrote:

> > --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
> > > --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
> > > > --- On Tue, 2013/3/26, Christoph Ellenberger  wrote:
> > > > > I am experiencing some strange behavior and wanted to check if it is  a bug or me....
> > > > > It happens when I try to read in and manipulating some ascii data files. It only happens with the VS version of octave. (tested with several gcc versions on windows and didn't showed up).
> > > > > Ok here is some example code to reproduce:
> > > > > First generating an ascii file:
> > > > > s="0123456789";
> > > > > for i=1:21, s=cstrcat(s,s);,endfor
> > > > > save TestData.dat s
> > > > >  
> > > > > Then readin and manipulate:
> > > > > fid = fopen("TestData.dat","r");
> > > > > s = fscanf(fid,'%c');
> > > > > fclose(fid);
> > > > > s = regexprep(s,'0','2');
> > > > >  
> > > > > Clear all and repeat:
> > > > > clear all
> > > > >
> > > > > fid = fopen("TestData.dat","r");
> > > > > s = fscanf(fid,'%c');
> > > > > fclose(fid);
> > > > > s = regexprep(s,'0','2');
> > > > >  
> > > > > Depending on datasize and how many replacement were performed it either fails on the new readin or regexprep with a: "error: out of memory or dimension too large for Octave's index type"
> > > > > Any help is appreciated
> > > > > Christoph
> > > >
> > > > I have tested on octave-3.6.2 VS, 3.6.2 MinGW, 3.7.2+ VS and 3.6.2 cygwin
> > > > (win 7 64 bit Home premium, 4GB memory)
> > > >
> > > >
> > > > 1. octave-3.6.2 VS and 3.6.2 MinGW*****************************
> > > > >> fid = fopen("TestData.dat","r");
> > > > >> s = fscanf(fid,'%c');
> > > > warning: range error for conversion to character value
> > > > >> fclose(fid);
> > > > >> s = regexprep(s,'0','2');
> > > > >> clear all
> > > > >> fid = fopen("TestData.dat","r");
> > > > >> s = fscanf(fid,'%c');
> > > > error: memory exhausted or requested size too large for range of Octave's index type -- trying to return to prompt
> > > > ********************************************************************
> > > >
> > > >
> > > > 2. octave-3.7.2+ VS*****************************
> > > > >> fid = fopen("TestData.dat","r");
> > > > >> s = fscanf(fid,'%c');
> > > > warning: range error for conversion to character value
> > > > >> fclose(fid);
> > > > >> s = regexprep(s,'0','2');
> > > > >> clear all
> > > > >> fid = fopen("TestData.dat","r");
> > > > >> s = fscanf(fid,'%c');
> > > > error: out of memory or dimension too large for Octave's index type
> > > > ********************************************************************
> > > >
> > > > 3.Octave-3.6.2 Cygwin**********************************************
> > > > >> fid = fopen("TestData.dat","r");
> > > > >> s = fscanf(fid,'%c');
> > > > >> fclose(fid);
> > > > >> s = regexprep(s,'0','2');
> > > > >> clear all
> > > > >> fid = fopen("TestData.dat","r");
> > > > >> s = fscanf(fid,'%c');
> > > > >> fclose(fid);
> > > > >> s = regexprep(s,'0','2');
> > > > ********************************************************************
> > > >
> > > > For me, both VS and MinGW version caused problem. Cygwin version seemed to work correctly.
> > > >
> > > > Regards
> > > >
> > > > Tatsuro
> > >
> > >
> > > I have also tested on
> > > octave-3.2.4 and 3.4.3 MinGW version.
> > >
> > > 4. octave-3.2.4 and 3.4.3 MinGW *****************************
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > warning: range error for conversion to character value
> > > >> fclose(fid);
> > > >> s = regexprep(s,'0','2');
> > > >> clear all
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > warning: range error for conversion to character value
> > > >> fclose(fid);
> > > >> s = regexprep(s,'0','2');
> > > **************************************************
> > >
> > > The problem seemed to be happened on octave-3.6 or later on native windows.
> > >
> > > Regards
> > >
> > > Tatsuro
> >
> > Sorry for successive post.
> >
> > I have tested the following  :
> >
> > %*********************************
> > s="0123456789";
> > for i=1:21, s=cstrcat(s,s);,endfor
> > save TestData.dat s
> > load TestData.dat
> > s = regexprep(s,'0','2');
> > clear all
> > load TestData.dat
> > s = regexprep(s,'0','2');
> > %*********************************
> >
> > The above worked correctly on octave-3.2.4 (MinGW), 3.4.3 (MinGW), 3.6.2 (MinGW,VS, Cygwin) and 3.7.2+ (VS).
> >
> > I think that the problem does not rely on regexprep but on fscanf for large data.
> >
> > Regards
> >
> > Tatsuro
> >
> Thank you very much for the fast confirmation of the same problem. Actually it must be a strange combination of both I tested:
>
> octave-3.7.2+ VS*****************************
> fid = fopen("TestData.dat","r");
> s = fscanf(fid,'%c');
> warning: range error for conversion to character value
> fclose(fid);
> clear all
> fid = fopen("TestData.dat","r");
> s = fscanf(fid,'%c');
> warning: range error for conversion to character value
> fclose(fid);
> clear all
> fid = fopen("TestData.dat","r");
> s = fscanf(fid,'%c');
> warning: range error for conversion to character value
> fclose(fid);
> clear all
> ********************************************************************
>
> So it can't be fscanf only. On the other hand I tested as well (because your last example the second regexprep does nothing, as all 0 were already exchanged)
>
> octave-3.7.2+ VS*****************************
> fid = fopen("TestData.dat","r");
> s = fscanf(fid,'%c');
> warning: range error for conversion to character value
> fclose(fid);
> s = regexprep(s,'0','2');
> s = regexprep(s,'1','2');
> s = regexprep(s,'2','3');
> error: out of memory or dimension too large for Octave's index type
> ********************************************************************
>
> So it clearly also depends on how many replacements are performed within the regexprep. (same fails as well in 3.6.0_gcc4.6.2 and 3.2.4_gcc4.4.0)
> As something like this always works:
>
> octave-3.7.2+ VS*****************************
> fid = fopen("TestData.dat","r");
> s = fscanf(fid,'%c');
> warning: range error for conversion to character value
> fclose(fid);
> s = ctsrcat(s,"aaaaaaaaa", s);
> s = regexprep(s,'a','2');
> clear all
> fid = fopen("TestData.dat","r");
> s = fscanf(fid,'%c');
> warning: range error for conversion to character value
> fclose(fid);
> clear all
> ********************************************************************
>
> Kind regards
> Christoph

You are right.
#**************************************
s="0123456789";
for i=1:21, s=cstrcat(s,s);,endfor

s = regexprep(s,'0','2');
s = regexprep(s,'1','2');
s = regexprep(s,'2','3');
#**************************************

The above caused 'out of memory ...' or 'memory exhausted ...' on octave 3.7.2+(VS),
3.6.2(VS, MinGW), 3.4.3 (MinGW) and 3.2.4 (MinGW).

But for cygwin version octave(3.6.2), the above test did not cause memory error.

I think that it is better that you resist this strange behavior of regexprep as a bug to the bug tracker.

Regards

Tatsuro
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Aw: Re: memory leak in regexprep ? (windows/3.7.2+/VS)

Sergei Steshenko




----- Original Message -----

> From: Tatsuro MATSUOKA <[hidden email]>
> To: Christoph Ellenberger <[hidden email]>
> Cc: [hidden email]
> Sent: Thursday, March 28, 2013 4:07 AM
> Subject: Re: Aw: Re: memory leak in regexprep ? (windows/3.7.2+/VS)
>
> --- On Wed, 2013/3/27, Christoph Ellenberger wrote:
>>  > --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
>>  > > --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
>>  > > > --- On Tue, 2013/3/26, Christoph Ellenberger  wrote:
>>  > > > > I am experiencing some strange behavior and wanted to
> check if it is  a bug or me....
>>  > > > > It happens when I try to read in and manipulating some
> ascii data files. It only happens with the VS version of octave. (tested with
> several gcc versions on windows and didn't showed up).
>>  > > > > Ok here is some example code to reproduce:
>>  > > > > First generating an ascii file:
>>  > > > > s="0123456789";
>>  > > > > for i=1:21, s=cstrcat(s,s);,endfor
>>  > > > > save TestData.dat s
>>  > > > >  
>>  > > > > Then readin and manipulate:
>>  > > > > fid = fopen("TestData.dat","r");
>>  > > > > s = fscanf(fid,'%c');
>>  > > > > fclose(fid);
>>  > > > > s = regexprep(s,'0','2');
>>  > > > >  
>>  > > > > Clear all and repeat:
>>  > > > > clear all
>>  > > > >
>>  > > > > fid = fopen("TestData.dat","r");
>>  > > > > s = fscanf(fid,'%c');
>>  > > > > fclose(fid);
>>  > > > > s = regexprep(s,'0','2');
>>  > > > >  
>>  > > > > Depending on datasize and how many replacement were
> performed it either fails on the new readin or regexprep with a: "error:
> out of memory or dimension too large for Octave's index type"
>>  > > > > Any help is appreciated
>>  > > > > Christoph
>>  > > >
>>  > > > I have tested on octave-3.6.2 VS, 3.6.2 MinGW, 3.7.2+ VS and
> 3.6.2 cygwin
>>  > > > (win 7 64 bit Home premium, 4GB memory)
>>  > > >
>>  > > >
>>  > > > 1. octave-3.6.2 VS and 3.6.2
> MinGW*****************************
>>  > > > >> fid =
> fopen("TestData.dat","r");
>>  > > > >> s = fscanf(fid,'%c');
>>  > > > warning: range error for conversion to character value
>>  > > > >> fclose(fid);
>>  > > > >> s = regexprep(s,'0','2');
>>  > > > >> clear all
>>  > > > >> fid =
> fopen("TestData.dat","r");
>>  > > > >> s = fscanf(fid,'%c');
>>  > > > error: memory exhausted or requested size too large for
> range of Octave's index type -- trying to return to prompt
>>  > > >
> ********************************************************************
>>  > > >
>>  > > >
>>  > > > 2. octave-3.7.2+ VS*****************************
>>  > > > >> fid =
> fopen("TestData.dat","r");
>>  > > > >> s = fscanf(fid,'%c');
>>  > > > warning: range error for conversion to character value
>>  > > > >> fclose(fid);
>>  > > > >> s = regexprep(s,'0','2');
>>  > > > >> clear all
>>  > > > >> fid =
> fopen("TestData.dat","r");
>>  > > > >> s = fscanf(fid,'%c');
>>  > > > error: out of memory or dimension too large for Octave's
> index type
>>  > > >
> ********************************************************************
>>  > > >
>>  > > > 3.Octave-3.6.2
> Cygwin**********************************************
>>  > > > >> fid =
> fopen("TestData.dat","r");
>>  > > > >> s = fscanf(fid,'%c');
>>  > > > >> fclose(fid);
>>  > > > >> s = regexprep(s,'0','2');
>>  > > > >> clear all
>>  > > > >> fid =
> fopen("TestData.dat","r");
>>  > > > >> s = fscanf(fid,'%c');
>>  > > > >> fclose(fid);
>>  > > > >> s = regexprep(s,'0','2');
>>  > > >
> ********************************************************************
>>  > > >
>>  > > > For me, both VS and MinGW version caused problem. Cygwin
> version seemed to work correctly.
>>  > > >
>>  > > > Regards
>>  > > >
>>  > > > Tatsuro
>>  > >
>>  > >
>>  > > I have also tested on
>>  > > octave-3.2.4 and 3.4.3 MinGW version.
>>  > >
>>  > > 4. octave-3.2.4 and 3.4.3 MinGW *****************************
>>  > > >> fid = fopen("TestData.dat","r");
>>  > > >> s = fscanf(fid,'%c');
>>  > > warning: range error for conversion to character value
>>  > > >> fclose(fid);
>>  > > >> s = regexprep(s,'0','2');
>>  > > >> clear all
>>  > > >> fid = fopen("TestData.dat","r");
>>  > > >> s = fscanf(fid,'%c');
>>  > > warning: range error for conversion to character value
>>  > > >> fclose(fid);
>>  > > >> s = regexprep(s,'0','2');
>>  > > **************************************************
>>  > >
>>  > > The problem seemed to be happened on octave-3.6 or later on
> native windows.
>>  > >
>>  > > Regards
>>  > >
>>  > > Tatsuro
>>  >
>>  > Sorry for successive post.
>>  >
>>  > I have tested the following  :
>>  >
>>  > %*********************************
>>  > s="0123456789";
>>  > for i=1:21, s=cstrcat(s,s);,endfor
>>  > save TestData.dat s
>>  > load TestData.dat
>>  > s = regexprep(s,'0','2');
>>  > clear all
>>  > load TestData.dat
>>  > s = regexprep(s,'0','2');
>>  > %*********************************
>>  >
>>  > The above worked correctly on octave-3.2.4 (MinGW), 3.4.3 (MinGW),
> 3.6.2 (MinGW,VS, Cygwin) and 3.7.2+ (VS).
>>  >
>>  > I think that the problem does not rely on regexprep but on fscanf for
> large data.
>>  >
>>  > Regards
>>  >
>>  > Tatsuro
>>  >
>>  Thank you very much for the fast confirmation of the same problem. Actually
> it must be a strange combination of both I tested:
>>
>>  octave-3.7.2+ VS*****************************
>>  fid = fopen("TestData.dat","r");
>>  s = fscanf(fid,'%c');
>>  warning: range error for conversion to character value
>>  fclose(fid);
>>  clear all
>>  fid = fopen("TestData.dat","r");
>>  s = fscanf(fid,'%c');
>>  warning: range error for conversion to character value
>>  fclose(fid);
>>  clear all
>>  fid = fopen("TestData.dat","r");
>>  s = fscanf(fid,'%c');
>>  warning: range error for conversion to character value
>>  fclose(fid);
>>  clear all
>>  ********************************************************************
>>
>>  So it can't be fscanf only. On the other hand I tested as well (because
> your last example the second regexprep does nothing, as all 0 were already
> exchanged)
>>
>>  octave-3.7.2+ VS*****************************
>>  fid = fopen("TestData.dat","r");
>>  s = fscanf(fid,'%c');
>>  warning: range error for conversion to character value
>>  fclose(fid);
>>  s = regexprep(s,'0','2');
>>  s = regexprep(s,'1','2');
>>  s = regexprep(s,'2','3');
>>  error: out of memory or dimension too large for Octave's index type
>>  ********************************************************************
>>
>>  So it clearly also depends on how many replacements are performed within
> the regexprep. (same fails as well in 3.6.0_gcc4.6.2 and 3.2.4_gcc4.4.0)
>>  As something like this always works:
>>
>>  octave-3.7.2+ VS*****************************
>>  fid = fopen("TestData.dat","r");
>>  s = fscanf(fid,'%c');
>>  warning: range error for conversion to character value
>>  fclose(fid);
>>  s = ctsrcat(s,"aaaaaaaaa", s);
>>  s = regexprep(s,'a','2');
>>  clear all
>>  fid = fopen("TestData.dat","r");
>>  s = fscanf(fid,'%c');
>>  warning: range error for conversion to character value
>>  fclose(fid);
>>  clear all
>>  ********************************************************************
>>
>>  Kind regards
>>  Christoph
>
> You are right.
> #**************************************
> s="0123456789";
> for i=1:21, s=cstrcat(s,s);,endfor
>
> s = regexprep(s,'0','2');
> s = regexprep(s,'1','2');
> s = regexprep(s,'2','3');
> #**************************************
>
> The above caused 'out of memory ...' or 'memory exhausted ...'
> on octave 3.7.2+(VS),
> 3.6.2(VS, MinGW), 3.4.3 (MinGW) and 3.2.4 (MinGW).
>
> But for cygwin version octave(3.6.2), the above test did not cause memory error.
>
> I think that it is better that you resist this strange behavior of regexprep as
> a bug to the bug tracker.
>
> Regards
>
> Tatsuro
> _______________________________________________


I've tried the above code on self-built octave-3.6.4 under Linux.

Memory consumption constantly grows, I killed the session when memory consumption reached 2.5G.

I think it's a bug, and since I can reproduce the failure under Linux, the bug is not OS-specific.

Regards,
  Sergei.

_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Aw: Re: Re: memory leak in regexprep ? (windows/3.7.2+/VS)

Christoph Ellenberger-2
Ok thanks for testing, I filed a bug (#38616)
christoph

> Gesendet: Donnerstag, 28. März 2013 um 04:05 Uhr
> Von: "Sergei Steshenko" <[hidden email]>
> An: "[hidden email]" <[hidden email]>, "Christoph Ellenberger" <[hidden email]>
> Cc: "[hidden email]" <[hidden email]>
> Betreff: Re: Aw: Re: memory leak in regexprep ? (windows/3.7.2+/VS)
>
>
>
>
>
> ----- Original Message -----
> > From: Tatsuro MATSUOKA <[hidden email]>
> > To: Christoph Ellenberger <[hidden email]>
> > Cc: [hidden email]
> > Sent: Thursday, March 28, 2013 4:07 AM
> > Subject: Re: Aw: Re: memory leak in regexprep ? (windows/3.7.2+/VS)
> >
> > --- On Wed, 2013/3/27, Christoph Ellenberger wrote:
> >>  > --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
> >>  > > --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
> >>  > > > --- On Tue, 2013/3/26, Christoph Ellenberger  wrote:
> >>  > > > > I am experiencing some strange behavior and wanted to
> > check if it is  a bug or me....
> >>  > > > > It happens when I try to read in and manipulating some
> > ascii data files. It only happens with the VS version of octave. (tested with
> > several gcc versions on windows and didn't showed up).
> >>  > > > > Ok here is some example code to reproduce:
> >>  > > > > First generating an ascii file:
> >>  > > > > s="0123456789";
> >>  > > > > for i=1:21, s=cstrcat(s,s);,endfor
> >>  > > > > save TestData.dat s
> >>  > > > >  
> >>  > > > > Then readin and manipulate:
> >>  > > > > fid = fopen("TestData.dat","r");
> >>  > > > > s = fscanf(fid,'%c');
> >>  > > > > fclose(fid);
> >>  > > > > s = regexprep(s,'0','2');
> >>  > > > >  
> >>  > > > > Clear all and repeat:
> >>  > > > > clear all
> >>  > > > >
> >>  > > > > fid = fopen("TestData.dat","r");
> >>  > > > > s = fscanf(fid,'%c');
> >>  > > > > fclose(fid);
> >>  > > > > s = regexprep(s,'0','2');
> >>  > > > >  
> >>  > > > > Depending on datasize and how many replacement were
> > performed it either fails on the new readin or regexprep with a: "error:
> > out of memory or dimension too large for Octave's index type"
> >>  > > > > Any help is appreciated
> >>  > > > > Christoph
> >>  > > >
> >>  > > > I have tested on octave-3.6.2 VS, 3.6.2 MinGW, 3.7.2+ VS and
> > 3.6.2 cygwin
> >>  > > > (win 7 64 bit Home premium, 4GB memory)
> >>  > > >
> >>  > > >
> >>  > > > 1. octave-3.6.2 VS and 3.6.2
> > MinGW*****************************
> >>  > > > >> fid =
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > warning: range error for conversion to character value
> >>  > > > >> fclose(fid);
> >>  > > > >> s = regexprep(s,'0','2');
> >>  > > > >> clear all
> >>  > > > >> fid =
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > error: memory exhausted or requested size too large for
> > range of Octave's index type -- trying to return to prompt
> >>  > > >
> > ********************************************************************
> >>  > > >
> >>  > > >
> >>  > > > 2. octave-3.7.2+ VS*****************************
> >>  > > > >> fid =
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > warning: range error for conversion to character value
> >>  > > > >> fclose(fid);
> >>  > > > >> s = regexprep(s,'0','2');
> >>  > > > >> clear all
> >>  > > > >> fid =
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > error: out of memory or dimension too large for Octave's
> > index type
> >>  > > >
> > ********************************************************************
> >>  > > >
> >>  > > > 3.Octave-3.6.2
> > Cygwin**********************************************
> >>  > > > >> fid =
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > >> fclose(fid);
> >>  > > > >> s = regexprep(s,'0','2');
> >>  > > > >> clear all
> >>  > > > >> fid =
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > >> fclose(fid);
> >>  > > > >> s = regexprep(s,'0','2');
> >>  > > >
> > ********************************************************************
> >>  > > >
> >>  > > > For me, both VS and MinGW version caused problem. Cygwin
> > version seemed to work correctly.
> >>  > > >
> >>  > > > Regards
> >>  > > >
> >>  > > > Tatsuro
> >>  > >
> >>  > >
> >>  > > I have also tested on
> >>  > > octave-3.2.4 and 3.4.3 MinGW version.
> >>  > >
> >>  > > 4. octave-3.2.4 and 3.4.3 MinGW *****************************
> >>  > > >> fid = fopen("TestData.dat","r");
> >>  > > >> s = fscanf(fid,'%c');
> >>  > > warning: range error for conversion to character value
> >>  > > >> fclose(fid);
> >>  > > >> s = regexprep(s,'0','2');
> >>  > > >> clear all
> >>  > > >> fid = fopen("TestData.dat","r");
> >>  > > >> s = fscanf(fid,'%c');
> >>  > > warning: range error for conversion to character value
> >>  > > >> fclose(fid);
> >>  > > >> s = regexprep(s,'0','2');
> >>  > > **************************************************
> >>  > >
> >>  > > The problem seemed to be happened on octave-3.6 or later on
> > native windows.
> >>  > >
> >>  > > Regards
> >>  > >
> >>  > > Tatsuro
> >>  >
> >>  > Sorry for successive post.
> >>  >
> >>  > I have tested the following  :
> >>  >
> >>  > %*********************************
> >>  > s="0123456789";
> >>  > for i=1:21, s=cstrcat(s,s);,endfor
> >>  > save TestData.dat s
> >>  > load TestData.dat
> >>  > s = regexprep(s,'0','2');
> >>  > clear all
> >>  > load TestData.dat
> >>  > s = regexprep(s,'0','2');
> >>  > %*********************************
> >>  >
> >>  > The above worked correctly on octave-3.2.4 (MinGW), 3.4.3 (MinGW),
> > 3.6.2 (MinGW,VS, Cygwin) and 3.7.2+ (VS).
> >>  >
> >>  > I think that the problem does not rely on regexprep but on fscanf for
> > large data.
> >>  >
> >>  > Regards
> >>  >
> >>  > Tatsuro
> >>  >
> >>  Thank you very much for the fast confirmation of the same problem. Actually
> > it must be a strange combination of both I tested:
> >>
> >>  octave-3.7.2+ VS*****************************
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  clear all
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  clear all
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  clear all
> >>  ********************************************************************
> >>
> >>  So it can't be fscanf only. On the other hand I tested as well (because
> > your last example the second regexprep does nothing, as all 0 were already
> > exchanged)
> >>
> >>  octave-3.7.2+ VS*****************************
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  s = regexprep(s,'0','2');
> >>  s = regexprep(s,'1','2');
> >>  s = regexprep(s,'2','3');
> >>  error: out of memory or dimension too large for Octave's index type
> >>  ********************************************************************
> >>
> >>  So it clearly also depends on how many replacements are performed within
> > the regexprep. (same fails as well in 3.6.0_gcc4.6.2 and 3.2.4_gcc4.4.0)
> >>  As something like this always works:
> >>
> >>  octave-3.7.2+ VS*****************************
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  s = ctsrcat(s,"aaaaaaaaa", s);
> >>  s = regexprep(s,'a','2');
> >>  clear all
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  clear all
> >>  ********************************************************************
> >>
> >>  Kind regards
> >>  Christoph
> >
> > You are right.
> > #**************************************
> > s="0123456789";
> > for i=1:21, s=cstrcat(s,s);,endfor
> >
> > s = regexprep(s,'0','2');
> > s = regexprep(s,'1','2');
> > s = regexprep(s,'2','3');
> > #**************************************
> >
> > The above caused 'out of memory ...' or 'memory exhausted ...'
> > on octave 3.7.2+(VS),
> > 3.6.2(VS, MinGW), 3.4.3 (MinGW) and 3.2.4 (MinGW).
> >
> > But for cygwin version octave(3.6.2), the above test did not cause memory error.
> >
> > I think that it is better that you resist this strange behavior of regexprep as
> > a bug to the bug tracker.
> >
> > Regards
> >
> > Tatsuro
> > _______________________________________________
>
>
> I've tried the above code on self-built octave-3.6.4 under Linux.
>
> Memory consumption constantly grows, I killed the session when memory consumption reached 2.5G.
>
> I think it's a bug, and since I can reproduce the failure under Linux, the bug is not OS-specific.
>
> Regards,
>   Sergei.
>
>
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

memory leak in regexprep ? (windows/3.7.2+/VS)

Paryshaan
I have this problem too, but i don't used regexprep in my code.
I use octave version is 3.7.2+ VS in win vista 32 bit with 2 GB of RAM.
At the error time taskmanager shows that octave uses about 127,676 K of memory.

The error is:
error: out of memory or dimension too large for Octave's index type
and refer to a line which is set to create a new variable  with 28*187079 elements.

this is my workspace when error occurs(variable names are simplified):

>> whos
Variables in the current scope:

Attr Name Size                     Bytes  Class
==== ==== ====                     =====  =====
        a 1x7                          7  char
        b 1x1                          8  double
        c 20x1                  110225280  cell
        d 28x1                        112  double
        e 1x20                        20  char
        f 9x2x10                    1440  double
        g 1x1                          8  double
        h 1x1                          8  double
        i 1x70                        70  char
        j 1x1                          8  double
        k 1x1                        152  struct
        l 9x2x10                    1440  double
        m 1x1                          8  double
        n 1x1                          8  double
        o 1x1                       1408  struct
        p 1x1                       1408  struct
        q 1x1                          8  double
        r 1000x4                      32000  double
        s 1000x4                      32000  double
        t 10x1                      60256  cell
        u 10x1                      60256  cell
        v 1000x5                      40000  double
        w 1x8                          8  char
        x 28x1                        224  double
        y 1x1                          8  double
        z 1x20                1543118080  cell
        z1 28x1                        224  double
        z2 1x1                          8  double
        z3 9x9                        648  double
        z4 1x1                          8  double
        z5 1x1                          8  double
        z6 1x1                          8  double
        z7 187070x10                  14965600  double
        z8 10x2                        160  double
        z9 1x10                        80  double
        z10 1x1                          8  double
        z11 1x1                          8  double

Total is 1884437 elements using 1668540985 bytes

Any helps will be appreciated. Thanks in advance.
Paryshaan
Loading...