Directory Listing in Octave

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Directory Listing in Octave

Greg_G
Hello,

This should be an easy question but the answer is eluding me.  I am running Windows XP Octave-3.2.3-3_i686-pc-mingw32.

I am trying to create a cell array of file names of a specific extension from a tree of directories.  Each of the files in the list will later be opened and scanned for data.  
I would like to use one “glob” or “dir” or “ls” command to search all levels of the tree for the file type.  

Currently, I am using "glob" but I have to specify a pattern for each hierarchical level and I have to know how many levels there are.  Is there a way to search all of the unkown number of levels of subdirectories?
   
The “dir /b /s” and “ls –R” commands look like they should work but I have not been able to get what I want using the system("dir /b /s") command.  

This is my current code:

Data_dir= “X:\\Test_stand\\2009”      % Base directory
Pat1= {[Data_dir,”/*/*.dat”]};          % pattern for 1st level
Pat2= {[Data_dir,”/*/*/*.dat”]};       % pattern for 2nd level
Pat3= {[Data_dir,”/*/*/*/*.dat”]};
ZZ= glob({pat1,pat2,pat3});


Regards,
Greg


Reply | Threaded
Open this post in threaded view
|

Re: Directory Listing in Octave -- Recursion

Greg_G
Here is a Windows-specific method of recursion using "system" and the "dir" command:

patdir= (['dir /b /s "',<directory>,'\*.dat"']);
[status data] = system ( patdir, 1);

Unfortunately, the string variable "data" is a (1 x N) char array and has to be split into strings based on the end-of-line character "\n".

Is there anything a little cleaner?



Greg_G wrote
Hello,

This should be an easy question but the answer is eluding me.  I am running Windows XP Octave-3.2.3-3_i686-pc-mingw32.

I am trying to create a cell array of file names of a specific extension from a tree of directories.  Each of the files in the list will later be opened and scanned for data.  
I would like to use one “glob” or “dir” or “ls” command to search all levels of the tree for the file type.  

Currently, I am using "glob" but I have to specify a pattern for each hierarchical level and I have to know how many levels there are.  Is there a way to search all of the unkown number of levels of subdirectories?
   
The “dir /b /s” and “ls –R” commands look like they should work but I have not been able to get what I want using the system("dir /b /s") command.  

This is my current code:

Data_dir= “X:\\Test_stand\\2009”      % Base directory
Pat1= {[Data_dir,”/*/*.dat”]};          % pattern for 1st level
Pat2= {[Data_dir,”/*/*/*.dat”]};       % pattern for 2nd level
Pat3= {[Data_dir,”/*/*/*/*.dat”]};
ZZ= glob({pat1,pat2,pat3});


Regards,
Greg

Reply | Threaded
Open this post in threaded view
|

RE: Directory Listing in Octave -- Recursion

HALL, BENJAMIN            PW
>-----Original Message-----
>From: Greg_G [mailto:[hidden email]]
>Sent: Wednesday, December 09, 2009 11:26 AM
>To: [hidden email]
>Subject: Re: Directory Listing in Octave -- Recursion
>
>
>Here is a Windows-specific method of recursion using "system" and the
"dir"
>command:
>
>patdir= (['dir /b /s "',<directory>,'\*.dat"']);
>[status data] = system ( patdir, 1);
>
>Unfortunately, the string variable "data" is a (1 x N) char array and
has to
>be split into strings based on the end-of-line character "\n".
>
>Is there anything a little cleaner?

Here is something I found in my files...

## Windows versions of octave come packaged with the find utility
## as provided by MSYS.  This is the found in the path before the
## Windows built-in (and worthless) find utility.  So, it turns
## out to be easy...

[stat,lsout] = system( sprintf("find %s -name '*.cmp'",cpath), 1, "sync"
);

fls = cellstr( split( lsout, "\n" ) );


_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

RE: Directory Listing in Octave -- Recursion

John W. Eaton
Administrator
On  9-Dec-2009, HALL, BENJAMIN            PW wrote:

| Here is something I found in my files...
|
| ## Windows versions of octave come packaged with the find utility
| ## as provided by MSYS.  This is the found in the path before the
| ## Windows built-in (and worthless) find utility.  So, it turns
| ## out to be easy...
|
| [stat,lsout] = system( sprintf("find %s -name '*.cmp'",cpath), 1, "sync"
| );
|
| fls = cellstr( split( lsout, "\n" ) );

Here is a solution that does not require external utilties:

  function retval = rdir (d)
    ## info about D
    d_info = dir (d);
    ## list of directories in D, not including . and ..
    names = {d_info.name};
    subdirs = names([d_info.isdir]);
    subdirs = subdirs(3:end);
    ## adjust filenames to include directory info
    for j = 1:numel (d_info)
      d_info(j).name = fullfile (d, d_info(j).name);
    endfor
    retval = d_info;
    for i = 1:numel (subdirs)
      ## recurse and append results for subdirectories to the list
      d_info = rdir (fullfile (d, subdirs{i}));
      retval = [retval; d_info];
    endfor
  endfunction

However, this solution is fairly slow for large directories at least
in part because Octave's dir function is also an interpreted
function.  It might be good to convert that function to C++.

Also, this function assumes that the first two entries returned from
the dir function are "." and "..".  Is that always true on Windows
systems as well?

jwe
_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: Directory Listing in Octave -- Recursion

ellocco
I found also a solution here:
rdir by Gus Brow.
I have not tried another variant:
by  Core.

I modified the one from Gus Browto make it compatible to Octave.
Matlab does not need this function any longer, as the "**" syntax is since Matlab v2016b integrated into dir().

Attached the version that I use.

Regards,

Stefan

Am Mi., 9. Dez. 2009 um 19:33 Uhr schrieb John W. Eaton <[hidden email]>:
On  9-Dec-2009, HALL, BENJAMIN            PW wrote:

| Here is something I found in my files...
|
| ## Windows versions of octave come packaged with the find utility
| ## as provided by MSYS.  This is the found in the path before the
| ## Windows built-in (and worthless) find utility.  So, it turns
| ## out to be easy...
|
| [stat,lsout] = system( sprintf("find %s -name '*.cmp'",cpath), 1, "sync"
| );
|
| fls = cellstr( split( lsout, "\n" ) );

Here is a solution that does not require external utilties:

  function retval = rdir (d)
    ## info about D
    d_info = dir (d);
    ## list of directories in D, not including . and ..
    names = {d_info.name};
    subdirs = names([d_info.isdir]);
    subdirs = subdirs(3:end);
    ## adjust filenames to include directory info
    for j = 1:numel (d_info)
      d_info(j).name = fullfile (d, d_info(j).name);
    endfor
    retval = d_info;
    for i = 1:numel (subdirs)
      ## recurse and append results for subdirectories to the list
      d_info = rdir (fullfile (d, subdirs{i}));
      retval = [retval; d_info];
    endfor
  endfunction

However, this solution is fairly slow for large directories at least
in part because Octave's dir function is also an interpreted
function.  It might be good to convert that function to C++.

Also, this function assumes that the first two entries returned from
the dir function are "." and "..".  Is that always true on Windows
systems as well?

jwe
_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave


--
Stefan Pofahl
Zollgasse 5
8020 Graz
Österreich
Tel.: +43 (316) 33 2001



MyLib_rdir.m (18K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Directory Listing in Octave -- Recursion

mmuetzel
Am 17. Juni 2020 um 15:53 Uhr schrieb "Stefan Pofahl":

> I found also a solution here:
> https://www.mathworks.com/matlabcentral/fileexchange/19550-recursive-directory-listing
> rdir by Gus Brow.[https://www.mathworks.com/matlabcentral/profile/authors/1231514-gus-brown]
> I have not tried another variant:
> https://www.mathworks.com/matlabcentral/fileexchange/47125-rdir-m[https://www.mathworks.com/matlabcentral/fileexchange/47125-rdir-m]
> by  Core.[https://www.mathworks.com/matlabcentral/profile/authors/4964828-corey]

> I modified the one from Gus Browto make it compatible to Octave.
> Matlab does not need this function any longer, as the "**" syntax is since Matlab v2016b integrated into dir().

> Attached the version that I use.

Please, use bottom posting on this list.

The license agreement of The Mathworks' FileExchange doesn't permit using those functions (or work derived of them) with other software than their own.
You should have received a copy of that license when downloading the files from the FileExchange:

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in
the documentation and/or other materials provided with the distribution.
* In all cases, the software is, and all modifications and derivatives
of the software shall be, licensed to you solely for use in conjunction
with MathWorks products and service offerings.


Please, do not distribute such files on the Octave mailing list.

Markus



Reply | Threaded
Open this post in threaded view
|

Re: Directory Listing in Octave -- Recursion

Andrew Janke-2


On 6/17/20 10:15 AM, Markus Mützel wrote:

> Am 17. Juni 2020 um 15:53 Uhr schrieb "Stefan Pofahl":
>> I found also a solution here:
>> https://www.mathworks.com/matlabcentral/fileexchange/19550-recursive-directory-listing
>> rdir by Gus Brow.[https://www.mathworks.com/matlabcentral/profile/authors/1231514-gus-brown]
>> I have not tried another variant:
>> https://www.mathworks.com/matlabcentral/fileexchange/47125-rdir-m[https://www.mathworks.com/matlabcentral/fileexchange/47125-rdir-m]
>> by  Core.[https://www.mathworks.com/matlabcentral/profile/authors/4964828-corey]
>>  
>> I modified the one from Gus Browto make it compatible to Octave.
>> Matlab does not need this function any longer, as the "**" syntax is since Matlab v2016b integrated into dir().
>>  
>> Attached the version that I use.
>>  
>
> Please, use bottom posting on this list.
>
> The license agreement of The Mathworks' FileExchange doesn't permit using those functions (or work derived of them) with other software than their own.
> You should have received a copy of that license when downloading the files from the FileExchange:
>
> Redistribution and use in source and binary forms, with or without
> modification, are permitted provided that the following conditions are
> met:
>
> * Redistributions of source code must retain the above copyright
> notice, this list of conditions and the following disclaimer.
> * Redistributions in binary form must reproduce the above copyright
> notice, this list of conditions and the following disclaimer in
> the documentation and/or other materials provided with the distribution.
> * In all cases, the software is, and all modifications and derivatives
> of the software shall be, licensed to you solely for use in conjunction
> with MathWorks products and service offerings.
>
>
> Please, do not distribute such files on the Octave mailing list.
>
> Markus

I'm not sure that this is true any more. MathWorks File Exchange has
switched to using the BSD 3-Clause license for all programs posted
there. So it may depend on exactly when the file was published.

The license for this particular posting appears to be BSD 2-Clause, not
the variant license with the "MathWorks productions only" clause.

Cheers,
Andrew


Reply | Threaded
Open this post in threaded view
|

Re: Directory Listing in Octave -- Recursion

siko1056
On 6/18/20 12:19 AM, Andrew Janke wrote:

>
>
> On 6/17/20 10:15 AM, Markus Mützel wrote:
>> Am 17. Juni 2020 um 15:53 Uhr schrieb "Stefan Pofahl":
>>> I found also a solution here:
>>> https://www.mathworks.com/matlabcentral/fileexchange/19550-recursive-directory-listing
>>> rdir by Gus Brow.[https://www.mathworks.com/matlabcentral/profile/authors/1231514-gus-brown]
>>> I have not tried another variant:
>>> https://www.mathworks.com/matlabcentral/fileexchange/47125-rdir-m[https://www.mathworks.com/matlabcentral/fileexchange/47125-rdir-m]
>>> by  Core.[https://www.mathworks.com/matlabcentral/profile/authors/4964828-corey]
>>>  
>>> I modified the one from Gus Browto make it compatible to Octave.
>>> Matlab does not need this function any longer, as the "**" syntax is since Matlab v2016b integrated into dir().
>>>  
>>> Attached the version that I use.
>>>  
>>
>> Please, use bottom posting on this list.
>>
>> The license agreement of The Mathworks' FileExchange doesn't permit using those functions (or work derived of them) with other software than their own.
>> You should have received a copy of that license when downloading the files from the FileExchange:
>>
>> Redistribution and use in source and binary forms, with or without
>> modification, are permitted provided that the following conditions are
>> met:
>>
>> * Redistributions of source code must retain the above copyright
>> notice, this list of conditions and the following disclaimer.
>> * Redistributions in binary form must reproduce the above copyright
>> notice, this list of conditions and the following disclaimer in
>> the documentation and/or other materials provided with the distribution.
>> * In all cases, the software is, and all modifications and derivatives
>> of the software shall be, licensed to you solely for use in conjunction
>> with MathWorks products and service offerings.
>>
>>
>> Please, do not distribute such files on the Octave mailing list.
>>
>> Markus
>
> I'm not sure that this is true any more. MathWorks File Exchange has
> switched to using the BSD 3-Clause license for all programs posted
> there. So it may depend on exactly when the file was published.
>
> The license for this particular posting appears to be BSD 2-Clause, not
> the variant license with the "MathWorks productions only" clause.
>
> Cheers,
> Andrew
>


@Markus, where did you get this text from, especially the third * looks
interesting precise?  For me it was not contained in the download.

@Andrew, nevertheless, I think The Mathworks has not changed it's terms
of use since 2016 .  There they are very explicit about the usage of
"their" hosted code.

Kai

[1]
https://wiki.octave.org/FAQ#Why_can.27t_I_use_code_from_File_Exchange_in_Octave.3F


Reply | Threaded
Open this post in threaded view
|

Re: Directory Listing in Octave -- Recursion

apjanke-floss


On 6/17/20 12:00 PM, Kai Torben Ohlhus wrote:

> On 6/18/20 12:19 AM, Andrew Janke wrote:
>>
>>
>> On 6/17/20 10:15 AM, Markus Mützel wrote:
>>>
>>> The license agreement of The Mathworks' FileExchange doesn't permit using those functions (or work derived of them) with other software than their own.
>>> You should have received a copy of that license when downloading the files from the FileExchange:
>>>
>>> Redistribution and use in source and binary forms, with or without
>>> modification, are permitted provided that the following conditions are
>>> met:
>>>
>>> * Redistributions of source code must retain the above copyright
>>> notice, this list of conditions and the following disclaimer.
>>> * Redistributions in binary form must reproduce the above copyright
>>> notice, this list of conditions and the following disclaimer in
>>> the documentation and/or other materials provided with the distribution.
>>> * In all cases, the software is, and all modifications and derivatives
>>> of the software shall be, licensed to you solely for use in conjunction
>>> with MathWorks products and service offerings.
>>>
>>>
>>> Please, do not distribute such files on the Octave mailing list.
>>>
>>> Markus
>>
>> I'm not sure that this is true any more. MathWorks File Exchange has
>> switched to using the BSD 3-Clause license for all programs posted
>> there. So it may depend on exactly when the file was published.
>>
>> The license for this particular posting appears to be BSD 2-Clause, not
>> the variant license with the "MathWorks productions only" clause.
>>
>> Cheers,
>> Andrew
>>
>
>
> @Markus, where did you get this text from, especially the third * looks
> interesting precise?  For me it was not contained in the download.
>
> @Andrew, nevertheless, I think The Mathworks has not changed it's terms
> of use since 2016 .  There they are very explicit about the usage of
> "their" hosted code.
>
> Kai
>
> [1]
> https://wiki.octave.org/FAQ#Why_can.27t_I_use_code_from_File_Exchange_in_Octave.3F
>

jwe found that "only witbh MathWorks products" thing for me: It's in
section 2.iii of the Matlab Central Terms of use [1].

Sounds like the site terms of use haven't caught up with the recent File
Exchange migration to BSD licensing. But I'm guessing Kai's right and we
should probably avoid redistributing any File Exchange material here as
long as there's any ambiguity.

Cheers,
Andrew

[1]
https://www.mathworks.com/matlabcentral/termsofuse.html


Reply | Threaded
Open this post in threaded view
|

Re: Directory Listing in Octave -- Recursion

mmuetzel
Am 17. Juni 2020 um 18:25 Uhr schrieb "Andrew Janke":

> On 6/17/20 12:00 PM, Kai Torben Ohlhus wrote:
> > On 6/18/20 12:19 AM, Andrew Janke wrote:
> >>
> >>
> >> On 6/17/20 10:15 AM, Markus Mützel wrote:
> >>>
> >>> The license agreement of The Mathworks' FileExchange doesn't permit using those functions (or work derived of them) with other software than their own.
> >>> You should have received a copy of that license when downloading the files from the FileExchange:
> >>>
> >>> Redistribution and use in source and binary forms, with or without
> >>> modification, are permitted provided that the following conditions are
> >>> met:
> >>>
> >>> * Redistributions of source code must retain the above copyright
> >>> notice, this list of conditions and the following disclaimer.
> >>> * Redistributions in binary form must reproduce the above copyright
> >>> notice, this list of conditions and the following disclaimer in
> >>> the documentation and/or other materials provided with the distribution.
> >>> * In all cases, the software is, and all modifications and derivatives
> >>> of the software shall be, licensed to you solely for use in conjunction
> >>> with MathWorks products and service offerings.
> >>>
> >>>
> >>> Please, do not distribute such files on the Octave mailing list.
> >>>
> >>> Markus
> >>
> >> I'm not sure that this is true any more. MathWorks File Exchange has
> >> switched to using the BSD 3-Clause license for all programs posted
> >> there. So it may depend on exactly when the file was published.
> >>
> >> The license for this particular posting appears to be BSD 2-Clause, not
> >> the variant license with the "MathWorks productions only" clause.
> >>
> >> Cheers,
> >> Andrew
> >>
> >
> >
> > @Markus, where did you get this text from, especially the third * looks
> > interesting precise?  For me it was not contained in the download.
> >
> > @Andrew, nevertheless, I think The Mathworks has not changed it's terms
> > of use since 2016 .  There they are very explicit about the usage of
> > "their" hosted code.
> >
> > Kai
> >
> > [1]
> > https://wiki.octave.org/FAQ#Why_can.27t_I_use_code_from_File_Exchange_in_Octave.3F
> >
>
> jwe found that "only witbh MathWorks products" thing for me: It's in
> section 2.iii of the Matlab Central Terms of use [1].
>
> Sounds like the site terms of use haven't caught up with the recent File
> Exchange migration to BSD licensing. But I'm guessing Kai's right and we
> should probably avoid redistributing any File Exchange material here as
> long as there's any ambiguity.
>
> Cheers,
> Andrew
>
> [1]
> https://www.mathworks.com/matlabcentral/termsofuse.html
>

Thanks for the hint and finding that clause in their terms of use.
I wasn't aware that they changed the license for contributions on the file exchange.

The terms I copied are from here:
https://de.mathworks.com/matlabcentral/fileexchange/71156-getting-started-with-matlab-video-example-files

I was (wrongfully) assuming that the license was the same for all downloads from there.

I am not a lawyer. But given their terms of use, I'd still recommend to not use any work with Octave that is based on their file exchange.

Markus




Reply | Threaded
Open this post in threaded view
|

Re: Directory Listing in Octave -- Recursion

ellocco
The same is true for me, I am not a loyer.
Is it clear if the next Octave version will support the '**' notation for dir()?

Am Mi., 17. Juni 2020 um 20:04 Uhr schrieb Markus Mützel <[hidden email]>:
Am 17. Juni 2020 um 18:25 Uhr schrieb "Andrew Janke":
> On 6/17/20 12:00 PM, Kai Torben Ohlhus wrote:
> > On 6/18/20 12:19 AM, Andrew Janke wrote:
> >>
> >>
> >> On 6/17/20 10:15 AM, Markus Mützel wrote:
> >>>
> >>> The license agreement of The Mathworks' FileExchange doesn't permit using those functions (or work derived of them) with other software than their own.
> >>> You should have received a copy of that license when downloading the files from the FileExchange:
> >>>
> >>> Redistribution and use in source and binary forms, with or without
> >>> modification, are permitted provided that the following conditions are
> >>> met:
> >>>
> >>> * Redistributions of source code must retain the above copyright
> >>> notice, this list of conditions and the following disclaimer.
> >>> * Redistributions in binary form must reproduce the above copyright
> >>> notice, this list of conditions and the following disclaimer in
> >>> the documentation and/or other materials provided with the distribution.
> >>> * In all cases, the software is, and all modifications and derivatives
> >>> of the software shall be, licensed to you solely for use in conjunction
> >>> with MathWorks products and service offerings.
> >>>
> >>>
> >>> Please, do not distribute such files on the Octave mailing list.
> >>>
> >>> Markus
> >>
> >> I'm not sure that this is true any more. MathWorks File Exchange has
> >> switched to using the BSD 3-Clause license for all programs posted
> >> there. So it may depend on exactly when the file was published.
> >>
> >> The license for this particular posting appears to be BSD 2-Clause, not
> >> the variant license with the "MathWorks productions only" clause.
> >>
> >> Cheers,
> >> Andrew
> >>
> >
> >
> > @Markus, where did you get this text from, especially the third * looks
> > interesting precise?  For me it was not contained in the download.
> >
> > @Andrew, nevertheless, I think The Mathworks has not changed it's terms
> > of use since 2016 .  There they are very explicit about the usage of
> > "their" hosted code.
> >
> > Kai
> >
> > [1]
> > https://wiki.octave.org/FAQ#Why_can.27t_I_use_code_from_File_Exchange_in_Octave.3F
> >
>
> jwe found that "only witbh MathWorks products" thing for me: It's in
> section 2.iii of the Matlab Central Terms of use [1].
>
> Sounds like the site terms of use haven't caught up with the recent File
> Exchange migration to BSD licensing. But I'm guessing Kai's right and we
> should probably avoid redistributing any File Exchange material here as
> long as there's any ambiguity.
>
> Cheers,
> Andrew
>
> [1]
> https://www.mathworks.com/matlabcentral/termsofuse.html
>

Thanks for the hint and finding that clause in their terms of use.
I wasn't aware that they changed the license for contributions on the file exchange.

The terms I copied are from here:
https://de.mathworks.com/matlabcentral/fileexchange/71156-getting-started-with-matlab-video-example-files

I was (wrongfully) assuming that the license was the same for all downloads from there.

I am not a lawyer. But given their terms of use, I'd still recommend to not use any work with Octave that is based on their file exchange.

Markus




--
Stefan Pofahl
Zollgasse 5
8020 Graz
Österreich
Tel.: +43 (316) 33 2001