gunzip incompatible

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

gunzip incompatible

Mats Hedlund
In matlab, this works for unzipping an ODF spreadsheet: gunzip('file.ods').
In octave, it does not, replying with 'unknown file format'.
Any thoughts on this and ways to fix it?

Thanks,
Mats

_______________________________________________
Help-octave mailing list
[hidden email]
https://www.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

gunzip incompatible

John W. Eaton-6
On 28-Jan-2008, Mats Hedlund wrote:

| In matlab, this works for unzipping an ODF spreadsheet: gunzip('file.ods').
| In octave, it does not, replying with 'unknown file format'.
| Any thoughts on this and ways to fix it?

Octave's gunzip function is just a .m file.  It ultimately calls
another function called unpack to do the work.  I think the problem is
that gunzip is leaving it up to unpack to determine which
uncompression utility to use, when it should probably be forcing it to
use gzip.  It would be great if someone could have a look at this
problem and propose a patch.  Similar changes would likely be
appropriate for the bunzip2, untar, and unzip functions.

jwe
_______________________________________________
Help-octave mailing list
[hidden email]
https://www.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: gunzip incompatible

Thomas Weber-8
On 28/01/08 13:45 -0500, John W. Eaton wrote:

> On 28-Jan-2008, Mats Hedlund wrote:
>
> | In matlab, this works for unzipping an ODF spreadsheet: gunzip('file.ods').
> | In octave, it does not, replying with 'unknown file format'.
> | Any thoughts on this and ways to fix it?
>
> Octave's gunzip function is just a .m file.  It ultimately calls
> another function called unpack to do the work.  I think the problem is
> that gunzip is leaving it up to unpack to determine which
> uncompression utility to use, when it should probably be forcing it to
> use gzip.  It would be great if someone could have a look at this
> problem and propose a patch.  Similar changes would likely be
> appropriate for the bunzip2, untar, and unzip functions.

Well, how do you determine the compressing algorithm if not by the
extension (by the way, .odf can be extracted by unzip, not gunzip)?

"file" is highly system-dependant. I guess the only solution would be to
try all commands on an unknown file type and check their return value -
bah, that's ugly.

        Thomas
_______________________________________________
Help-octave mailing list
[hidden email]
https://www.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: gunzip incompatible

Muthiah Annamalai-3
Thomas Weber wrote:

> On 28/01/08 13:45 -0500, John W. Eaton wrote:
>  
>> On 28-Jan-2008, Mats Hedlund wrote:
>>
>> | In matlab, this works for unzipping an ODF spreadsheet: gunzip('file.ods').
>> | In octave, it does not, replying with 'unknown file format'.
>> | Any thoughts on this and ways to fix it?
>>
>> Octave's gunzip function is just a .m file.  It ultimately calls
>> another function called unpack to do the work.  I think the problem is
>> that gunzip is leaving it up to unpack to determine which
>> uncompression utility to use, when it should probably be forcing it to
>> use gzip.  It would be great if someone could have a look at this
>> problem and propose a patch.  Similar changes would likely be
>> appropriate for the bunzip2, untar, and unzip functions.
>>    
>
> Well, how do you determine the compressing algorithm if not by the
> extension (by the way, .odf can be extracted by unzip, not gunzip)?
>
> "file" is highly system-dependant. I guess the only solution would be to
> try all commands on an unknown file type and check their return value -
> bah, that's ugly.
>
> Thomas
>  
No I think you could change 'unpack.m' and allow a 4th argument that can
be used
as a fallback.

Otherwise, we could somehow make unpack to figure out who called it, i.e
tar, gzip, etc,
and do the appropriate fallback method.

-Muthu


_______________________________________________
Help-octave mailing list
[hidden email]
https://www.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: gunzip incompatible

John W. Eaton-6
In reply to this post by Thomas Weber-8
On 28-Jan-2008, Thomas Weber wrote:

| On 28/01/08 13:45 -0500, John W. Eaton wrote:
| > On 28-Jan-2008, Mats Hedlund wrote:
| >
| > | In matlab, this works for unzipping an ODF spreadsheet: gunzip('file.ods').
| > | In octave, it does not, replying with 'unknown file format'.
| > | Any thoughts on this and ways to fix it?
| >
| > Octave's gunzip function is just a .m file.  It ultimately calls
| > another function called unpack to do the work.  I think the problem is
| > that gunzip is leaving it up to unpack to determine which
| > uncompression utility to use, when it should probably be forcing it to
| > use gzip.  It would be great if someone could have a look at this
| > problem and propose a patch.  Similar changes would likely be
| > appropriate for the bunzip2, untar, and unzip functions.
|
| Well, how do you determine the compressing algorithm if not by the
| extension (by the way, .odf can be extracted by unzip, not gunzip)?

It's OK for the unpack function to look at the extension to decide
what to do, but but the specific functions like gunzip should just
use the specific extraction program ("gzip -d" for gunzip; unzip for
unzip, etc.) and not care about the extension.  I think this would be
a fairly straightforward change to make.

jwe

_______________________________________________
Help-octave mailing list
[hidden email]
https://www.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: gunzip incompatible

David Bateman
In reply to this post by Thomas Weber-8

Thomas Weber-8 wrote
On 28/01/08 13:45 -0500, John W. Eaton wrote:
> On 28-Jan-2008, Mats Hedlund wrote:
>
> | In matlab, this works for unzipping an ODF spreadsheet: gunzip('file.ods').
> | In octave, it does not, replying with 'unknown file format'.
> | Any thoughts on this and ways to fix it?
>
> Octave's gunzip function is just a .m file.  It ultimately calls
> another function called unpack to do the work.  I think the problem is
> that gunzip is leaving it up to unpack to determine which
> uncompression utility to use, when it should probably be forcing it to
> use gzip.  It would be great if someone could have a look at this
> problem and propose a patch.  Similar changes would likely be
> appropriate for the bunzip2, untar, and unzip functions.

Well, how do you determine the compressing algorithm if not by the
extension (by the way, .odf can be extracted by unzip, not gunzip)?

"file" is highly system-dependant. I guess the only solution would be to
try all commands on an unknown file type and check their return value -
bah, that's ugly.

        Thomas
__________________________
I think john means that the gunzip, etc implicitly have the compression type defined by the function name that is called, whereas as unpack is the generic function that determines the compression type based on the file extension. Therefore if we pass a file to gunzip it should  be assumed that its in  gzip format regardless of any file extension. Should be a relatively simple patch to implement.

D.
Reply | Threaded
Open this post in threaded view
|

Re: gunzip incompatible

John W. Eaton-6
In reply to this post by Muthiah Annamalai-3
On 28-Jan-2008, Muthiah Annamalai wrote:

| No I think you could change 'unpack.m' and allow a 4th argument that can
| be used
| as a fallback.

It wouldn't really be a fallback method, since it would force a
particular method instead of trying to guess then falling back on some
other method if the guess failed.

Also, looking at unpack, it seems that it is already set up to have
the third argument be the filetype (method to use) but that it is not
being used appropriately.  So I think fixing this just a matter of a
small patch to unpack.

jwe
_______________________________________________
Help-octave mailing list
[hidden email]
https://www.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: gunzip incompatible

Muthiah Annamalai-3
John W. Eaton wrote:

> On 28-Jan-2008, Muthiah Annamalai wrote:
>
> | No I think you could change 'unpack.m' and allow a 4th argument that can
> | be used
> | as a fallback.
>
> It wouldn't really be a fallback method, since it would force a
> particular method instead of trying to guess then falling back on some
> other method if the guess failed.
>
> Also, looking at unpack, it seems that it is already set up to have
> the third argument be the filetype (method to use) but that it is not
> being used appropriately.  So I think fixing this just a matter of a
> small patch to unpack.
>
> jwe
>
>  
I think this change makes something like what you say, but it doesnt
handle tar.gz,
tar.bz, tar.bz2. I am not attaching a changelog, as I expect further
improvement.

Index: unpack.m
===================================================================
RCS file: /cvs/octave/scripts/miscellaneous/unpack.m,v
retrieving revision 1.7
diff -c -r1.7 unpack.m
*** unpack.m 12 Oct 2007 21:27:23 -0000 1.7
--- unpack.m 28 Jan 2008 21:42:50 -0000
***************
*** 139,148 ****
@__parse_zip__, false};
endif

! nodotext = ext(! ismember (ext, "."));
!
! origdir = pwd ();

if (isfield (commandlist, nodotext))
[commandv, commandq, parser, move] = deal (commandlist.(nodotext){:});
cstartdir = canonicalize_file_name (origdir);
--- 139,159 ----
@__parse_zip__, false};
endif

! switch ( filetype )
! case "gunzip"
! nodotext = "gz";
! case "bunzip"
! nodotext = "bz";
! case "untar"
! nodotext = "tar";
! case "unzip"
! nodotext = "zip";
! otherwise
! nodotext = ext(! ismember (ext, "."));
! endswitch

+ origdir = pwd ();
+
if (isfield (commandlist, nodotext))
[commandv, commandq, parser, move] = deal (commandlist.(nodotext){:});
cstartdir = canonicalize_file_name (origdir);

Patch is also attached.

Thanks,
Muthu




? Makefile
? PKG_ADD
? unkompress.patch
Index: unpack.m
===================================================================
RCS file: /cvs/octave/scripts/miscellaneous/unpack.m,v
retrieving revision 1.7
diff -c -r1.7 unpack.m
*** unpack.m 12 Oct 2007 21:27:23 -0000 1.7
--- unpack.m 28 Jan 2008 21:42:50 -0000
***************
*** 139,148 ****
        @__parse_zip__, false};
    endif
 
!   nodotext = ext(! ismember (ext, "."));
!  
!   origdir = pwd ();
 
    if (isfield (commandlist, nodotext))
      [commandv, commandq, parser, move] = deal (commandlist.(nodotext){:});
      cstartdir = canonicalize_file_name (origdir);
--- 139,159 ----
        @__parse_zip__, false};
    endif
 
!   switch ( filetype )
!     case "gunzip"
!       nodotext = "gz";
!     case "bunzip"
!       nodotext = "bz";
!     case "untar"
!       nodotext = "tar";
!     case "unzip"
!       nodotext = "zip";
!     otherwise
!       nodotext = ext(! ismember (ext, "."));  
!   endswitch
 
+   origdir = pwd ();
+      
    if (isfield (commandlist, nodotext))
      [commandv, commandq, parser, move] = deal (commandlist.(nodotext){:});
      cstartdir = canonicalize_file_name (origdir);

_______________________________________________
Help-octave mailing list
[hidden email]
https://www.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: gunzip incompatible

Mats Hedlund
In reply to this post by Thomas Weber-8
Thomas Weber wrote:
>
> extension (by the way, .odf can be extracted by unzip, not gunzip)?
>  
Really? Not straight from octave anyway. Maybe you mean using shell
prompt: unix('unzip file.ods')?
This works but it would be nice to be able to use unzip.m (or gunzip.m,
for matlab compatibility).

Thanks,
Mats

_______________________________________________
Help-octave mailing list
[hidden email]
https://www.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: gunzip incompatible

John W. Eaton-6
In reply to this post by Muthiah Annamalai-3
On 28-Jan-2008, Muthiah Annamalai wrote:

| I think this change makes something like what you say, but it doesnt
| handle tar.gz,
| tar.bz, tar.bz2. I am not attaching a changelog, as I expect further
| improvement.

I don't think this is quite right, but I will admit that it is
somewhat more complicated than I initially thought it would be, partly
because of the way unpack is currently written, and partly because for
compatibility, we have to handle things like

  gunzip ("foo")

when the file is actually "foo.tar" or "foo.tar.gz" or "foo.tgz", etc.

How about we move this to the maintainers list?

jwe
_______________________________________________
Help-octave mailing list
[hidden email]
https://www.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: gunzip incompatible

Thomas Weber-8
In reply to this post by Mats Hedlund

Am Montag, den 28.01.2008, 15:03 -0700 schrieb Mats Hedlund:
> Thomas Weber wrote:
> >
> > extension (by the way, .odf can be extracted by unzip, not gunzip)?
> >  
> Really? Not straight from octave anyway. Maybe you mean using shell
> prompt: unix('unzip file.ods')?

Uh, I meant that from the shell. .odf files or more generally, all
OO.org formats are zipped rather than gzipped. Your are right that
currently Octave is limited to a specific set of extensions.

> This works but it would be nice to be able to use unzip.m (or gunzip.m,
> for matlab compatibility).
Well, actually I think that Matlab should at least issue a warning here,
as it is clearly not using gunzip for extraction. However, it seems it
does:

========================================================================
>> gunzip('v.ods')
??? Error using ==> gunzip>gunzipwrite
File "/tmp/z/v.ods" is not in GZIP format.

Error in ==> gunzip>gunzipEntries at 99
  names{end+1} = gunzipwrite(entries(i).file, outputDir, baseName,
streamCopier);

Error in ==> gunzip at 54
   names = gunzipEntries(entries, outputDir);

>> ver
-------------------------------------------------------------------------------------
MATLAB Version 7.1.0.183 (R14) Service Pack 3
========================================================================

Which version of Matlab are you using?

        Thomas

_______________________________________________
Help-octave mailing list
[hidden email]
https://www.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: gunzip incompatible

Mats Hedlund
Thomas Weber wrote:

> Am Montag, den 28.01.2008, 15:03 -0700 schrieb Mats Hedlund:
>  
>> Thomas Weber wrote:
>>    
>>> extension (by the way, .odf can be extracted by unzip, not gunzip)?
>>>  
>>>      
>> Really? Not straight from octave anyway. Maybe you mean using shell
>> prompt: unix('unzip file.ods')?
>>    
>
> Uh, I meant that from the shell. .odf files or more generally, all
> OO.org formats are zipped rather than gzipped. Your are right that
> currently Octave is limited to a specific set of extensions.
>
>  
>> This works but it would be nice to be able to use unzip.m (or gunzip.m,
>> for matlab compatibility).
>>    
> Well, actually I think that Matlab should at least issue a warning here,
> as it is clearly not using gunzip for extraction. However, it seems it
> does:
>
> ========================================================================
>  
>>> gunzip('v.ods')
>>>      
> ??? Error using ==> gunzip>gunzipwrite
> File "/tmp/z/v.ods" is not in GZIP format.
>
> Error in ==> gunzip>gunzipEntries at 99
>   names{end+1} = gunzipwrite(entries(i).file, outputDir, baseName,
> streamCopier);
>
> Error in ==> gunzip at 54
>    names = gunzipEntries(entries, outputDir);
>
>  
>>> ver
>>>      
> -------------------------------------------------------------------------------------
> MATLAB Version 7.1.0.183 (R14) Service Pack 3
> ========================================================================
>
> Which version of Matlab are you using?
>
> Thomas
>
>
>
>  
I believe John brought this zipping discussion to maintainers list.
As far as matlab is concerned, I use it as little as possible :). I do
all serious number crunching in octave. But if it helps I'm on 7.5.
Mainly for plotting, but as plotting gets better in octave I'm looking
forward to a future in complete freedom and without any silly matlab
GUI's. The prompt rules and octave is an order of magnitude more stable.
/Mats
_______________________________________________
Help-octave mailing list
[hidden email]
https://www.cae.wisc.edu/mailman/listinfo/help-octave