cellfunc vs cell2mat speed?

classic Classic list List threaded Threaded
12 messages Options
Lev
Reply | Threaded
Open this post in threaded view
|

cellfunc vs cell2mat speed?

Lev

Hi All,

I am just wondering why is it so difficult to convert a cell array to matrix?

I used cellfunc on  cell array of 59000 cells which took 7 seconds
but the conversion to a matrix of size
443320 , 7
took about 1080 seconds on my machine.

The cell2mat on the same data took appx 150 times more time.
I am just shocked. Should I replace the original cell2mat with a C routine?

Lev

--
Blogger of http://fapuma.blogspot.com
_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: cellfunc vs cell2mat speed?

David Bateman-6
Levente Torok wrote:

> Hi All,
>
> I am just wondering why is it so difficult to convert a cell array to matrix?
>
> I used cellfunc on  cell array of 59000 cells which took 7 seconds
> but the conversion to a matrix of size
> 443320 , 7
> took about 1080 seconds on my machine.
>
> The cell2mat on the same data took appx 150 times more time.
> I am just shocked. Should I replace the original cell2mat with a C routine?
>
> Lev
>

Short example please. The above is not much use as a bug report.

D.
_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Lev
Reply | Threaded
Open this post in threaded view
|

Re: cellfunc vs cell2mat speed?

Lev
Hi David and others

This is small speed test:

function t = test( s )

    t = time;
    c=cell( s, 1 );
    r = rand( 8, 1 );
    c{:,1} = r;
    cc = cell2mat( c );
    t = time - t;

end

octave:4> test( 1000 )
ans =  0.27966
octave:5> test( 10000 )
ans =  12.347
octave:6> test( 13000 )
ans =  26.865
octave:3> test( 16000 )
ans =  47.761
octave:4> test( 19000 )
ans =  83.082


As it seems, each adding of 3000 elements doubles the time it needs to convert cell to matrix.
I believe the slowness is due to the recursive scripting nature of the cell2mat.m function.
I thought I will write a small conversion program in C with limited capabilities for my own needs
( ie. only c{i,1} =matrix will be handled ) but I am so much confused with the interface of the
Cell class.


( NB: if I replace cell2mat with cellfun( c, "mean" ) I would get:
octave:10> test( 20000 )
ans =  8.0793
and this increases linearly with size of the cell array)
Thank you very much in advance,

Levente




--
Blogger of http://fapuma.blogspot.com
_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: cellfunc vs cell2mat speed?

David Bateman-2
Levente Torok wrote:

> Hi David and others
>
> This is small speed test:
>
> function t = test( s )
>
>     t = time;
>     c=cell( s, 1 );
>     r = rand( 8, 1 );
>     c{:,1} = r;
>     cc = cell2mat( c );
>     t = time - t;
>
> end
>
> octave:4> test( 1000 )
> ans =  0.27966
> octave:5> test( 10000 )
> ans =  12.347
> octave:6> test( 13000 )
> ans =  26.865
> octave:3> test( 16000 )
> ans =  47.761
> octave:4> test( 19000 )
> ans =  83.082
>
>
> As it seems, each adding of 3000 elements doubles the time it needs to convert cell to matrix.
> I believe the slowness is due to the recursive scripting nature of the cell2mat.m function.
> I thought I will write a small conversion program in C with limited capabilities for my own needs
> ( ie. only c{i,1} =matrix will be handled ) but I am so much confused with the interface of the
> Cell class.
>
>
> ( NB: if I replace cell2mat with cellfun( c, "mean" ) I would get:
> octave:10> test( 20000 )
> ans =  8.0793
> and this increases linearly with size of the cell array)
> Thank you very much in advance,
>
> Levente
>
>
>
>
>  

The code that handles this is essentially

 t = cputime; cc = cat(1, c{:}); cputime - t

There are two issues with the above. The first is that the cat function
is called with an enormously large number of arguments and the second is
that the cat function, unlikely [], doesn't make any specializations for
the same class of data in the concatenation. Compare the above with

t = cputime; cc = [c{:}]; cputime - t;

I'm working on a fix..


D.
_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Lev
Reply | Threaded
Open this post in threaded view
|

Re: cellfunc vs cell2mat speed?

Lev
Hi David,

On Tuesday 09 September 2008, David Bateman wrote:

> Levente Torok wrote:
> > Hi David and others
> >
> > This is small speed test:
> >
> > function t = test( s )
> >
> >     t = time;
> >     c=cell( s, 1 );
> >     r = rand( 8, 1 );
> >     c{:,1} = r;
> >     cc = cell2mat( c );
> >     t = time - t;
> >
> > end
> >
> > octave:4> test( 1000 )
> > ans =  0.27966
> > octave:5> test( 10000 )
> > ans =  12.347
> > octave:6> test( 13000 )
> > ans =  26.865
> > octave:3> test( 16000 )
> > ans =  47.761
> > octave:4> test( 19000 )
> > ans =  83.082
> >
> >
> > As it seems, each adding of 3000 elements doubles the time it needs to convert cell to matrix.
> > I believe the slowness is due to the recursive scripting nature of the cell2mat.m function.
> > I thought I will write a small conversion program in C with limited capabilities for my own needs
> > ( ie. only c{i,1} =matrix will be handled ) but I am so much confused with the interface of the
> > Cell class.
> >
> >
> > ( NB: if I replace cell2mat with cellfun( c, "mean" ) I would get:
> > octave:10> test( 20000 )
> > ans =  8.0793
> > and this increases linearly with size of the cell array)
> > Thank you very much in advance,
> >
> > Levente
> >
> >
> >
> >
> >  
>
> The code that handles this is essentially
>
>  t = cputime; cc = cat(1, c{:}); cputime - t
I replaced it. This is just a little bit better now:

octave:2> test( 10000 )
ans =  9.3086
octave:3> test( 13000 )
ans =  20.437
octave:4> test( 16000 )
ans =  39.590
The trend is the same.

>
> There are two issues with the above. The first is that the cat function
> is called with an enormously large number of arguments and the second is
> that the cat function, unlikely [], doesn't make any specializations for
> the same class of data in the concatenation. Compare the above with
>
> t = cputime; cc = [c{:}]; cputime - t;

octave:7> test( 40000 )
ans =  0.30402

This is extremely fast however it does not do the thing I want since it concatenates the
columns rowwise rather than columnwise. ie.
c=cell(2,1)
c{1} = [ 1;2 ; 3];
c{2} = [ 4;2 ; 3];
octave:12> [c{:}]
ans =

   1   4
   2   2
   3   3

It wouldn't be the problem if I wouldn't need to handle vectors of different sizes, hence I cannot use
reshape to it. Isn't there such a simple expression to do column wise concatenation?

> I'm working on a fix..
Thank you very much in advance.

Lev

--
Blogger of http://fapuma.blogspot.com
_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: cellfunc vs cell2mat speed?

David Bateman-2
Levente Torok wrote:

> Hi David,
>
> On Tuesday 09 September 2008, David Bateman wrote:
>  
>> Levente Torok wrote:
>>    
>>> Hi David and others
>>>
>>> This is small speed test:
>>>
>>> function t = test( s )
>>>
>>>     t = time;
>>>     c=cell( s, 1 );
>>>     r = rand( 8, 1 );
>>>     c{:,1} = r;
>>>     cc = cell2mat( c );
>>>     t = time - t;
>>>
>>> end
>>>
>>> octave:4> test( 1000 )
>>> ans =  0.27966
>>> octave:5> test( 10000 )
>>> ans =  12.347
>>> octave:6> test( 13000 )
>>> ans =  26.865
>>> octave:3> test( 16000 )
>>> ans =  47.761
>>> octave:4> test( 19000 )
>>> ans =  83.082
>>>
>>>
>>> As it seems, each adding of 3000 elements doubles the time it needs to convert cell to matrix.
>>> I believe the slowness is due to the recursive scripting nature of the cell2mat.m function.
>>> I thought I will write a small conversion program in C with limited capabilities for my own needs
>>> ( ie. only c{i,1} =matrix will be handled ) but I am so much confused with the interface of the
>>> Cell class.
>>>
>>>
>>> ( NB: if I replace cell2mat with cellfun( c, "mean" ) I would get:
>>> octave:10> test( 20000 )
>>> ans =  8.0793
>>> and this increases linearly with size of the cell array)
>>> Thank you very much in advance,
>>>
>>> Levente
>>>
>>>
>>>
>>>
>>>  
>>>      
>> The code that handles this is essentially
>>
>>  t = cputime; cc = cat(1, c{:}); cputime - t
>>    
> I replaced it. This is just a little bit better now:
>
> octave:2> test( 10000 )
> ans =  9.3086
> octave:3> test( 13000 )
> ans =  20.437
> octave:4> test( 16000 )
> ans =  39.590
> The trend is the same.
>
>  
>> There are two issues with the above. The first is that the cat function
>> is called with an enormously large number of arguments and the second is
>> that the cat function, unlikely [], doesn't make any specializations for
>> the same class of data in the concatenation. Compare the above with
>>
>> t = cputime; cc = [c{:}]; cputime - t;
>>    
>
> octave:7> test( 40000 )
> ans =  0.30402
>
> This is extremely fast however it does not do the thing I want since it concatenates the
> columns rowwise rather than columnwise. ie.
> c=cell(2,1)
> c{1} = [ 1;2 ; 3];
> c{2} = [ 4;2 ; 3];
> octave:12> [c{:}]
> ans =
>
>    1   4
>    2   2
>    3   3
>
> It wouldn't be the problem if I wouldn't need to handle vectors of different sizes, hence I cannot use
> reshape to it. Isn't there such a simple expression to do column wise concatenation?
>
>  
Yes something like

function c = mycell2mat (c)
  k = 1;
  while (numel (c) > 1)
    n = rows (c);
    if (n == 1)
      if (ndims (c) == 2)
    c = reshape (c, [numel(c), 1]);
      else
    c = reshape (c, size(c)(2:end));
      endif
    else
      sz = size (c)(2 : end);
      if (isscalar (sz))
    c1 = cell ([sz, 1]);
      else
    c1 = cell (sz);
    sz = prod (sz);
      endif
      if (k == 2)
    for i = 1 : sz
      c1{i} = [c{(i - 1) * n + (1 : n)}];
    endfor
      else
    ## This is faster than
    ##   for i = 1:sz, c1{i} = cat (k, c{(i - 1) * n + (1 : n)}); endfor
    idx = 1 : ndims(c);
    idx(k) = idx (2);
    idx(2) = k;
    c = cellfun(@(x) permute (x, idx), c, "UniformOutput", false);
    for i = 1:sz
      c1{i} = ipermute ([c{(i - 1) * n + (1 : n)}], idx);
    endfor
      endif
      c = c1;
    endif
    k++;
  endwhile
  c = [c{:}];
endfunction


is generic. Though the case of "k=1" could be vastly accelerated if
there existed a transpose opertaor of a cs-list that created a
semi-colon separated list. I don't think it exists, so maybe the above
is the best we can do without reworking the "cat" function to make the
same speedups as the "[]" code.

D.


_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

[Changeset} Re: cellfunc vs cell2mat speed?

David Bateman-2
I'd suggest something like teh attached changeset that improves the
speed but keeps teh code compatible.

D.


# HG changeset patch
# User David Bateman <[hidden email]>
# Date 1221002489 -7200
# Node ID 7bd7ce1d2634dab6969acc76af0de702f9eb222f
# Parent  eb774ade7516a05fdc6a3dec59eaa15ab1827ddd
improve speed of cell2mat

diff --git a/scripts/ChangeLog b/scripts/ChangeLog
--- a/scripts/ChangeLog
+++ b/scripts/ChangeLog
@@ -1,3 +1,7 @@ 2008-09-09  John W. Eaton  <[hidden email]
+2008-09-09  David Bateman  <[hidden email]>
+
+ * general/cell2mat.m: Improve the speed..
+
 2008-09-09  John W. Eaton  <[hidden email]>
 
  * time/datestr.m: Convert format and use strftime to do most of
diff --git a/scripts/general/cell2mat.m b/scripts/general/cell2mat.m
--- a/scripts/general/cell2mat.m
+++ b/scripts/general/cell2mat.m
@@ -48,20 +48,46 @@ function m = cell2mat (c)
     else
       error ("cell2mat: all elements of cell array must be numeric, logical or char");
     endif
+  elseif (ndims (c) == 2)
+    nr = rows (c);
+    c1 = cell (nr, 1);
+    for i = 1 : nr
+      c1{i} = [c{i : nr : end}];
+    endfor
+    ## This is faster than "c = cat(1, c{:})"
+    c = [cellfun(@(x) x.', c1, "UniformOutput", false){:}].';
   else
-    ## n dimensions case
-    for k = ndims (c):-1:2,
+   nd = ndims (c);
+   for k = nd : -1 : 2
       sz = size (c);
+      if (k > ndims (c) || sz(end) == 1)
+ continue;
+      endif
       sz(end) = 1;
       c1 = cell (sz);
-      for i = 1:(prod (sz))
-        c1{i} = cat (k, c{i:(prod (sz)):end});
-      endfor
+      sz = prod (sz);
+      if (k == 2)
+        for i = 1 : sz
+  c1{i} = [c{i : sz : end}];
+        endfor
+      else
+        ## This is faster than
+        ##   for i = 1:sz, c1{i} = cat (k, c{i:(prod (sz)):end}); endfor
+ idx = [1, k, (3 : (k - 1)), 2, ((k + 1): nd)];
+        c = cellfun(@(x) permute (x, idx), c, "UniformOutput", false);
+        for i = 1: sz
+  c1{i} = ipermute ([c{i : sz : end}], idx);
+        endfor
+      endif
       c = c1;
     endfor
-    m = cat (1, c1{:});
+    if (numel (c) > 1)
+      idx = [2, 1, 3 : nd];
+      c = ipermute([cellfun(@(x) permute (x, idx), c, "UniformOutput", false){:}], idx);
+    else
+      c = c{1};
+    endif
   endif
-
 endfunction
 
 ## Tests

_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

[Changeset} Re: cellfunc vs cell2mat speed?

John W. Eaton-6
On 10-Sep-2008, David Bateman wrote:

| I'd suggest something like teh attached changeset that improves the
| speed but keeps teh code compatible.

Thanks, I applied it along with the additional change you sent
off the list.  The combined change is attached below as a single
changeset.

jwe


# HG changeset patch
# User David Bateman <[hidden email]>
# Date 1221167006 14400
# Node ID 3b2346046d32ad8a819b26236c575a6fc2a1436a
# Parent  c066714ee5d53d68fe5db9c69f2e47b8edfaab14
improve speed of cell2mat

diff --git a/scripts/ChangeLog b/scripts/ChangeLog
--- a/scripts/ChangeLog
+++ b/scripts/ChangeLog
@@ -1,3 +1,7 @@
+2008-09-11  David Bateman  <[hidden email]>
+
+ * general/cell2mat.m: Improve the speed.
+
 2008-09-09  John W. Eaton  <[hidden email]>
 
  * time/datestr.m: Convert format and use strftime to do most of
diff --git a/scripts/general/cell2mat.m b/scripts/general/cell2mat.m
--- a/scripts/general/cell2mat.m
+++ b/scripts/general/cell2mat.m
@@ -48,20 +48,46 @@
     else
       error ("cell2mat: all elements of cell array must be numeric, logical or char");
     endif
+  elseif (ndims (c) == 2)
+    nr = rows (c);
+    c1 = cell (nr, 1);
+    for i = 1 : nr
+      c1{i} = [c{i : nr : end}];
+    endfor
+    ## This is faster than "c = cat(1, c{:})"
+    m = [cellfun(@(x) x.', c1, "UniformOutput", false){:}].';
   else
-    ## n dimensions case
-    for k = ndims (c):-1:2,
+   nd = ndims (c);
+   for k = nd : -1 : 2
       sz = size (c);
+      if (k > ndims (c) || sz(end) == 1)
+ continue;
+      endif
       sz(end) = 1;
       c1 = cell (sz);
-      for i = 1:(prod (sz))
-        c1{i} = cat (k, c{i:(prod (sz)):end});
-      endfor
+      sz = prod (sz);
+      if (k == 2)
+        for i = 1 : sz
+  c1{i} = [c{i : sz : end}];
+        endfor
+      else
+        ## This is faster than
+        ##   for i = 1:sz, c1{i} = cat (k, c{i:(prod (sz)):end}); endfor
+ idx = [1, k, (3 : (k - 1)), 2, ((k + 1): nd)];
+        c = cellfun(@(x) permute (x, idx), c, "UniformOutput", false);
+        for i = 1: sz
+  c1{i} = ipermute ([c{i : sz : end}], idx);
+        endfor
+      endif
       c = c1;
     endfor
-    m = cat (1, c1{:});
+    if (numel (c) > 1)
+      idx = [2, 1, 3 : nd];
+      m = ipermute([cellfun(@(x) permute (x, idx), c, "UniformOutput", false){:}], idx);
+    else
+      m = c{1};
+    endif
   endif
-
 endfunction
 
 ## Tests

_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: [Changeset} Re: cellfunc vs cell2mat speed?

David Bateman-3
John W. Eaton wrote:

> On 10-Sep-2008, David Bateman wrote:
>
> | I'd suggest something like teh attached changeset that improves the
> | speed but keeps teh code compatible.
>
> Thanks, I applied it along with the additional change you sent
> off the list.  The combined change is attached below as a single
> changeset.
>
> jwe
>
>  
Thinking about this further we should in fact have single type
concatenation specializations in Fcat as well in the same manner as in
pt-mat.cc. The attached patch does this and then reworks cell2mat again
for even better speed.

Cheers
David


--
David Bateman                                [hidden email]
Motorola Labs - Paris                        +33 1 69 35 48 04 (Ph)
Parc Les Algorithmes, Commune de St Aubin    +33 6 72 01 06 33 (Mob)
91193 Gif-Sur-Yvette FRANCE                  +33 1 69 35 77 01 (Fax)

The information contained in this communication has been classified as:

[x] General Business Information
[ ] Motorola Internal Use Only
[ ] Motorola Confidential Proprietary


# HG changeset patch
# User David Bateman <[hidden email]>
# Date 1221494504 -7200
# Node ID 6eb9b1bd1e6b0be4cc718c7fb0fbfcd53afb00e7
# Parent  adeab8d7be8c0d5b7600623be381b94b15b43211
Special case single type conacation in Fcat. Rework cell2mat to take advantage

diff --git a/scripts/ChangeLog b/scripts/ChangeLog
--- a/scripts/ChangeLog
+++ b/scripts/ChangeLog
@@ -1,3 +1,8 @@ 2008-09-09  David Bateman  <dbateman@fre
+2008-09-15  David Bateman  <[hidden email]>
+
+ * general/cell2mat.m: Backout previous change. Special case 2D
+ case for speed.
+
 2008-09-09  David Bateman  <[hidden email]>
 
  * general/cell2mat.m: Improve the speed..
diff --git a/scripts/general/cell2mat.m b/scripts/general/cell2mat.m
--- a/scripts/general/cell2mat.m
+++ b/scripts/general/cell2mat.m
@@ -50,44 +50,34 @@ function m = cell2mat (c)
     endif
   elseif (ndims (c) == 2)
     nr = rows (c);
-    c1 = cell (nr, 1);
-    for i = 1 : nr
-      c1{i} = [c{i : nr : end}];
-    endfor
-    ## This is faster than "c = cat(1, c{:})"
-    m = [cellfun(@(x) x.', c1, "UniformOutput", false){:}].';
+    nc = columns (c);
+    if (nc > nr)
+      c1 = cell (nr, 1);
+      for i = 1 : nr
+ c1{i} = [c{i : nr : end}];
+      endfor
+      m = cat (1, c1 {:});
+    else
+      c1 = cell (nc, 1);
+      for i = 1 : nc
+ c1{i} = cat (1, c{(i - 1) * nr  + [1 : nr]});
+      endfor
+      m = [c1{:}];
+    endif
   else
-   nd = ndims (c);
-   for k = nd : -1 : 2
+    ## n dimensions case
+    for k = ndims (c):-1:2,
       sz = size (c);
-      if (k > ndims (c) || sz(end) == 1)
- continue;
-      endif
       sz(end) = 1;
       c1 = cell (sz);
-      sz = prod (sz);
-      if (k == 2)
-        for i = 1 : sz
-  c1{i} = [c{i : sz : end}];
-        endfor
-      else
-        ## This is faster than
-        ##   for i = 1:sz, c1{i} = cat (k, c{i:(prod (sz)):end}); endfor
- idx = [1, k, (3 : (k - 1)), 2, ((k + 1): nd)];
-        c = cellfun(@(x) permute (x, idx), c, "UniformOutput", false);
-        for i = 1: sz
-  c1{i} = ipermute ([c{i : sz : end}], idx);
-        endfor
-      endif
+      for i = 1:(prod (sz))
+        c1{i} = cat (k, c{i:(prod (sz)):end});
+      endfor
       c = c1;
     endfor
-    if (numel (c) > 1)
-      idx = [2, 1, 3 : nd];
-      m = ipermute([cellfun(@(x) permute (x, idx), c, "UniformOutput", false){:}], idx);
-    else
-      m = c{1};
-    endif
+    m = cat (1, c1{:});
   endif
+
 endfunction
 
 ## Tests
diff --git a/src/ChangeLog b/src/ChangeLog
--- a/src/ChangeLog
+++ b/src/ChangeLog
@@ -1,5 +1,14 @@ 2008-09-15  David Bateman  <dbateman@fre
 2008-09-15  David Bateman  <[hidden email]>
 
+ * data.cc (SINGLE_TYPE_CONCAT, DO_SINGLE_TYPE_CONCAT): New macros
+ (do_cat): Special case single type concatenations for speed.
+ * pt.mat.cc (std::string get_concat_class (const std::string&,
+ const std::string&), void maybe_warn_string_concat (bool, bool)):
+ Remove static declaration.
+ * pt-mat.h (std::string get_concat_class (const std::string&,
+ const std::string&), void maybe_warn_string_concat (bool, bool)):
+ Define extern here.
+
  * DLD-FUNCTIONS/sparse.cc (Fsparse): Clarify the help string.
 
 2008-09-13  David Bateman  <[hidden email]>
diff --git a/src/data.cc b/src/data.cc
--- a/src/data.cc
+++ b/src/data.cc
@@ -1718,6 +1718,51 @@ return the product of the elements.\n\
 
  */
 
+#define SINGLE_TYPE_CONCAT(TYPE, EXTRACTOR) \
+  do \
+    { \
+      int dv_len = dv.length (); \
+      Array<octave_idx_type> ra_idx (dv_len > 1 ? dv_len : 2, 0); \
+      \
+      for (int j = 1; j < n_args; j++) \
+ { \
+  OCTAVE_QUIT; \
+  \
+  TYPE ra = args(j).EXTRACTOR (); \
+  \
+  if (! error_state) \
+    { \
+      result.insert (ra, ra_idx); \
+      \
+      if (error_state) \
+        return retval; \
+      \
+      dim_vector dv_tmp = args (j).dims (); \
+      \
+      if (dim >= dv_len) \
+        { \
+  if (j > 1) \
+    error ("%s: indexing error", fname.c_str ()); \
+  break; \
+ } \
+      else \
+ ra_idx (dim) += (dim < dv_tmp.length () ? dv_tmp (dim) : 1); \
+    } \
+ } \
+    } \
+ while (0)
+
+#define DO_SINGLE_TYPE_CONCAT(TYPE, EXTRACTOR) \
+  do \
+    { \
+      TYPE result (dv); \
+      \
+      SINGLE_TYPE_CONCAT(TYPE, EXTRACTOR); \
+      \
+      retval = result; \
+    } \
+ while (0)
+
 static octave_value
 do_cat (const octave_value_list& args, std::string fname)
 {
@@ -1743,7 +1788,13 @@ do_cat (const octave_value_list& args, s
  {
   
   dim_vector  dv = args(1).dims ();
+  std::string result_type = args(1).class_name ();
   
+  bool all_sq_strings_p = args(1).is_sq_string ();
+  bool all_dq_strings_p = args(1).is_dq_string ();
+  bool all_real_p = args(1).is_real_type ();
+  bool any_sparse_p = args(1).is_sparse_type();
+
   for (int i = 2; i < args.length (); i++)
       {
       // add_dims constructs a dimension vector which holds the
@@ -1755,68 +1806,145 @@ do_cat (const octave_value_list& args, s
   error ("cat: dimension mismatch");
   return retval;
  }
-    }
-
-  // The lines below might seem crazy, since we take a copy
-  // of the first argument, resize it to be empty and then resize
-  // it to be full. This is done since it means that there is no
-  // recopying of data, as would happen if we used a single resize.
-  // It should be noted that resize operation is also significantly
-  // slower than the do_cat_op function, so it makes sense to have an
-  // empty matrix and copy all data.
-  //
-  // We might also start with a empty octave_value using
-  //   tmp = octave_value_typeinfo::lookup_type (args(1).type_name());
-  // and then directly resize. However, for some types there might be
-  // some additional setup needed, and so this should be avoided.
-
-  octave_value tmp = args (1);
-  tmp = tmp.resize (dim_vector (0,0)).resize (dv);
-
-  if (error_state)
-    return retval;
-
-  int dv_len = dv.length ();
-  Array<octave_idx_type> ra_idx (dv_len, 0);
-
-  for (int j = 1; j < n_args; j++)
-    {
-      // Can't fast return here to skip empty matrices as something
-      // like cat(1,[],single([])) must return an empty matrix of
-      // the right type.
-      tmp = do_cat_op (tmp, args (j), ra_idx);
+      
+      result_type =
+ get_concat_class (result_type, args(i).class_name ());
+
+      if (all_sq_strings_p && ! args(i).is_sq_string ())
+ all_sq_strings_p = false;
+      if (all_dq_strings_p && ! args(i).is_dq_string ())
+ all_dq_strings_p = false;
+      if (all_real_p && ! args(i).is_real_type ())
+ all_real_p = false;
+      if (!any_sparse_p && args(i).is_sparse_type ())
+ any_sparse_p = true;
+    }
+
+  if (result_type == "double")
+    {
+      if (any_sparse_p)
+ {    
+  if (all_real_p)
+    DO_SINGLE_TYPE_CONCAT (SparseMatrix, sparse_matrix_value);
+  else
+    DO_SINGLE_TYPE_CONCAT (SparseComplexMatrix, sparse_complex_matrix_value);
+ }
+      else
+ {
+  if (all_real_p)
+    DO_SINGLE_TYPE_CONCAT (NDArray, array_value);
+  else
+    DO_SINGLE_TYPE_CONCAT (ComplexNDArray, complex_array_value);
+ }
+    }
+  else if (result_type == "single")
+    {
+      if (all_real_p)
+ DO_SINGLE_TYPE_CONCAT (FloatNDArray, float_array_value);
+      else
+ DO_SINGLE_TYPE_CONCAT (FloatComplexNDArray,
+       float_complex_array_value);
+    }
+  else if (result_type == "char")
+    {
+      char type = all_dq_strings_p ? '"' : '\'';
+
+      maybe_warn_string_concat (all_dq_strings_p, all_sq_strings_p);
+
+      charNDArray result (dv, Vstring_fill_char);
+
+      SINGLE_TYPE_CONCAT (charNDArray, char_array_value);
+
+      retval = octave_value (result, true, type);
+    }
+  else if (result_type == "logical")
+    {
+      if (any_sparse_p)
+ DO_SINGLE_TYPE_CONCAT (SparseBoolMatrix, sparse_bool_matrix_value);
+      else
+ DO_SINGLE_TYPE_CONCAT (boolNDArray, bool_array_value);
+    }
+  else if (result_type == "int8")
+    DO_SINGLE_TYPE_CONCAT (int8NDArray, int8_array_value);
+  else if (result_type == "int16")
+    DO_SINGLE_TYPE_CONCAT (int16NDArray, int16_array_value);
+  else if (result_type == "int32")
+    DO_SINGLE_TYPE_CONCAT (int32NDArray, int32_array_value);
+  else if (result_type == "int64")
+    DO_SINGLE_TYPE_CONCAT (int64NDArray, int64_array_value);
+  else if (result_type == "uint8")
+    DO_SINGLE_TYPE_CONCAT (uint8NDArray, uint8_array_value);
+  else if (result_type == "uint16")
+    DO_SINGLE_TYPE_CONCAT (uint16NDArray, uint16_array_value);
+  else if (result_type == "uint32")
+    DO_SINGLE_TYPE_CONCAT (uint32NDArray, uint32_array_value);
+  else if (result_type == "uint64")
+    DO_SINGLE_TYPE_CONCAT (uint64NDArray, uint64_array_value);
+  else
+    {
+      // The lines below might seem crazy, since we take a copy
+      // of the first argument, resize it to be empty and then resize
+      // it to be full. This is done since it means that there is no
+      // recopying of data, as would happen if we used a single resize.
+      // It should be noted that resize operation is also significantly
+      // slower than the do_cat_op function, so it makes sense to have
+      // an empty matrix and copy all data.
+      //
+      // We might also start with a empty octave_value using
+      //   tmp = octave_value_typeinfo::lookup_type
+      //                                (args(1).type_name());
+      // and then directly resize. However, for some types there might
+      // be some additional setup needed, and so this should be avoided.
+
+      octave_value tmp = args (1);
+      tmp = tmp.resize (dim_vector (0,0)).resize (dv);
 
       if (error_state)
  return retval;
 
-      dim_vector dv_tmp = args (j).dims ();
-
-      if (dim >= dv_len)
- {
-  if (j > 1)
-    error ("%s: indexing error", fname.c_str ());
-  break;
- }
-      else
- ra_idx (dim) += (dim < dv_tmp.length () ?
- dv_tmp (dim) : 1);
-    }
-
-  // Reshape, chopping trailing singleton dimensions
-  dv.chop_trailing_singletons ();
-  tmp = tmp.reshape (dv);
-
-  retval = tmp;
+      int dv_len = dv.length ();
+      Array<octave_idx_type> ra_idx (dv_len, 0);
+
+      for (int j = 1; j < n_args; j++)
+ {
+  // Can't fast return here to skip empty matrices as something
+  // like cat(1,[],single([])) must return an empty matrix of
+  // the right type.
+  tmp = do_cat_op (tmp, args (j), ra_idx);
+
+  if (error_state)
+    return retval;
+
+  dim_vector dv_tmp = args (j).dims ();
+
+  if (dim >= dv_len)
+    {
+      if (j > 1)
+ error ("%s: indexing error", fname.c_str ());
+      break;
+    }
+  else
+    ra_idx (dim) += (dim < dv_tmp.length () ?
+     dv_tmp (dim) : 1);
+ }
+      retval = tmp;
+    }
+
+  if (! error_state)
+    {
+      // Reshape, chopping trailing singleton dimensions
+      dv.chop_trailing_singletons ();
+      retval = retval.reshape (dv);
+    }
  }
       else
  error ("%s: invalid dimension argument", fname.c_str ());
     }
   else
     print_usage ();
-
+
   return retval;
 }
-
 
 DEFUN (horzcat, args, ,
        "-*- texinfo -*-\n\
diff --git a/src/pt-mat.cc b/src/pt-mat.cc
--- a/src/pt-mat.cc
+++ b/src/pt-mat.cc
@@ -184,7 +184,7 @@ private:
   tm_row_const_rep *rep;
 };
 
-static std::string
+std::string
 get_concat_class (const std::string& c1, const std::string& c2)
 {
   std::string retval = octave_base_value::static_class_name ();
@@ -699,7 +699,7 @@ tree_matrix::rvalue (int nargout)
   return retval;
 }
 
-static void
+void
 maybe_warn_string_concat (bool all_dq_strings_p, bool all_sq_strings_p)
 {
   if (! (all_dq_strings_p || all_sq_strings_p))
diff --git a/src/pt-mat.h b/src/pt-mat.h
--- a/src/pt-mat.h
+++ b/src/pt-mat.h
@@ -81,6 +81,12 @@ private:
 // The character to fill with when creating string arrays.
 extern char Vstring_fill_char;
 
+extern std::string
+get_concat_class (const std::string& c1, const std::string& c2);
+
+extern void
+maybe_warn_string_concat (bool all_dq_strings_p, bool all_sq_strings_p);
+
 #endif
 
 /*

_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: [Changeset} Re: cellfunc vs cell2mat speed?

John W. Eaton-6
On 15-Sep-2008, David Bateman wrote:

| John W. Eaton wrote:
| > On 10-Sep-2008, David Bateman wrote:
| >
| > | I'd suggest something like teh attached changeset that improves the
| > | speed but keeps teh code compatible.
| >
| > Thanks, I applied it along with the additional change you sent
| > off the list.  The combined change is attached below as a single
| > changeset.
| >
| > jwe
| >
| >  
|
| Thinking about this further we should in fact have single type
| concatenation specializations in Fcat as well in the same manner as in
| pt-mat.cc. The attached patch does this and then reworks cell2mat again
| for even better speed.

I applied this changeset.

Thanks,

jwe
_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: [Changeset} Re: cellfunc vs cell2mat speed?

David Bateman
It appears that this patch was partially rejected due to the fact that there was another patch to do_cat for the trailing singletons that changes the same bit of code. Now that it is also applied the rejected bit of code should also apply correctly.

D.
Reply | Threaded
Open this post in threaded view
|

Re: [Changeset} Re: cellfunc vs cell2mat speed?

John W. Eaton-6
On 17-Sep-2008, dbateman wrote:

|
| It appears that this patch was partially rejected due to the fact that there
| was another patch to do_cat for the trailing singletons that changes the
| same bit of code. Now that it is also applied the rejected bit of code
| should also apply correctly.

OK, I think I have it right now.

Thanks,

jwe
_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave