# a difficult homework

18 messages
Open this post in threaded view
|

## a difficult homework

 hi, I want to vectorize it but it's so difficult   c=magic(20)   a=[ 20 41 45 50]   b=[ 60 82 90 61]   for i=1:length(a)     x(i)=min(c(a(i):b(i)));    endfor i -- Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html
Open this post in threaded view
|

## Re: a difficult homework

 it result: x = 2    3   17   38 to avoid to use loop i try code it: x=min(c(a:b)); but it's not correct.. x=2 -- Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html
Open this post in threaded view
|

## Re: a difficult homework

 If you really have to avoid the loop, you could try to use "arrayfun" with an anonymous function (which kind of hides the loop). I can't think of a straight-forward "non-loopy" solution of your problem from the top of my head. I would be interested to see one, too. Markus -- Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html
Open this post in threaded view
|

## Re: a difficult homework

 On Thu, Aug 22, 2019 at 8:55 AM mmuetzel <[hidden email]> wrote:If you really have to avoid the loop, you could try to use "arrayfun" with an anonymous function (which kind of hides the loop). I can't think of a straight-forward "non-loopy" solution of your problem from the top of my head. I would be interested to see one, too. MarkusI have a working solution, but is this a homework assignment?  I'm hesitant to just post the full code.  The solution is almost always to spread the looped operation over an expanded array (here it can be kept to two dimensions. I've frequently made use of n-dimensional operations, and broadcast projection for this. for large data sets you have to be careful, as you're trading time for memory)you're working with you c matrix as a single column of data, and looping 4 times, once over each a (and b, but they're looped together, so it's only one extra dimension).  you can make four parallel copies of the c matrix, and then apply the min function.  a common way to do quick array duplication is:c = c(:);cc = c .* ones(1,4);    or more generally cc = c.* ones(1,numel(a));then, you notice that min and max functions apply only in one dimension.  so min(cc) will give you the min of each column.  the tricky part is that you aren't taking the min of the full column, just a portion of it.  so you'll need to figure out how to sample only a portion of each column, as specified in a and b, maybe using logical indexing, and take the min over that. this is complicated by it being a different length portion each time.
Open this post in threaded view
|

## RE: a difficult homework

 Alternatively, you could use ACCUMARRAY (with subscripts created by REPELEM) to apply MIN to groups of values extracted from 'c'.  The only slightly non-trivial bit is generating the indices into 'c', but there are very standard CUMSUM-based ways of doing so.   Regards,   Bård Skaflestad   SINTEF Digital, Mathematics & Cybernetics Computational Geosciences group   From: Help-octave On Behalf Of Nicholas Jankowski Sent: Thursday, August 22, 2019 4:07 PM To: mmuetzel <[hidden email]>; shivax <[hidden email]>; LucaLuca <[hidden email]> Cc: Help GNU Octave <[hidden email]> Subject: Re: a difficult homework   On Thu, Aug 22, 2019 at 8:55 AM mmuetzel <[hidden email]> wrote: If you really have to avoid the loop, you could try to use "arrayfun" with an anonymous function (which kind of hides the loop). I can't think of a straight-forward "non-loopy" solution of your problem from the top of my head. I would be interested to see one, too. Markus I have a working solution, but is this a homework assignment?  I'm hesitant to just post the full code.     The solution is almost always to spread the looped operation over an expanded array (here it can be kept to two dimensions. I've frequently made use of n-dimensional operations, and broadcast projection for this. for large data sets you have to be careful, as you're trading time for memory)   you're working with you c matrix as a single column of data, and looping 4 times, once over each a (and b, but they're looped together, so it's only one extra dimension).  you can make four parallel copies of the c matrix, and then apply the min function.  a common way to do quick array duplication is:   c = c(:); cc = c .* ones(1,4);       or more generally    cc = c.* ones(1,numel(a));   then, you notice that min and max functions apply only in one dimension.  so min(cc) will give you the min of each column.     the tricky part is that you aren't taking the min of the full column, just a portion of it.  so you'll need to figure out how to sample only a portion of each column, as specified in a and b, maybe using logical indexing, and take the min over that. this is complicated by it being a different length portion each time.
Open this post in threaded view
|

## RE: a difficult homework

 thank you for answer mmuetzel : can you give me an example to use arrayfun with mine? nrjank : i can't use your method because i need to matrix so large...and i'm afraid to finish memory Bård Skaflestad:  can you give me an example to use your method with mine? thank for patience -- Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html
Open this post in threaded view
|

## Re: a difficult homework

 On Thu, Aug 22, 2019 at 1:46 PM shivax via Help list for GNU Octave <[hidden email]> wrote:nrjank : i can't use your method because i need to matrix so large...and i'm afraid to finish memoryhow large does yours need to be?   It would take exceptionally large amounts of data to hit memory limits for something you showed that can be done in 2D.  example: I had another problem where I was manipulating arrays of 2x2 matrices. I used broadcast projectionof the matrices along 'axes' for time, length, frequency, and eigen space, and then additional artificial axes to avoid loops, simplify calculations, etc. with arbitrary refinement for convergence, especially with poorly chosen initial base, I found out how to quickly exhaust memory on a 32bit system.  (a) Most of us are now working on 64bit systems with much more accessible memory, (b) estimating memory use is fairly simple and can usually be avoided. I was just lazy.
Open this post in threaded view
|

## Re: a difficult homework

 hi nrjank , thank you for answer i found this solution : (i try to use matrix this large matrix but the PC stops me for lack of memory...i use pc portable intel i-5 with 8 giga Ram )     a1=fix(rand(225).*100);  a=a1(:);  b=1:50624;  b=b';  c=(1:50624)+1;  c=c';    cc=a.*ones(1,numel(b));     %d=cc(b:end)  g=(1:rows(cc))'.*ones(1,numel(b));   g1=g>=b&g<=c;    e=min(~g1.*max(cc).+g1.*cc) -- Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html
Open this post in threaded view
|

## Re: a difficult homework

 On Sun, Aug 25, 2019 at 12:27 AM shivax via Help list for GNU Octave <[hidden email]> wrote:hi nrjank , thank you for answer i found this solution : (i try to use matrix this large matrix but the PC stops me for lack of memory...i use pc portable intel i-5 with 8 giga Ram )   a1=fix(rand(225).*100);  a=a1(:);  b=1:50624;  b=b';  c=(1:50624)+1;  c=c';  cc=a.*ones(1,numel(b));   %d=cc(b:end)  g=(1:rows(cc))'.*ones(1,numel(b));   g1=g>=b&g<=c;  e=min(~g1.*max(cc).+g1.*cc) Since you're only dealing with integers between 0 and 100, if you change the first line to:a1 = int8(rand(225).*100);you will significantly decrease the memory required. again, later, g=(uint16(1:rows(cc)))'.*ones(1,numel(b));you can use the 'whos' command to see the variable memory usage. cc      50625x50624             2562840000  int8  g       50625x50624             5125680000  uint16this gets to your next line which produces an error:>> g1=g>=b&g<=c;error: mx_el_ge: nonconformant arguments (op1 is 50625x50624, op2 is 50624x1)so you should re-examine what you're trying to do there.
Open this post in threaded view
|

## Re: a difficult homework

 sorry .. I made a mistake in writing the code The correct version is: (I recreated an exercise with the number of numbers of my work..this is a test to show you that I can't run it for memory) B=5000; a1=fix(rand(B).*100);  a=a1(:);  b=1:(B*B-1);  c=((1:(B*B-1)))+1; size(b) size(c)  cc=uint16(a.*ones(1,numel(b)));   %d=cc(b:end)  g=uint16((1:rows(cc))'.*ones(1,numel(b)));   g1=g>=b&g<=c;  e=min(~g1.*max(cc).+g1.*cc) error: out of memory or dimension too large for Octave's index type error: called from     prova2 at line 8 column 4 i used unint16..but the result is the same..:( if i try this code with B=50 it runny perfectly -- Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html
Open this post in threaded view
|

## Re: a difficult homework

 On Tue, Aug 27, 2019 at 8:24 AM shivax via Help list for GNU Octave <[hidden email]> wrote:sorry .. I made a mistake in writing the code The correct version is: (I recreated an exercise with the number of numbers of my work..this is a test to show you that I can't run it for memory) B=5000; a1=fix(rand(B).*100);  a=a1(:);  b=1:(B*B-1);  c=((1:(B*B-1)))+1; size(b) size(c)  cc=uint16(a.*ones(1,numel(b)));   %d=cc(b:end)  g=uint16((1:rows(cc))'.*ones(1,numel(b)));   g1=g>=b&g<=c;  e=min(~g1.*max(cc).+g1.*cc) error: out of memory or dimension too large for Octave's index type error: called from     prova2 at line 8 column 4 i used unint16..but the result is the same..:( if i try this code with B=50 it runny perfectly ok, since it runs for  B=50 and you've avoided looping, it seems you've managed to solve the problem as you first presented it.  there are numerous computer algorithms that will work perfectly well for certain data sizes but require completely different approaches for 'big data'. Perhaps you could share the actual problem you're trying to solve.  I feel like we could keep proposing an infinite number of suggestions that would lead you in the wrong direction if we don't understand what it is that the homework is trying to teach you. Is the goal to work on code vectorization, to manage large dataset problems, etc.  While we don't want to solve the actual problem for you, we should be able to provide more useful guidance.
Open this post in threaded view
|

## Re: a difficult homework

 I simplified the problem with an example code .. the only difference is that the vector b and the vector c are not as in the example (1: x) but they are pointers to the position of the base matrix Now if I have to bring back my original code I do a little effort .. and it's very long .. and it's divided by modules so I find it hard to bring back all the code executed However I can tell you what it is for or the basic matrix is a financial curve and what I want to find is the distance between different points of the curve .. These points are given to me by another function that I built that takes into account many factors Excuse for my bad english (i use google translate) -- Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html
Open this post in threaded view
|

## Re: a difficult homework

 On Tue, Aug 27, 2019 at 11:40 AM shivax via Help list for GNU Octave <[hidden email]> wrote:I simplified the problem with an example code .. the only difference is that the vector b and the vector c are not as in the example (1: x) but they are pointers to the position of the base matrixok, so if we look back at your original example:you have a matrix, and you have two sets of points that refer to locations in that matrix.  the distance between these locations can vary, and you want to perform some operations on the subset of the matrix between those locations.Assuming you have other functions that the create matrix, c, and vectors of points, a and b, the real questions are 'what is that operation' (in your first example it was the min operation. is that not actually the case? in the last email you said 'calculate the distance'?)   and 'are there any other factors/requirements/constraints?'  (how large is the data set? how large are c, a, and b (separately) allowed to be? are they all int type? is b always larger than a? is there anything unique about c?)Two useful things to read.  (1) What we're getting into falls into X-Y problems.  Where you are asking how to solve X when you're actual problem is Y, because you hink solving X will solve Y, and that isn't always true.  see:  http://xyproblem.info(2) please use 'bottom posting' on this help list.  it makes it easier for others to follow the conversation.  See http://www.idallen.com/topposting.html   (if you are using Gmail, you have to click on the 'three dots' to show previous replies when you type your message.)
Open this post in threaded view
|

## Re: a difficult homework

 now i try to expian better NEW EXAMPLE: a= Matrix of all equity 10  4   13   20 25  -6  10   15 13  10   8   17 8   8   15   22 b and c is a matrix position of a  c=     1    5    9   13     2    6   10   14     3    7   11   15     4    8   12   16  b=     1    5    9   13     2    5    9   13     2    6   9   14     3    7   11   15  c: is a matrix of "a" linear position       1 a(1,1)   5 a(1,2)    9 a(1,3)   13 a(1,4)     2 a(2,1)   6 a(2,2)   10 a(2,3)   14 a(2,4)     3 a(3,1)   7 a(3,2)   11 a(3,3)   15 a(3,4)     4 a(4,1)   8 a(4,2)   12 a(4,3)   16 a(4,4)    b: is a matrix of "a" linear position    1 a(1,1)   5 a(1,2)    9 a(1,3)   13 a(1,4)     2 a(2,1)   5 a(2,2)   9 a(2,3)    13 a(2,4)     2 a(3,1)   6 a(3,2)   9 a(3,3)   14 a(3,4)     3 a(4,1)   7 a(4,2)   11 a(4,3)   15 a(4,4) I thought of creating the matrix c to analyze the minimum of the equity between the 2 points why do I need the minimums between the various points of the matrix a? because a is the equity matrix..and I want to find the minimum peak between the various surveys (positions) using matrix b and matrix c - the elements of b are less than a -the element of b and c are egual/lower of length(a)  (because are position of a) - a is a matrix so large.. the matrix a is fed from time therefore the dimension is variable. However at present it occupies 5000 (row) * 88 (columns) size I don't think it can exceed 10,000 * 500 that's all ... I think I was exhaustive ( 55/5000 I hope I didn't make mistakes in rebuilding the matrices) -- Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html
Open this post in threaded view
|

## Re: a difficult homework

 an error: - the elements of b are less than a    ===> delete it! -- Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html
Open this post in threaded view
|

## Re: a difficult homework

 In reply to this post by Octave - General mailing list On Tue, Aug 27, 2019 at 2:42 PM shivax via Help list for GNU Octave <[hidden email]> wrote:now i try to expian better NEW EXAMPLE: ok.  now what do you define as a successful solution?  Your initial, very simple but effective script with loops works. it should work for very large arrays. It may not be the fastest on octave. Matlab is probably much faster because it has a working JIT (Just In Time) compiler that knows how to optimize loops.  In the script that I threw together, it removed the loops but only sped things up by about 30% for small cases. I suspect that might increase as the data size increases, but as you have seen, we're trading memory for speed. We can only hold so much data at one time. Your looping case holds very little. Projection operations to parallelize operation are not memory efficient. They multiply, and often duplicate, the data in memory to try to operate on it faster in parallel. For any solution and a particular data set you could tabulate speed and peak memory requirement to compare them and decide which is best.What are your requirements for 'success' on the homework? You've created two working algorithms. Is vectorization or speed a requirement?