automating GUI with java robot on Octave

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

automating GUI with java robot on Octave

Erdem Pulcu
Dear Members,

I am trying to implement a function which was working perfectly fine in Matlab, and I am trying to implement the same in Octave. I am using a Windows 10 PC with Octave 4.2.1.

Initially when I was installing the Octave for the first time it complained that JRE were not installed, but I fixed this issue by installing and rebooting the program, so it doesn't flag this up anymore.

The Matlab code that I want to run on Octane is below, it should implement some mouse clicks on a GUI in an automated fashion.

robot = java.awt.Robot;
screenSizes = get(0, 'MonitorPositions');
store(loops,2)=zeros;

for i=1:loops

%Mouse to upper left of primary monitor
robot.mouseMove(1, 1)

%Mouse to center of primary monitor
robot.mouseMove(mean(screenSizes(1,[1 3])),mean(screenSizes(1,[2 4]))) 

%Mouse to hardcoded point 200 pixels down and 500 pixels to the right
robot.mouseMove(500, 200)


robot.mouseMove(170,85);
pause(1.5)
import java.awt.Robot;
import java.awt.event.*;
mouse = Robot;

%[rest of the code is not relevant so deleted for simplicity]

end

when I run the script Octave says in the command window
error: 'java' undefined near line bla bla bla

This GUI automation is very crucial thing for me, so I highly appreciate any recommendations (please give step by step details, as I am not so familiar of using Octave or Java, although fairly ok with Matlab)

Looking forward to hear from you all

Erdem



_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: automating GUI with java robot on Octave

siko1056
Erdem Pulcu wrote
Dear Members,

I am trying to implement a function which was working perfectly fine in Matlab, and I am trying to implement the same in Octave. I am using a Windows 10 PC with Octave 4.2.1.

Initially when I was installing the Octave for the first time it complained that JRE were not installed, but I fixed this issue by installing and rebooting the program, so it doesn't flag this up anymore.

The Matlab code that I want to run on Octane is below, it should implement some mouse clicks on a GUI in an automated fashion.

robot = java.awt.Robot;
screenSizes = get(0, 'MonitorPositions');
store(loops,2)=zeros;

for i=1:loops

%Mouse to upper left of primary monitor
robot.mouseMove(1, 1)

%Mouse to center of primary monitor
robot.mouseMove(mean(screenSizes(1,[1 3])),mean(screenSizes(1,[2 4])))

%Mouse to hardcoded point 200 pixels down and 500 pixels to the right
robot.mouseMove(500, 200)


robot.mouseMove(170,85);
pause(1.5)
import java.awt.Robot;
import java.awt.event.*;
mouse = Robot;

%[rest of the code is not relevant so deleted for simplicity]

end

when I run the script Octave says in the command window
error: 'java' undefined near line bla bla bla

This GUI automation is very crucial thing for me, so I highly appreciate any recommendations (please give step by step details, as I am not so familiar of using Octave or Java, although fairly ok with Matlab)

Looking forward to hear from you all

Erdem
Dear Erdem,

The java support in Octave isn't as "native", as in Matlab. You cannot instantiate a Java class using "robot = java.awt.Robot", but "robot = javaObject ('java.awt.Robot');" works in both Octave and Matlab.

See [1] for details.

HTH,
Kai

[1]: https://www.gnu.org/software/octave/doc/interpreter/How-to-use-Java-from-within-Octave.html#How-to-use-Java-from-within-Octave
Reply | Threaded
Open this post in threaded view
|

Re: automating GUI with java robot on Octave

PhilipNienhuis
siko1056 wrote
Erdem Pulcu wrote
Dear Members,

I am trying to implement a function which was working perfectly fine in Matlab, and I am trying to implement the same in Octave. I am using a Windows 10 PC with Octave 4.2.1.

Initially when I was installing the Octave for the first time it complained that JRE were not installed, but I fixed this issue by installing and rebooting the program, so it doesn't flag this up anymore.

The Matlab code that I want to run on Octane is below, it should implement some mouse clicks on a GUI in an automated fashion.

robot = java.awt.Robot;
screenSizes = get(0, 'MonitorPositions');
store(loops,2)=zeros;

for i=1:loops

%Mouse to upper left of primary monitor
robot.mouseMove(1, 1)

%Mouse to center of primary monitor
robot.mouseMove(mean(screenSizes(1,[1 3])),mean(screenSizes(1,[2 4])))

%Mouse to hardcoded point 200 pixels down and 500 pixels to the right
robot.mouseMove(500, 200)


robot.mouseMove(170,85);
pause(1.5)
import java.awt.Robot;
import java.awt.event.*;
mouse = Robot;

%[rest of the code is not relevant so deleted for simplicity]

end

when I run the script Octave says in the command window
error: 'java' undefined near line bla bla bla

This GUI automation is very crucial thing for me, so I highly appreciate any recommendations (please give step by step details, as I am not so familiar of using Octave or Java, although fairly ok with Matlab)

Looking forward to hear from you all

Erdem
Dear Erdem,

The java support in Octave isn't as "native", as in Matlab. You cannot instantiate a Java class using "robot = java.awt.Robot", but "robot = javaObject ('java.awt.Robot');" works in both Octave and Matlab.

See [1] for details.

HTH,
Kai

[1]: https://www.gnu.org/software/octave/doc/interpreter/How-to-use-Java-from-within-Octave.html#How-to-use-Java-from-within-Octave
Plus, "import" isn't implemented (yet) in Octave.
For some more Java wishes see here:
https://savannah.gnu.org/task/?12601

Philip
Rik
Reply | Threaded
Open this post in threaded view
|

Re: automating GUI with java robot on Octave

Rik
I am trying to do something similar (i.e. UI control in Octave with the Java robot).

I want to send the key combination alt-space followed by x to maximize a figure window (see code below). I have two problems:
- It does not seem to actually send the key combination (nothing visible happens, but there is no error either)
- I can't get the key IDs from java.awt.event.KeyEvent

If anyone has an idea of how to solve this, I'dd be much obliged.

My code (snippet) so far:
(I use single quotes for strings, because otherwise Matlab start complaining)

%hardcoded values from java.awt.event.KeyEvent
VK_ALT=18;
VK_SPACE=32;
VK_X=88;

figure(hFig)
robot = javaObject('java.awt.Robot');
javaMethod('keyPress',robot,VK_ALT);     %// send ALT
javaMethod('keyPress',robot,VK_SPACE);   %// send SPACE
javaMethod('keyRelease',robot,VK_SPACE); %// release SPACE
javaMethod('keyRelease',robot,VK_ALT);   %// release ALT
javaMethod('keyPress',robot,VK_X);   %// send X
javaMethod('keyRelease',robot,VK_X); %// release X