more import tests

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

more import tests

John W. Eaton
Administrator
I'm attaching a tar file with some more tests.  Unpack it, cd to the
resulting import-test directory and run each of the s*.m scripts in a
separate matlab session.  There is a doit.sh script that attempts to do
that.

The things I'm looking for are

   * does import affect all function calls, or just the ones that follow
the import statement?

   * is there a difference in behavior for command-style vs.
function-style calls to import?

   * is there a difference when the import item is stored in a variable
instead of a literal string?

   * what happens if the import statement is hidden inside an eval
statement?

   * what happens if there is a user-defined import function in the load
path?

It would be nice to see the results for the latest version of Matlab.

jwe

import-test.tar.gz (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: more import tests

Rik-4
On 08/10/2017 06:20 AM, [hidden email] wrote:
Subject:
more import tests
From:
"John W. Eaton" [hidden email]
Date:
08/10/2017 06:19 AM
To:
Octave Maintainers List [hidden email]
CC:
[hidden email]
List-Post:
[hidden email]
Precedence:
list
MIME-Version:
1.0
Message-ID:
[hidden email]
Content-Type:
multipart/mixed; boundary="------------2AFEE2CD8E5ED8D8CE023953"
Message:
5

I'm attaching a tar file with some more tests.  Unpack it, cd to the resulting import-test directory and run each of the s*.m scripts in a separate matlab session.  There is a doit.sh script that attempts to do that.

The things I'm looking for are

  * does import affect all function calls, or just the ones that follow the import statement?

On this point, it seems that the order of the import statement and code which uses the imported name is immaterial.  Having the import statement after the first use of an imported name, or having an import statement protected by a conditional which is not executed, made no difference in the tests I asked for.  I can't be certain of course, but this suggests at least a 2-pass compilation step for each function where the first step is to expand out import statements.  The expansion would appear to occur before detailed parsing because otherwise the parser could recognize the fact that the import is inside a conditional and issue a warning or error.

According to the documentation, import is localized to function scope.  The documentation mentions that if the definition is needed in several subfunctions then the import statement must be repeated for each function.  On the other hand, nested functions automatically have access to the imported names.  This suggests, again, that the parsing unit is one function and that an expansion step of imported names occurs throughout the function which necessarily applies to the nested function text.


  * is there a difference in behavior for command-style vs. function-style calls to import?

  * is there a difference when the import item is stored in a variable instead of a literal string?

  * what happens if the import statement is hidden inside an eval statement?

  * what happens if there is a user-defined import function in the load path?

Since one can re-define import to be a variable, I don't see why one couldn't write an import function which supplants the Matlab-provided one.  It's bad language design, however, because programmers will write code assuming import works the way it does in the documentation, but a particular installation may have an import function available and then all the code breaks.

In Octave, re-defining a built-in function at least produces a warning.  For the following file, sin.m,

function retval = sin (x)
  retval = cos (x);
endfunction

I get "warning: function ./sin.m shadows a built-in function".  But I get no such warning if I create a new m-file import.m which shadows the current one in scripts/path/import.m.  Maybe that suggests implementing import as a C++ function.

--Rik

It would be nice to see the results for the latest version of Matlab.

jwe

Reply | Threaded
Open this post in threaded view
|

Re: more import tests

John W. Eaton
Administrator
On 08/10/2017 11:59 AM, Rik wrote:

> On this point, it seems that the order of the import statement and code
> which uses the imported name is immaterial.  Having the import statement
> after the first use of an imported name, or having an import statement
> protected by a conditional which is not executed, made no difference in
> the tests I asked for.

I'd still like to see all the cases evaluated.  Matlab behavior has
surprised me before...

Also, in case it wasn't clear from my earlier message, I'd like for each
test script to be run in a new Matlab session.  That way we can be sure
that there are no issues due to caching from previous tests.

> According to the documentation,

Haha, good one!  ;-)

jwe


Reply | Threaded
Open this post in threaded view
|

Re: more import tests

NJank
in process, should have it up shortly

On Thu, Aug 10, 2017 at 1:13 PM, John W. Eaton <[hidden email]> wrote:

> On 08/10/2017 11:59 AM, Rik wrote:
>
>> On this point, it seems that the order of the import statement and code
>> which uses the imported name is immaterial.  Having the import statement
>> after the first use of an imported name, or having an import statement
>> protected by a conditional which is not executed, made no difference in the
>> tests I asked for.
>
>
> I'd still like to see all the cases evaluated.  Matlab behavior has
> surprised me before...
>
> Also, in case it wasn't clear from my earlier message, I'd like for each
> test script to be run in a new Matlab session.  That way we can be sure that
> there are no issues due to caching from previous tests.
>
>> According to the documentation,
>
>
> Haha, good one!  ;-)
>
> jwe
>
>

Reply | Threaded
Open this post in threaded view
|

Re: more import tests

NJank
In reply to this post by John W. Eaton
On Thu, Aug 10, 2017 at 9:19 AM, John W. Eaton <[hidden email]> wrote:
> There is a doit.sh script that attempts to do
> that.
>
which also tries to open 20 matlab sessions simultaneously making some
people's machines a bit unhappy. :)

diary file attached with Rik's tests followed by all of yours. each
one run with a separate instance by calling

>> !matlab &
>> exit

and then

>> diary('importtests.txt')
>> s2b

etc.

So after Rik's tests, since the diary call doesn't get recorded, each
section of yours will be separated by the following lines:

!matlab &
exit
s1b

etc

I checked that they were all there. let me know if anything wasn't clear.

importtests.txt (48K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: more import tests

John W. Eaton
Administrator
Thanks for running the tests.

Some things I noticed:

** The test of a user-defined import function shows that it is only
executed if it is called with the function-style syntax, but not if
called with command-style syntax.  For command-style syntax, it appears
that the user-defined import function is ignored.  Does that happen for
ANY other function name in Matlab?

** If the command-style syntax is used, then it appears to affect all
function calls, as Rik noted.  So when a command-style import statement
is parsed, the import name is added to a list that will affect function
lookup.  This can be done when the function that contains the import
statement is parsed or when it is evaluated.  Doing it at parse time
requires a lot more smarts for the parser because it must determine what
is a variable vs. what is a function.  Octave is not quite there yet.
Also, I need to test what happens when eval is used to give a symbol a
value when that symbol also exists as a function.  For example:

   import pkg.sin;
   eval ('sin = pi');
   sin (1)

Is pkg.sin called, or does sin (1) index the variable sin?  Does the
result change if sin is "declared" to be a variable?

   sin = [];
   import pkg.sin;
   eval ('sin = pi')
   sin (1)

I suspect that in the former case, the variable sin is ignored, but in
the latter case, sin is treated as a variable, not the pkg.sin function.

** If the function-style syntax is used, then import still has an
effect, but it behaves more like a normal function, and only changes the
lookup for symbols that appear after the call to import.

** If an import function isn't found, it's an error.

jwe


Reply | Threaded
Open this post in threaded view
|

Re: more import tests

Rik-4
On 08/10/2017 01:29 PM, John W. Eaton wrote:
> Thanks for running the tests.
>
> Some things I noticed:
>
> ** The test of a user-defined import function shows that it is only
> executed if it is called with the function-style syntax, but not if
> called with command-style syntax.  For command-style syntax, it appears
> that the user-defined import function is ignored.  Does that happen for
> ANY other function name in Matlab?

My guess is that this is unique to import.  I think what Matlab may do is
check at parse time whether the function or command-style syntax is being
used.  If it is command-style, then the parser knows that whatever follows
should be interpreted as a static string.  Since the string is static it
can be substituted at parse time into the code.  However, if the function
syntax is used then what follows could be a variable and the value of that
can only be known at evaluation time.  There would be even one more slight
optimization which would be to have the parser try and check for static
strings ('...') even when the function syntax was used.  If strings were
detected it could still substitute the values at parse-time.

Although Matlab says that the syntax "import pkg.*" is deprecated because
it could pull too many symbols into the current namespace, another reason
might be that this sort of expression could be too difficult to handle at
parse time.  It looks like a static string, but it really isn't.

>
> ** If the command-style syntax is used, then it appears to affect all
> function calls, as Rik noted.  So when a command-style import statement
> is parsed, the import name is added to a list that will affect function
> lookup.  This can be done when the function that contains the import
> statement is parsed or when it is evaluated.  Doing it at parse time
> requires a lot more smarts for the parser because it must determine what
> is a variable vs. what is a function.  Octave is not quite there yet.

I already made my guess which is that command-style => parse time while
function-style => evaluation time.  If Matlab is really doing some sort of
pre-processing on the function code, then it may not be building an import
list for that function.  It would be useful to test

function tstfunc ()
  import containers.Map
  x = Map ();
  L = import ()
end

> Also, I need to test what happens when eval is used to give a symbol a
> value when that symbol also exists as a function.  For example:
>
>   import pkg.sin;
>   eval ('sin = pi');
>   sin (1)
>
> Is pkg.sin called, or does sin (1) index the variable sin?  Does the
> result change if sin is "declared" to be a variable?
>
>   sin = [];
>   import pkg.sin;
>   eval ('sin = pi')
>   sin (1)
>
> I suspect that in the former case, the variable sin is ignored, but in
> the latter case, sin is treated as a variable, not the pkg.sin function.
>

Yes, these two tests will be interesting.

--Rik

> ** If the function-style syntax is used, then import still has an effect,
> but it behaves more like a normal function, and only changes the lookup
> for symbols that appear after the call to import.
>
> ** If an import function isn't found, it's an error.
>
> jwe
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: more import tests

NJank


On Tue, Aug 15, 2017 at 2:27 AM, Rik <[hidden email]> wrote:
On 08/10/2017 01:29 PM, John W. Eaton wrote:
 
 
It would be useful to test

function tstfunc ()
  import containers.Map
  x = Map ();
  L = import ()
end

-----------------
>> tstfunc

L =

  cell

    'containers.Map'
-----------------

 
> Also, I need to test what happens when eval is used to give a symbol a
> value when that symbol also exists as a function.  For example:
>
>   import pkg.sin;
>   eval ('sin = pi');
>   sin (1)

pkg.sin doesn't exist, but maybe this with answers your question with Map?

-------------------------
>> import containers.Map
>> eval ('Map = pi')

Map =

    3.1416

>> Map (1)

ans =

    3.1416

--------------------------

 
>
> Is pkg.sin called, or does sin (1) index the variable sin?  Does the
> result change if sin is "declared" to be a variable?
>
>   sin = [];
>   import pkg.sin;
>   eval ('sin = pi')
>   sin (1)

Again with Map:

-----------------
>> Map = []

Map =

     []

>> import containers.Map
>> eval ('Map = pi')

Map =

    3.1416

>> Map (1)

ans =

    3.1416

>>
-----------------