Suggestion on priorities for improving pytave

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Suggestion on priorities for improving pytave

Abhinav Tripathi

Hi, 
I have tried many more things but still am stuck at the point where Tatsuro was stuck on the building pytave on windows front (or probably I couldn't even reach that point).
.
So, I have thought to start on some other improvements to pytave.
I wanted to know your thoughts on priority/acceptance of any of the following changes:
.
1- remove ALL the numpy stuff from octave_to_python.cc in order to store everything directly into corresponding python types. Might take some time to complete but should be straight forward.
2 - add a new 'pystore.{h,  cc}' to allow users to store a variable into python directly. In many cases we have to either make a proxy function and use pycall to store values in a list by calling that proxy function. This new function would really help.
3 -  extend pycall.cc to support arbitrary number of parameters by storing all the prameters in a python array (pystore could come in handy here), say '_tmp_args_array' and then use something like pyeval("function_to_call(*_tmp_args_array)")
The same capability can be used in @pyobject class to call functions with arbitrary number of arguments.
4 -  Start getting rid of all the boost stuff from all the files. This might take much time and can probably prove a little dificult at places. But since Mike said he wanted pytave to be not dependent on boost, it can be a place to start.
.
If you have any other tasks for improving pytave/octsympy that should be prioritized then pls feel free to add them.
Also I mentioned the above tasks in decreasing order of priority (as per me).
If you wish, remove and/or change the order of tasks.
.
Regards, 
Abhinav

Reply | Threaded
Open this post in threaded view
|

Re: Suggestion on priorities for improving pytave

Colin Macdonald-2
On 06/07/16 14:01, Abhinav Tripathi wrote:
> 1- remove ALL the numpy stuff from octave_to_python.cc in order to store
> everything directly into corresponding python types. Might take some
> time to complete but should be straight forward.

I don't think we want to drop numpy.  I think the numpy dependency is
the right thing for 2D/3D/nD arrays.  But I also think scalars should
perhaps not be wrapped in numpy.  I think we have a bitbucket issue
about this?

The other priorities sound fine to me.  I agree de-boosting could be
time-consuming.

I'll follow-up about Symbolic priorities later...

Colin

Reply | Threaded
Open this post in threaded view
|

Re: Suggestion on priorities for improving pytave

Mike Miller-4
In reply to this post by Abhinav Tripathi
On Thu, Jul 07, 2016 at 02:31:18 +0530, Abhinav Tripathi wrote:
> I have tried many more things but still am stuck at the point where Tatsuro
> was stuck on the building pytave on windows front (or probably I couldn't
> even reach that point).

Please make some notes, at some point it might be nice to get a summary
of what has been done so far and what the blockers are, either in a
README or on the wiki.

> So, I have thought to start on some other improvements to pytave.
> I wanted to know your thoughts on priority/acceptance of any of the
> following changes:
> .
> 1- remove ALL the numpy stuff from octave_to_python.cc in order to store
> everything directly into corresponding python types. Might take some time
> to complete but should be straight forward.

Agree with Colin, I think we may want to keep numpy for arrays. Needs
some thought put into it. It may be possible to remove it if it's easy
enough to construct a numpy array from an Octave array. But that mode of
operation, whether done implicitly or explicitly, should remain easy.

Once the pyobject stuff is merged, though, you could look at removing
some of the other non-array conversions, for example the automatic
conversions to cell arrays and struct arrays.

> 2 - add a new 'pystore.{h,  cc}' to allow users to store a variable into
> python directly. In many cases we have to either make a proxy function and
> use pycall to store values in a list by calling that proxy function. This
> new function would really help.

Sorry, I'm not sure I follow. Can you demonstrate with some code what
you mean by this? I'm all for making it easier to convert Octave data
types into Python, just not sure what use case you are talking about.

> 3 -  extend pycall.cc to support arbitrary number of parameters by storing
> all the prameters in a python array (pystore could come in handy here), say
> '_tmp_args_array' and then use something like
> pyeval("function_to_call(*_tmp_args_array)")
> The same capability can be used in @pyobject class to call functions with
> arbitrary number of arguments.

Yes, not sure about the method you describe, but definitely adding
support for arbitary argument lists, and also unpacking multiple return
values.

Maybe some kind of support for function kwargs as well? A scalar struct
to pass named parameters?

> 4 -  Start getting rid of all the boost stuff from all the files. This
> might take much time and can probably prove a little dificult at places.
> But since Mike said he wanted pytave to be not dependent on boost, it can
> be a place to start.

Yes, and this could easily be done incrementally. As you've seen and
started doing in your PRs, you can mix and match the Python native API
with the Boost.Python API to some extent.

I have been idle on pytave for the past week or so, but I am hoping to
get back into it and review and merge all of the work you and Colin have
been putting into it. Once I get all of the outstanding PRs and issues
handled, I will try to brainstorm some more immediate ideas for
improvements.

Thank you for your contributions so far!

--
mike

Reply | Threaded
Open this post in threaded view
|

Re: Suggestion on priorities for improving pytave

Abhinav Tripathi
Colin Macdonald <[hidden email][hidden email]> wrote:
I don't think we want to drop numpy.  I think the numpy dependency is the right thing for 2D/3D/nD arrays.  But I also think scalars should perhaps not be wrapped in numpy.  I think we have a bitbucket issue about this?

Oh. I forgot to consider nD arrays. Yes there is a bitbucket issue about scalar. I'll get to it then.
 
The other priorities sound fine to me.  I agree de-boosting could be time-consuming.

I'll follow-up about Symbolic priorities later...


On Thu, Jul 7, 2016 at 4:31 AM, Mike Miller <[hidden email]> wrote:

Please make some notes, at some point it might be nice to get a summary
of what has been done so far and what the blockers are, either in a
README or on the wiki.

Sure. I will make a README and then try to upload the contents to a wiki page too.
 
Agree with Colin, I think we may want to keep numpy for arrays. Needs
some thought put into it. It may be possible to remove it if it's easy
enough to construct a numpy array from an Octave array. But that mode of
operation, whether done implicitly or explicitly, should remain easy.

Wouldn't it be the same as now? I mean if we just add a special case for 1x1 arrays then everything else remain as is, right?
 
Once the pyobject stuff is merged, though, you could look at removing
some of the other non-array conversions, for example the automatic
conversions to cell arrays and struct arrays.

Definitely.
 
Sorry, I'm not sure I follow. Can you demonstrate with some code what
you mean by this? I'm all for making it easier to convert Octave data
types into Python, just not sure what use case you are talking about.

Currently if I have to store an octave variable in python with a particular name. I will do:
----------------------------------------------------------------------------------
pyexec ([ "global var_with_req_name" ...
              "def func_to_fill_var(x):" ...
              "    global var_with_req_name" ...
              "    var_with_req_name = x" ]);
pycall ("func_to_fill_var", req_octave_value);
----------------------------------------------------------------------------------
It seems a little too much to do for just storing a variable, instead I would like to do:
----------------------------------------------------------------------------------
pystore ("var_with_req_name", req_octave_value);
----------------------------------------------------------------------------------

Yes, not sure about the method you describe, but definitely adding
support for arbitary argument lists, and also unpacking multiple return
values.

I just meant we could store all the arguments to a python variable (tuple/list) using pystore and then call then function with unpacking that python variable.
 
Maybe some kind of support for function kwargs as well? A scalar struct
to pass named parameters?

This could prove to be a bit tricky. I tried but couldn't use map unpacking to call a python function.

Yes, and this could easily be done incrementally. As you've seen and
started doing in your PRs, you can mix and match the Python native API
with the Boost.Python API to some extent.

Sure.
 
I have been idle on pytave for the past week or so, but I am hoping to
get back into it and review and merge all of the work you and Colin have
been putting into it. Once I get all of the outstanding PRs and issues
handled, I will try to brainstorm some more immediate ideas for
improvements.

Thank you for your contributions so far!

--
mike

It had been nice working with both of you. I got a lot of support from the octave community which made it easier to work as per the timeline...
.
Abhinav
Reply | Threaded
Open this post in threaded view
|

Re: Suggestion on priorities for improving pytave

Colin Macdonald-2
On 07/07/16 00:06, Abhinav Tripathi wrote:
> Wouldn't it be the same as now? I mean if we just add a special case for
> 1x1 arrays then everything else remain as is, right?

At least for now, I think you're right.

> pystore ("var_with_req_name", req_octave_value);

Its possible that we don't want to expose such a thing (although we may
want it internally).

Why?  We don't really want the user to have to track a mental model of
variables in Python space.  She should just have some Octave objects
(doubles, strings) and some @pyobjects (dicts, modules, whatever).  And
then do things to those objects.

>     Maybe some kind of support for function kwargs as well? A scalar struct
>     to pass named parameters?
>
> This could prove to be a bit tricky. I tried but couldn't use map
> unpacking to call a python function.

Please file an issue for this kwargs stuff.  It'll be important, even if
we can't do it right now.

Colin

Reply | Threaded
Open this post in threaded view
|

Re: Suggestion on priorities for improving pytave

Abhinav Tripathi


On Jul 7, 2016 1:01 PM, "Colin Macdonald" <[hidden email]> wrote:
>
>
> At least for now, I think you're right.
>
>
>> pystore ("var_with_req_name", req_octave_value);
>
>
> Its possible that we don't want to expose such a thing (although we may want it internally).
>
> Why?  We don't really want the user to have to track a mental model of variables in Python space.  She should just have some Octave objects (doubles, strings) and some @pyobjects (dicts, modules, whatever).  And then do things to those objects.
>

Ohk. I understand. But still, if it an extra feature and we don't let the user rely on it. It can be helpful?!

>>     Maybe some kind of support for function kwargs as well? A scalar struct
>>     to pass named parameters?
>>
>> This could prove to be a bit tricky. I tried but couldn't use map
>> unpacking to call a python function.
>
>
> Please file an issue for this kwargs stuff.  It'll be important, even if we can't do it right now.
>
> Colin

Oh sorry. I think I missed kwargs thing and only remembered scalar struct thing.
I have used/created functions with variable arguments in c++ but never in python. I see how to do that.
I'll file an issues on bitbucket.

Abhinav

Reply | Threaded
Open this post in threaded view
|

Re: Suggestion on priorities for improving pytave

Mike Miller-4
On Thu, Jul 07, 2016 at 21:52:55 +0530, Abhinav Tripathi wrote:

> On Jul 7, 2016 1:01 PM, "Colin Macdonald" <[hidden email]> wrote:
> >
> >
> > At least for now, I think you're right.
> >
> >
> >> pystore ("var_with_req_name", req_octave_value);
> >
> >
> > Its possible that we don't want to expose such a thing (although we may
> want it internally).
> >
> > Why?  We don't really want the user to have to track a mental model of
> variables in Python space.  She should just have some Octave objects
> (doubles, strings) and some @pyobjects (dicts, modules, whatever).  And
> then do things to those objects.
> >
>
> Ohk. I understand. But still, if it an extra feature and we don't let the
> user rely on it. It can be helpful?!

I agree with Colin, and at least for now I think it would be best if we
try to use the Python interface the way we want users to use it
(dogfooding).

If it becomes obvious that this is a needed feature then we could add
it. But I hope that we can get to the point where we don't need to
assign names to values in Python at all, and just use pyobjects.

> >>     Maybe some kind of support for function kwargs as well? A scalar
> struct
> >>     to pass named parameters?
> >>
> >> This could prove to be a bit tricky. I tried but couldn't use map
> >> unpacking to call a python function.
> >
> >
> > Please file an issue for this kwargs stuff.  It'll be important, even if
> we can't do it right now.
> >
> > Colin
>
> Oh sorry. I think I missed kwargs thing and only remembered scalar struct
> thing.
> I have used/created functions with variable arguments in c++ but never in
> python. I see how to do that.
> I'll file an issues on bitbucket.

Thanks. I think the first step would be just discussing what kind of
Octave syntax we want to use to be able to pass named parameters to
Python functions, I threw out the idea of a scalar struct with
name/value pairs, but maybe other ideas?

--
mike

Reply | Threaded
Open this post in threaded view
|

Re: Suggestion on priorities for improving pytave

Abhinav Tripathi


On Jul 7, 2016 10:14 PM, "Mike Miller" <[hidden email]> wrote:
>
> I agree with Colin, and at least for now I think it would be best if we
> try to use the Python interface the way we want users to use it
> (dogfooding).
>
> If it becomes obvious that this is a needed feature then we could add
> it. But I hope that we can get to the point where we don't need to
> assign names to values in Python at all, and just use pyobjects.
>

Ohk. I just thought we would need naming variables in cases when we have to call some functions as in symbolic. We store every parameter in '_ins' list in python and have to do the pyexec then pycall mechanism that I stated earlier.

> Thanks. I think the first step would be just discussing what kind of
> Octave syntax we want to use to be able to pass named parameters to
> Python functions, I threw out the idea of a scalar struct with
> name/value pairs, but maybe other ideas?
>
> --
> mike

For the octave syntax, we could define a new function that would force unpacking of the pyobject before calling the function.
Somthing like -
pycall("func",  unpack(pyobject_of_a_list))
.
That's just from the top of my head. I'll try to think and will let you know if I get a better idea about the syntax to follow. Also, I'll think about other ideas that could be implemented.
.
Abhinav