Environment

• Pythonnet version: 2.3.0
• Python version: 3.6
• Operating System: Windows 10 x64

Details

I am trying to generate statically-typed wrappers for Python libraries, particularly TensorFlow (see Gradient).

Since I am automatically generating .NET-side types for wrappers, I can make their methods to use my custom converter written in C# to convert function arguments to PyObject instances. And then for any result values, I can convert PyObject instances back into my wrapper types.

Problem I am facing is when a user of my wrapper wants to inherit from one of the Python-derived classes, and override and/or extend its behavior.

An artificial sample:

// this part of my library is auto-generated
// represents tensorflow module
class tf {
  static dynamic tf = Py.Import("tensorflow");
  // set_default_session - made up method
  static void SetDefaultSession(Session session) => tf.set_default_session(session.underlyingSession);

  // made up method
  static void ComputeUsingDefaultSession() => tf.compute_using_default_session();
}

// represents tf.Tensor
class Tensor { PyObject underlyingTensor; }
// represents tf.Session
class Session {
  PyObject underlyingSession;
  virtual object Run(Tensor tensor) {
    var pyTensor = ConvertToPyObject(tensor); // simply does tensor.underlyingTensor here
    dynamic pySession = this.underlyingSession;
    var result = pySession.Run(pyTensor);
    return ConvertFromPyObject(result); // wraps any returned object into one of generated classes
  }
}

// this code is what user of my library wants to do:
class MyBetterSession : Session {
  override object Run(Tensor tensor) {
    ... here he writes custom code to run a Tensor ...
  }
}

// an attempts to use the above (functions made up):
var simpleSession = new Session();
tf.SetDefaultSession(simpleSession); // OK
tf.ComputeUsingDefaultSession(); // SUCCEEDS

var betterSession = new MyBetterSession();
tf.SetDefaultSession(betterSession); // OK
tf.ComputeUsingDefaultSession(); // FAILS

The last line will fail, because Python will attempt to call MyBetterSession.Run with Python's class tf.Tensor, but the method actually expects wrapped class Tensor. I need to somehow tell Python or rather Python.NET to invoke my ConvertFromPyObject on the argument, before trying to find a matching overload.

I looked into Python.NET source, and the corresponding objects are MethodBinding and MethodObject, however, neither seem to provide any extensibility.

Now I am considering several approaches to the problem, and I just wanted to discuss them with Python.NET team, as some of them involve modification of Python.NET.

1. Do not change anything in Python.NET, and for every user class like MyBetterSession generate a pythonic wrapper with the same set of methods, but replacing all parameter and return types with PyObject. It would require one of the following:

• System.Reflection.Emit, which is a very heavy dependency, and also not very pleasant to work with
• Roslyn, which is much heavier, but moderately OK to work with

2. I noticed, that while Python.NET supports representing any Python object as dynamic, it does not actually support passing dynamic objects (e.g. IDynamicMetaObjectProvider instances) back to Python. I mean, it would pass them, but if Python would try to access a dynamic attribute, it would not attempt to call TryGetMember. I could potentially implement something similar to ClassBase specifically for wrapping IDynamicMetaObjectProvider instances. Then it is much easier to implement its members, that would simply wrap PyObject arguments before forwarding them to an instance of MyBetterSession.

3. Have Python.NET directly expose some low-level interface, that would enable hooking into Python.NET's marshaling and, possibly, also method binding processes.