ArrayRef of C++ types wrapped using CxxWrap

[gzhang8]

Thank you again for your time answering my questions.

When wrapping a library, I want to return a vector or array of C++ objects for checking values in Julia and further computing in C++. The types of the objects are wrapped using CxxWrap. I find an example to take ArrayRef of C++ objects as arguments, but haven't found a way to return such an array. Is there a way to do this with current release? Below is an example of what I am trying to do.

#include "jlcxx/jlcxx.hpp"

namespace cpp_types
{
  struct Foo {
    Foo() {
      id = 6;
      data = std::vector<double>(10);
    }
    
    Foo(int aa,
	jlcxx::ArrayRef<double,1> d)
      : id(aa),
	data(d.begin(),
	     d.end()){
    }
    
    int id;
    std::vector<double> data;
  };
}

JLCXX_MODULE define_julia_module(jlcxx::Module& types)
{
  using namespace cpp_types;

  types.add_type<Foo>("Foo")
    .constructor<int, jlcxx::ArrayRef<double,1>>()
    .constructor()
    .method("id", [](Foo& f) { return f.id; })
    .method("data", [](Foo& f) { return jlcxx::ArrayRef<double,1>(&(f.data[0]), f.data.size()); });
  
  // This doesn't compile
  types.method("print_foo_array", [] (jlcxx::ArrayRef<Foo, 1> x_arr) {
      for (int i = 0; i < x_arr.size(); i++) {
   	std::cout << x_arr[i].id << std::endl;
      }
    });

  // This works, but Julia side has no type information
  types.method("print_foo_array_as_any", [] (jlcxx::ArrayRef<jl_value_t*, 1> x_arr) {
      for (int i = 0; i < x_arr.size(); i++) {
	const Foo& f = *jlcxx::unbox_wrapped_ptr<Foo>(x_arr[i]);
   	std::cout << f.id << std::endl;
      }
    });

  // This compiles, but it gives null type pointer runtime error
  types.method("make_foo_array", [] (int size) {
      Foo* arr = new Foo[size];
      return jlcxx::make_julia_array(arr, size);
    });
}

I also tried jlcxx::static_type_mapping, but it doesn't compile.

[barche]

Unfortunately right now the second method is the only way. It should be doable to make the first method work, but it needs some reworking of ArrayRef, which would certainly be a good thing.

[ekinakyurek]

Hi,

I am sorry that I am copying a long code here because I couldn't make more abstractions with my current C++ knowledge.

In the below code I am defining ViewPointJulia type and wrap it in the bottom, I also want to use array of ViewPointJulia in SimStateJulia class, but I couldn't make it compile. Is it same problem with this issue or is there a way to do it in my case?

#include "jlcxx/jlcxx.hpp"
#include "jlcxx/tuple.hpp"
#include "jlcxx/const_array.hpp"
#include <iostream>
#include "MatterSim.hpp"

namespace mattersim {

  struct ViewPointJulia{
    ViewPointJulia(ViewpointPtr locptr) {
      viewpointId = locptr->viewpointId;
      ix = locptr->ix;
      point = std::make_tuple(locptr->point.x,locptr->point.y,locptr->point.z);
      rel_heading = locptr->rel_heading;
      rel_elevation = locptr->rel_elevation;
      rel_distance = locptr->rel_distance;
    };
    std::string viewpointId;
    unsigned int ix;
    std::tuple<double,double,double> point;
    double rel_heading;
    double rel_elevation;
    double rel_distance;
  };

 struct SimStateJulia {
    SimStateJulia(SimStatePtr state, bool renderingEnabled)
      : step{state->step},
        viewIndex{state->viewIndex},
        location{state->location},
        heading{state->heading},
        elevation{state->elevation} {
        scanId = state->scanId;
        for (auto viewpoint : state->navigableLocations) {
          navigableLocations.push_back(ViewPointJulia{viewpoint});
	};
       };

    std::string scanId;
    unsigned int step;
    unsigned int viewIndex;
    ViewPointJulia location; // I am not sure about this line too
    double heading;
    double elevation;
    jlcxx::ArrayRef<ViewPointJulia,1> navigableLocations; //????? error occurs here
  };

using namespace mattersim;

JLCXX_MODULE define_types_module(jlcxx::Module& types)
{
  types.add_type<ViewPointJulia>("ViewPointJulia")
    .constructor<ViewpointPtr>();
  types.add_type<SimStateJulia>("SimStateJulia")
    .constructor<SimStatePtr,bool>();
}

[barche]

Can you also show the content of the header? I think here the problem is that ViewpointPtr and SimStatePtr are not wrapped.

[ekinakyurek]

Yes, there is a problem about that ptrs too. I am including related section from MatterSim.hpp. However, I am getting errors from ArrayRef in compile time. I did some tricks with using pointers, then I got error from ViewpointPtr in runtime. I need a little bit guide to complete this :(

Also you may see original pybind11 code that I am trying to convert: https://github.com/ronghanghu/speaker_follower/blob/master/src/lib_python/MatterSimPython.cpp

namespace mattersim {
    struct Viewpoint {
        //! Viewpoint identifier
        std::string viewpointId;
        //! Viewpoint index into connectivity graph
	unsigned int ix;
    	//! 3D position in world coordinates
        cv::Point3f point;
        //! Heading relative to the camera
        double rel_heading;
        //! Elevation relative to the camera
	double rel_elevation;
        //! Distance from the agent
	double rel_distance;
    };

    typedef std::shared_ptr<Viewpoint> ViewpointPtr;

 /**
     * Simulator state class.
     */
    struct SimState {
        //! Building / scan environment identifier
        std::string scanId;
        //! Number of frames since the last newEpisode() call
        unsigned int step = 0;
        //! RGB image taken from the agent's current viewpoint
        cv::Mat rgb;
        //! Depth image taken from the agent's current viewpoint (not implemented)
        cv::Mat depth;
        //! Agent's current 3D location
        ViewpointPtr location;
        //! Agent's current camera heading in radians
        double heading = 0;
        //! Agent's current camera elevation in radians
        double elevation = 0;
        //! Agent's current view [0-35] (set only when viewing angles are discretized)
        //! [0-11] looking down, [12-23] looking at horizon, [24-35] looking up
        unsigned int viewIndex = 0;
        //! Vector of nearby navigable locations representing state-dependent action candidates, i.e.
        //! viewpoints you can move to. Index 0 is always to remain at the current viewpoint.
        //! The remaining viewpoints are sorted by their angular distance from the centre of the image.

    std::vector<ViewpointPtr> navigableLocations;
    };

    typedef std::shared_ptr<SimState> SimStatePtr;
}

[barche]

OK, I see now, you need to always supply a Julia array to construct an ArrayRef, so SimStateJulia would look like this:

  SimStateJulia(SimStatePtr state, bool renderingEnabled, jlcxx::ArrayRef<ViewPointJulia> julia_array)
      : step{state->step},
        viewIndex{state->viewIndex},
        location{state->location},
        heading{state->heading},
        elevation{state->elevation},
        navigableLocations(julia_array)
  {
    scanId = state->scanId;
    for (auto viewpoint : state->navigableLocations)
    {
      navigableLocations.push_back(ViewPointJulia{viewpoint});
    };
  };

And then the constructor wrapper takes the extra argument too:

types.add_type<SimStateJulia>("SimStateJulia")
      .constructor<SimStatePtr, bool, jlcxx::ArrayRef<ViewPointJulia>>();

On the Julia side, you need to pass a 1D-array of ViewPointJulia.

All this seems rather elaborate to be honest, why not just add SimState and ViewPoint directly using add_type and wrap the relevant methods to expose their data?