Issue with template member whose parameters depend on the class template parameters

[kkrissian-gpfw]

Hello,

I am using Python 3.8.10 with cppyy 2.2.0, and I have an issue about the size of the std::array member, declared as std::array<T, (1<<X)+1> from a Lut<T,X,Y> class: when instantiating a Lut with lut = cppyy.gbl.Lut [ int, 14, 14 ](), the type of the data member is said to be cppyy.gbl.std.array<int,1> instead of cppyy.gbl.std.array<int,16385>. I could make it work in Lut2 class by adding a template parameter with default value.

import cppyy

cppyy.cppdef("""
	#include <array>
	#include <cstdint>

	template<class T, uint8_t X, uint8_t Y> struct Lut {
		// Constructors
		Lut() { }
		// Capacity
		constexpr size_t size() const noexcept { return (1<<X)+1; }
	public:
		std::array<T, 3>          data1 ; // OK
		std::array<T, X>          data2 ; // OK
		std::array<T, 2*X>        data3 ; // OK
		std::array<T, 16385>      data4 ; // OK
		std::array<T, (1UL<<(std::size_t)3)+1UL> data5 ; // not OK
		std::array<T, ((1<<3)+1)> data6 ; // not OK
		std::array<T, ((1<<X)+1)> data7 ; // not OK
		static int constexpr array_size =  X<<2;
		std::array<T, array_size> data8 ; // not OK
	};

	template<class T, uint8_t X, uint8_t Y, uint32_t asize=((1<<X)+1)> struct Lut2 {
		// Constructors
		Lut2() { }
		// Capacity
		constexpr size_t size() const noexcept { return (1<<X)+1; }
	public:
		std::array<T, asize>        data;
	};

 """)

lut = cppyy.gbl.Lut[int, 14, 14]()

print(lut)
# <cppyy.gbl.Lut<int,'\x0E','\x0E'> object at 0x3e47eb0>
print(lut.size())
# 16385
print(lut.data1)
# <cppyy.gbl.std.array<int,3> object at 0x3e47eb0>
print(lut.data2)
# <cppyy.gbl.std.array<int,14> object at 0x3e47ebc>
print(lut.data3)
# <cppyy.gbl.std.array<int,28> object at 0x3e47ef4>
print(lut.data4)
# <cppyy.gbl.std.array<int,16385> object at 0x3e47f64>
print(lut.data5)
# <cppyy.gbl.std.array<int,1> object at 0x3e57f68>
print(lut.data6)
# TypeError: C++ type cannot be converted from memory
print(lut.data7)
# TypeError: C++ type cannot be converted from memory
print(lut.data8)
# TypeError: C++ type cannot be converted from memory

lut2 = cppyy.gbl.Lut2[int, 14, 14]()
print(lut2)
# <cppyy.gbl.Lut2<int,'\x0E','\x0E',16385> object at 0x418e520>
print(lut2.size())
# 16385
print(lut2.data)
# <cppyy.gbl.std.array<int,16385> object at 0x418e520>

[wlav]

Thanks for the nice reproducer!

Two things are going wrong here: uint8_t resolves as an unsigned char (hence the appearance of \x0E) even as C++ intends it as a numeric type; and the presence of << in the template names breaks down the lookup of templated subtypes, so e.g. data6 resolves as std::array<int,((1<3)> > (note missing < in (1<3); the other failures, incl. data5 are for the same reason).

[wlav]

The name printing appears to be deep in Clang; I'm going to leave that as-is for now. Repos have otherwise been updated with a fix for all the lookup problems. (It's pretty ugly; but the bulk of that processing code is to go away when the currently on-going cleanup of clingwrapper is completed.)

[kkrissian-gpfw]

Thank you, I will give it a try.

Do you think it could be possible to resolve uint8_t as numbers?

[wlav]

I tried, but atm, I think not, as it appears to be a bit worse than simple Clang type printing.

Yes, the declared type can show the integer numbers, but implicit declarations (such as the copy constructor) use the canonical type that has the typedefs resolved and I'm unable to work back from there to uint8_t. As long as there are still string comparisons in the code (upstream is working on getting those removed, although to be sure, using the canonical decl as identifier doesn't work across the board either b/c of other corner cases), that could lead to spurious failures in use as the types are then considered different on the Python side.

Indeed, if fixing the declared type so that the class is named with integer constants and the above prints as desired, the (implicitly generated) copy constructor still looks like: Lut<int,14,14>::Lut<int,14,15>(const Lut<int,'\x0E','\x0E'>&) and Lut<int,14,14> instances will be rejected as argument.

I expect that even with use of decls everywhere and no more string comparisons, doc strings will still show the chr values (for the same reason as above). This goes for Cling proper too (hitting <tab> on line 2 to print suggested completions):

root [1] Lut<int, 14, 14> lut;
root [2] lut.
Lut<int,'\x0E','\x0E'>
array_size
data1
data2
data3
data4
data5
data6
data7
data8
operator=
size
~Lut<int,'\x0E','\x0E'>
root [3] lut.

The other problem is that although it seems obvious that here the declared type of Lut<int, 14, 14> is the preferred human-readable printing, the same is not true for the other types: I presume you'd want the expressions resolved to numeric constants? Cling doesn't (for the same reason why it does print 14 above):

root [4] Lut<int, 14, ((1<<3)+1)> lut2
(Lut<int, 14, ((1 << 3) + 1)> &) @0x7f0d2cd1a000
root [5]

For that matter:

root [5] Lut<int, 14, 7*2> lut3
(Lut<int, 14, 7 * 2> &) @0x7f0d2545c000
root [6] auto lut4 = Lut<int, 14, 14>(lut3)
(Lut<int, '\x0E', '\x0E'> &) @0x7f0d25410000
root [7]

for the same reasons as described. At least, as is now, this is consistent across the board in Python. :)

I'll give upstream a ping, see whether they have different ideas, but don't hold your breath.

[kkrissian-gpfw]

Thank you again for your help and let me know if you have updates,

Regards

[wlav]

Upstream pointed to this recent possible GSoC proposal, which would be a step in the right direction: https://discourse.llvm.org/t/clang-extend-clang-ast-to-provide-information-for-the-type-as-written-in-template-instantiations/60323

[wlav]

The original problem is now fixed with release 2.3.0 and its dependencies.

[kkrissian-gpfw]

That's great, thank you