Error: Don't know how to compare array and array (hlc) when compiling Heaps.io for HL/C with DX12

[dipyalov]

Haxe: latest git Hashlink: latest git Heaps.io: latest git

I'm compiling a sample Heaps.io project with DX12 backend to HL/C target. Compiler throws "Error: Don't know how to compare array and array (hlc)" message.

My investigation has shown the following:

The "flushPipeline" method of heaps/h3d/DX12Driver.hx has the following code:

	function flushPipeline() {
		if( !needPipelineFlush ) return;
		needPipelineFlush = false;
		var signature = pipelineSignature;
		var signatureSize = PSIGN_LAYOUT + currentShader.inputCount;
		adlerOut.setI32(0, 0);
		hl.Format.digest(adlerOut, signature, signatureSize, 3);
		var hash = adlerOut.getI32(0);
		var pipes = currentShader.pipelines.get(hash);
		if( pipes == null ) {
			pipes = new hl.NativeArray(1);
			currentShader.pipelines.set(hash, pipes);
		}

And something between:

var pipes = currentShader.pipelines.get(hash);
if( pipes == null ) {

makes HL/C generate the following C code:

	r36 = r0->currentShader;
	if( r36 == NULL ) hl_null_access();
	r35 = r36->pipelines;
	if( r35 == NULL ) hl_null_access();
	r37 = r32;
	r34 = haxe_ds_IntMap_get(r35,r37);
	r38 = (varray*)r34;
	r39 = (varray*)hl_dyn_castp(&r34,&t$_dyn,&t$_array);
	### WRONG 0 ### r38 == r39 -> label$6a3cd68_69_40 ###;
	hl_assert();
	label$6a3cd68_69_40:
	r40 = r38;
	r43 = r40;
	if( r43 ) goto label$6a3cd68_69_54;

The ### WRONG ### part is my modification to debug this as the haxe compiler actually fails at this point because it can't generate comparison statement for 2 arrays.

[dipyalov]

Forgot to mention - I have --debug option set. And my investigation has shown that this code is generated for debugging purposes in unsafe_cast HL op produced here:

https://github.com/HaxeFoundation/haxe/blob/584a42cb60370b6bf652061e7950603a7500e97e/src/generators/genhl.ml#L2208C4-L2208C4

Still don't understand how it should be fixed yet.

[dipyalov]

Should replacing unsafe_cast_to ctx ret rt e.epos with unsafe_cast_to ~debugchk:false ctx ret rt e.epos be OK? Or is there a special idea to not set debugchk to false?

[onehundredfeet]

I'm getting a similar error, but I have no idea where in my codebase it's coming from. It doesn't show up when compiling for hashlink HL

Error: Don't know how to compare bytes and bytes (hlc)

[onehundredfeet]

I was able to narrow it down to hl.Bytes. But that makes little sense.

class Junk{
	public static function exists2(a : hl.Bytes) {
		return a != null;
	}	
}

class HelloWorld {
    public static function main() {

        trace('Hello, World! ${Junk.exists2(null)}'); 
    }
}

Error: Don't know how to compare bytes and bytes (hlc)

[onehundredfeet]

I don't see HBytes anywhere here, but my ocaml is not great.

let common_type ctx e1 e2 for_eq p =
	let t1 = to_type ctx e1.etype in
	let t2 = to_type ctx e2.etype in
	let rec loop t1 t2 =
		if t1 == t2 then t1 else
		match t1, t2 with
		| HUI8, (HUI16 | HI32 | HI64 | HF32 | HF64) -> t2
		| HUI16, (HI32 | HI64 | HF32 | HF64) -> t2
		| (HI32 | HI64), HF32 -> t2 (* possible loss of precision *)
		| (HI32 | HI64 | HF32), HF64 -> t2
		| (HUI8|HUI16|HI32|HI64|HF32|HF64), (HUI8|HUI16|HI32|HI64|HF32|HF64) -> t1
		| (HUI8|HUI16|HI32|HI64|HF32|HF64), (HNull t2) -> if for_eq then HNull (loop t1 t2) else loop t1 t2
		| (HNull t1), (HUI8|HUI16|HI32|HI64|HF32|HF64) -> if for_eq then HNull (loop t1 t2) else loop t1 t2
		| (HNull t1), (HNull t2) -> if for_eq then HNull (loop t1 t2) else loop t1 t2
		| HDyn, (HUI8|HUI16|HI32|HI64|HF32|HF64) -> HF64
		| (HUI8|HUI16|HI32|HI64|HF32|HF64), HDyn -> HF64
		| HDyn, _ -> HDyn
		| _, HDyn -> HDyn
		| _ when for_eq && safe_cast t1 t2 -> t2
		| _ when for_eq && safe_cast t2 t1 -> t1
		| HBool, HNull HBool when for_eq -> t2
		| HNull HBool, HBool when for_eq -> t1
		| HObj _, HVirtual _ | HVirtual _, HObj _ | HVirtual _ , HVirtual _ -> HDyn
		| HFun _, HFun _ -> HDyn
		| _ ->
			abort ("Don't know how to compare " ^ tstr t1 ^ " and " ^ tstr t2) p
	in
	loop t1 t2
	```

But this is the regular HL code emitter, which works.  It's just the HL/C equivalent that doesn't work.

[onehundredfeet]

in hl2c.ml I wonder if adding HByte comparison case at line 747 should be:

match rtype a, rtype b with
			| (HUI8 | HUI16 | HI32 | HF32 | HF64 | HBool | HI64), (HUI8 | HUI16 | HI32 | HF32 | HF64 | HBool | HI64) ->
				phys_compare()
			| HBytes, HBytes ->
				phys_compare()
				

[onehundredfeet]

@Simn I saw your HArry / HArry compare case change and thought this was something similar.

[onehundredfeet]

Add PR.

https://github.com/HaxeFoundation/haxe/pull/11610