OpenShadingLanguage icon indicating copy to clipboard operation
OpenShadingLanguage copied to clipboard

Null Pointer Exception possibly due to Invalid Cast from `llvm::Value*` to `llvm::Function*`

Open 4134N4 opened this issue 7 months ago • 0 comments

Problem

I have the suspicion that the following cast in llvm_util.cpp line 4353 ist not correct, which results in a null value, which in turn results in a null pointer exception occurring.

static_cast<llvm::Function*>(func)->getFunctionType()

The cast is done in the following function, where a null pointer exception occurs when calling builder().CreateCall(...) (here in line 10):

llvm::Value*
LLVM_Util::call_function(llvm::Value* func, cspan<llvm::Value*> args)
{
    OSL_DASSERT(func);
#if 0
    ...
#endif
    //llvm_gen_debug_printf (std::string("start ") + std::string(name));
    llvm::Value* r = builder().CreateCall(
        static_cast<llvm::Function*>(func)->getFunctionType(), func,
        llvm::ArrayRef<llvm::Value*>(args.data(), args.size()));
    //llvm_gen_debug_printf (std::string(" end  ") + std::string(name));
    return r;
}

to get llvm::Value* func's FunctionType for CreateCall's FunctionType *FTy argument:

CallInst *CreateCall(FunctionType *FTy, Value *Callee, ArrayRef<Value *> Args,
                       ArrayRef<OperandBundleDef> OpBundles,
                       const Twine &Name = "", MDNode *FPMathTag = nullptr) {...}

(From IRBuilder.h in LLVM's llvm/IR.)


llvm::Value* func_ptr -- which is passed to LLVM_Util::call_function's llvm::Value* func argument -- is created in llvm_gen.cpp line 3879 ff. if the closure entry (clentry) has a (non-nullptr) ''prepare'' method (here in line 6 ff.):

// If the closure has a "prepare" method, call
// prepare(renderer, id, memptr).  If there is no prepare method, just
// zero out the closure parameter memory.
if (clentry->prepare) {
    // Call clentry->prepare(renderservices *, int id, void *mem)
    llvm::Value* funct_ptr
        = rop.ll.constant_ptr((void*)clentry->prepare,
                                rop.llvm_type_prepare_closure_func());
    llvm::Value* args[] = { render_ptr, id_int, mem_void_ptr };
    rop.ll.call_function(funct_ptr, args);
} else {
    rop.ll.op_memset(mem_void_ptr, 0, clentry->struct_size, 4 /*align*/);
}

Now from my testing static_cast<llvm::Function*>(func) does not cast llvm::Value* func to a correct llvm::Function*, hence calling members of llvm::Function (such as getFunctionType()) result in invalid results.

What Works

An older, deprecated (getPointerElementType() is deprectaed) way works to get llvm::FunctionType* from llvm::Value* func:

llvm::cast<llvm::FunctionType>(func->getType()->getPointerElementType())

This way llvm::Value* func's type is gotten and then cast to llvm::FunctionType.

As opposed to first casting llvm::Value to llvm::Function and then getting its (function) type.

Hence, from what I can tell, I think llvm::Value* func's type should be gotten (like in the deprecated way) and then cast to llvm::FunctionType. I could however not figure out how to do this in a way that aligns with the new opaque pointer dogma of LLVM.

and What also Doesn't Work

Using llvm::cast instead of static_cast resulted in the same error for me.

Using only llvm::cast<llvm::FunctionType>(func->getType()) also didn't work, resulting at a type mismatch at compile time.

Reproduction

Registering a closure using

void OSL::ShadingSystem::register_closure(string_view name, int id, const ClosureParam* params,
                          PrepareClosureFunc prepare, SetupClosureFunc setup);

with PrepareClosureFunc prepare being non-nullptr should cause this issue, as it will then fullfil the ''if the closure has a "prepare" method'' condition from above.


Unfortunately I'm not sure how I would go about creating a minimal reproducible example for this.

I've run into this problem working on Applessed; its closures can be seen in this file in the Appleseed repository.

Conclusion

Is my suspicion correct that the cast here is invalid? If so, how would be the correct (non-deprecated) way to get the llvm::FunctionType?

Or am I perhaps missing something and this should work and not result in a null pointer exception? If so, where might my mistake lie?

I'm thankful for any and all support.

Kind regards,

Alexander

My Versions

OSL: Release 1.13.11.0 OS: Ubuntu 22.04.5 LTS C++: Compiler: GNU 11.4.0 / Clang 18.1.8 (issue is the same between both) LLVM: 14.0.0 (via build_llvm.bash build script) OIIO: Release 2.5.16.0

4134N4 avatar Mar 21 '25 14:03 4134N4