Bug: Passing `nullptr` around crashes clad, even if they are not dereferenced

[guitargeek]

It would be nice if passing around null pointers would nor crash the gradient generation, because then one could apply AD to function with optional pointer arguments.

My setup: Arch Linux with Clad master and clang 16.06.

I hope you can reproduce this problem, and a fix will be very appreciated. If this would have worked, it would have saved me lots of time debugging :slightly_smiling_face:

Reproducer:

// Compile with clang++ -std=c++11 -fplugin=/usr/lib/clad.so cladIssue.cxx -o cladIssue

#include "clad/Differentiator/Differentiator.h"

#include <iostream>
#include <vector>

double func(double *params, double *)
{
   return params[0];
}

double wrapper(double *params)
{
   return func(params, 0);
   // also nullptr will not work (would be nice if it does)
   // At least with nullptr one gets this warning:
   // warning: attempted to differentiate unsupported statement, no changes applied
}

int main()
{
   auto grad = clad::gradient(wrapper, "params");

   double x = 2.0;
   double result = 0.0;
   grad.execute(&x, &result);

   std::cout << result << std::endl;
}

[guitargeek]

Here is the stack trace I get:

PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: /usr/bin/clang-16 -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name issue.cxx -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -fcoverage-compilation-dir=/home/rembserj/tmp/cladbug -resource-dir /usr/lib/clang/16 -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1 -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/x86_64-pc-linux-gnu -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/backward -internal-isystem /usr/lib/clang/16/include -internal-isystem /usr/local/include -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -std=c++11 -fdeprecated-macro -fdebug-compilation-dir=/home/rembserj/tmp/cladbug -ferror-limit 19 -stack-protector 2 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -load /usr/lib/clad.so -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/issue-f8497c.o -x c++ issue.cxx
1.	<eof> parser at end of file
 #0 0x00007fb287a1f503 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/lib/libLLVM-16.so+0xe1f503)
 #1 0x00007fb287a1c7bf llvm::sys::RunSignalHandlers() (/usr/lib/libLLVM-16.so+0xe1c7bf)
 #2 0x00007fb287a1c90d (/usr/lib/libLLVM-16.so+0xe1c90d)
 #3 0x00007fb28665c710 (/usr/lib/libc.so.6+0x3e710)
 #4 0x00007fb28414c498 clad::ReverseModeVisitor::VisitCallExpr(clang::CallExpr const*) (/usr/lib/clad.so+0xc0498)
 #5 0x00007fb28413aeb4 (/usr/lib/clad.so+0xaeeb4)
 #6 0x00007fb28413b0dd (/usr/lib/clad.so+0xaf0dd)
 #7 0x00007fb284151a26 clad::ReverseModeVisitor::DifferentiateSingleExpr(clang::Expr const*, clang::Expr*) (/usr/lib/clad.so+0xc5a26)
 #8 0x00007fb284151c8f clad::ReverseModeVisitor::VisitReturnStmt(clang::ReturnStmt const*) (/usr/lib/clad.so+0xc5c8f)
 #9 0x00007fb28413af1b (/usr/lib/clad.so+0xaef1b)
#10 0x00007fb28413b13e (/usr/lib/clad.so+0xaf13e)
#11 0x00007fb28414fe3f clad::ReverseModeVisitor::DifferentiateSingleStmt(clang::Stmt const*, clang::Expr*) (/usr/lib/clad.so+0xc3e3f)
#12 0x00007fb28415176f clad::ReverseModeVisitor::VisitCompoundStmt(clang::CompoundStmt const*) (/usr/lib/clad.so+0xc576f)
#13 0x00007fb28413af6f (/usr/lib/clad.so+0xaef6f)
#14 0x00007fb28413b0dd (/usr/lib/clad.so+0xaf0dd)
#15 0x00007fb284148a3c clad::ReverseModeVisitor::DifferentiateWithClad() (/usr/lib/clad.so+0xbca3c)
#16 0x00007fb284149c5c clad::ReverseModeVisitor::Derive(clang::FunctionDecl const*, clad::DiffRequest const&) (/usr/lib/clad.so+0xbdc5c)
#17 0x00007fb2840e4053 clad::DerivativeBuilder::Derive(clad::DiffRequest const&) (/usr/lib/clad.so+0x58053)
#18 0x00007fb2840dfc9f clad::plugin::CladPlugin::ProcessDiffRequest(clad::DiffRequest&) (/usr/lib/clad.so+0x53c9f)
#19 0x00007fb2840e02ee clad::plugin::CladPlugin::HandleTopLevelDecl(clang::DeclGroupRef) (/usr/lib/clad.so+0x542ee)
#20 0x00007fb29110d6b0 clang::MultiplexConsumer::HandleTopLevelDecl(clang::DeclGroupRef) (/usr/lib/libclang-cpp.so.16+0x210d6b0)
#21 0x00007fb28f8ecf33 clang::ParseAST(clang::Sema&, bool, bool) (/usr/lib/libclang-cpp.so.16+0x8ecf33)
#22 0x00007fb29111e7c9 clang::FrontendAction::Execute() (/usr/lib/libclang-cpp.so.16+0x211e7c9)
#23 0x00007fb2910c0d2f clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/usr/lib/libclang-cpp.so.16+0x20c0d2f)
#24 0x00007fb29117bf02 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/usr/lib/libclang-cpp.so.16+0x217bf02)
#25 0x0000564cb6c4fa6f cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/usr/bin/clang-16+0x14a6f)
#26 0x0000564cb6c53b10 (/usr/bin/clang-16+0x18b10)
#27 0x0000564cb6c54a88 clang_main(int, char**) (/usr/bin/clang-16+0x19a88)
#28 0x00007fb286645cd0 (/usr/lib/libc.so.6+0x27cd0)
#29 0x00007fb286645d8a __libc_start_main (/usr/lib/libc.so.6+0x27d8a)
#30 0x0000564cb6c47ed5 _start (/usr/bin/clang-16+0xced5)
clang-16: error: unable to execute command: Segmentation fault (core dumped)
clang-16: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 16.0.6
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /bin
clang-16: note: diagnostic msg:
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang-16: note: diagnostic msg: /tmp/issue-f79e0b.cpp
clang-16: note: diagnostic msg: /tmp/issue-f79e0b.sh
clang-16: note: diagnostic msg:

********************

[parth-07]

Hi @guitargeek

Thank you for reporting this issue. The issue is not due to passing nullptr. The root cause of the issue is the (very) limited support of pointers in the reverse mode. C arrays are supported but pointers are not. I know it's slightly confusing and surprising.

The below code which does not use any nullptr has the same issue which you have described:

// Compile with clang++ -std=c++11 -fplugin=/usr/lib/clad.so cladIssue.cxx -o cladIssue

#include "clad/Differentiator/Differentiator.h"

#include <iostream>
#include <vector>

double func(double *params, double *dummy)
{
   return params[0];
}

double wrapper(double *params)
{
    double t;
   return func(params, &t);
   // also nullptr will not work (would be nice if it does)
   // At least with nullptr one gets this warning:
   // warning: attempted to differentiate unsupported statement, no changes applied
}

int main()
{
   auto grad = clad::gradient(wrapper, "params");

   double x = 2.0;
   double result = 0.0;
   grad.execute(&x, &result);

   std::cout << result << std::endl;
}

The below code which instead use a dummy array works correctly:

// Compile with clang++ -std=c++11 -fplugin=/usr/lib/clad.so cladIssue.cxx -o cladIssue

#include "clad/Differentiator/Differentiator.h"

#include <iostream>
#include <vector>

double func(double *params, double *dummy)
{
   return params[0];
}

double wrapper(double *params)
{
    double t[1];
   return func(params, t);
   // also nullptr will not work (would be nice if it does)
   // At least with nullptr one gets this warning:
   // warning: attempted to differentiate unsupported statement, no changes applied
}

int main()
{
   auto grad = clad::gradient(wrapper, "params");

   double x = 2.0;
   double result = 0.0;
   grad.execute(&x, &result);

   std::cout << result << std::endl;
}

Clad in the near future will support pointers as well. Then this issue will implicitly get resolved.

One another concerning thing is that Clad does not report any meaningful error for features that aren't supported. If Clad did that, then it would have significantly saved your debugging time. @vgvassilev What do you think about adding extensive support for reporting errors for unsupported features? I think catching unsupported features shouldn't be difficult considering the Clang AST traversal.

[guitargeek]

Thank you so much for your reply! In fact, you already get a nice warning when using nullptr, but not when using 0.

Both adding pointer support or also adding a warning when passing 0 (or NULL I guess) are good solutions to this issue :+1: