pasta icon indicating copy to clipboard operation
pasta copied to clipboard

Published 20 hours ago • verified publisher icon trailofbits

dump-ast example program hits assertion: `!PA.isArgIdent(Idx)' failed.

Open seelabs opened this issue 1 year ago • 41 comments

I read about pasta on your blog post and decide to play with it. I am not familiar with the internals of libclang or llvm, so this issue may be user error.

I successfully built pasta and was able to run the demo programs. When I tried the dump-ast example on a toy example it worked as expected. However, when I tried it against my real project, it hits an assert. In particular, when I run it against this project: https://github.com/XRPLF/rippled using the following command line:


./dump-ast /home/swd/apps/clang-latest/bin/clang++ -DBOOST_ASIO_DISABLE_HANDLER_TYPE_REQUIREMENTS -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DBOOST_BEAST_ALLOW_DEPRECATED -DBOOST_CONTAINER_FWD_BAD_DEQUE -DBOOST_COROUTINES_NO_DEPRECATION_WARNING -DBOOST_FILESYSTEM_DEPRECATED -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=\\\"/usr/bin/addr2line\\\" -DBOOST_STACKTRACE_USE_ADDR2LINE -DBOOST_STACKTRACE_USE_BACKTRACE -DBOOST_STACKTRACE_USE_NOOP -DCARES_STATICLIB -DDATE_HEADER_ONLY -DDEBUG -DENABLE_TESTS -DGIT_COMMIT_HASH=\\\"ed9064ce3f6de6da6b63417e59f3100c8e6c4c6d\\\" -DHAS_UNCAUGHT_EXCEPTIONS=1 -DOPENSSL_NO_SSL2 -D_DEBUG -I/home/swd/projs/ripple/mine/src/ed25519-donna -I/home/swd/projs/ripple/mine/src/secp256k1/include -I/home/swd/projs/ripple/mine/src -isystem /home/swd/.conan/data/boost/1.82.0/_/_/package/f81afbcfa21cf6994cf0e45c1b11229c01c767a7/include -isystem /home/swd/.conan/data/openssl/1.1.1u/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/.conan/data/libarchive/3.6.2/_/_/package/d9fea6983af2a77bcbe3f0b088cea5aef9308bd7/include -isystem /home/swd/.conan/data/lz4/1.9.3/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/projs/ripple/mine/cur_build/proto_gen_grpc -isystem /home/swd/.conan/data/grpc/1.50.1/_/_/package/d156c58fd512defbc1155e665684d7875e800ba9/include -isystem /home/swd/.conan/data/abseil/20220623.0/_/_/package/e84b6d643e7f97d33bf72700134618fbae8bbbf6/include -isystem /home/swd/.conan/data/c-ares/1.19.0/_/_/package/ed86cef4487c6b9ce327a03d24ced8142000e8b9/include -isystem /home/swd/.conan/data/re2/20230301/_/_/package/e84b6d643e7f97d33bf72700134618fbae8bbbf6/include -isystem /home/swd/.conan/data/zlib/1.2.13/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/.conan/data/protobuf/3.21.9/_/_/package/681b3b1dffc7e4dc6f5551cdb6e1fe56bdda3e85/include -isystem /home/swd/projs/ripple/mine/cur_build/proto_gen -isystem /home/swd/projs/ripple/mine/cur_build/proto_gen/src/ripple/proto -isystem /home/swd/.conan/data/soci/4.0.3/_/_/package/dc82e3681e64ed36c447531c9e4883a5c616a774/include -isystem /home/swd/.conan/data/bzip2/1.0.8/_/_/package/5e357164f23f7c4e38ebe87433827878dbe1a74c/include -isystem /home/swd/.conan/data/libbacktrace/cci.20210118/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/.conan/data/sqlite3/3.42.0/_/_/package/ae0e146c4c82ac1e46c279d15c964c80fedbdf5f/include -isystem /home/swd/.conan/data/nudb/2.0.8/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/include -isystem /home/swd/.conan/data/date/3.0.1/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/include -m64 -stdlib=libstdc++ -Wall -Wdeprecated -Wextra -Wno-unused-parameter -g -std=c++20 -fPIE -frtti -Wnon-virtual-dtor -Wno-sign-compare -Wno-char-subscripts -Wno-format -Wno-unused-local-typedefs -fstack-protector -o CMakeFiles/rippled.dir/src/ripple/app/ledger/impl/LedgerReplay.cpp.o -c /home/swd/projs/ripple/mine/src/ripple/app/ledger/impl/LedgerReplay.cpp

It hits the following assert:


dump-ast: /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaType.cpp:8299: void HandleUnkownTypeAttrAsAnnotateTypeAttr({anonymous}::TypeProcessingState&, clang::QualType&, const clang::ParsedAttr&): Assertion `!PA.isArgIdent(Idx)' failed.
[1]    313418 IOT instruction (core dumped)  ./dump-ast /home/swd/apps/clang-latest/bin/clang++           -DCARES_STATICLI

I attempted to debug this. I built the debug version of the libraries and went into gdb and attempted to try to find the offending code so I could create a minimal repro, but unfortunately my unfamiliarity with llvm is an obstacle. Two things we may be able to use as staring points.

In SemaType.cpp:8299

p Str
$8 = {
  Data = 0x5555a3a89598 "__access__",
  Length = 10
}

So the offending attribute is called "access".

In ParseDeclCXX.cpp:2107

p PP.getSourceManager().getPresumedLoc(StartLoc,true)
$6 = {
  Filename = 0x5555a18575d0 "<pasta-input>",
  ID = {
    ID = 851074
  },
  Line = 1716508,
  Col = 1,
  IncludeLoc = {
    ID = 0
  }
}

What I'm trying to do here is find the name of the file that contains the offending code. Some searching pointed me to getPresumedLoc, but clearly <pasta-input> isn't useful to find the offending code. Perhaps you have a suggestion to find the code?

Also, I should mention I was only playing with pasta. Pasta obviously works and it's very possible I'm "holding it wrong". I'm not expecting help here, but thought I'd report the issue in case it was a possible bug in pasta (and will help trying to debug it after regular work hours if it's helpful to you).

seelabs avatar Aug 07 '23 01:08 seelabs

<pasta-input> is a synthetic input file that clang's parser operates on. PASTA first runs an instrumented preprocessor on the input, and in the process of this, forms <pasta-input>, which is just a very large std::string in memory. Each unique token is rendered onto a different line of this input file, so that a source location's line number acts as a kind of side-channel for a token index. Previously, I had attempted to patch Clang to embed more information into SourceLocations, but this became problematic as the underlying integral representation of SourceLocations is often stored directly in AST nodes. In retrospect, I regret the choice to use line numbers as a side-channel, as SourceLocations are basically byte offsets, and so I could have just used the byte offset itself with a list of "end of token bytes offsets", and then some binary searching. Oh well.

Anyway, probably you aren't holding it wrong. The reality is just that PASTA does some sketchy things to achieve its goals. Here's how I would proceed:

Patch the following code into lib/Compile/Run.cpp, around line 353:

  auto fd = open("/tmp/source.cpp", O_TRUNC | O_CREAT | O_WRONLY, 0666);
  write(fd, ast->preprocessed_code.data(), ast->preprocessed_code.size());
  close(fd);

Then build and run, and attach /tmp/source.cpp here. Another thing that could help is seeing the backtrace.

pgoodman avatar Aug 07 '23 21:08 pgoodman

Thanks for the explanation and next steps. The "source.cpp" is too large to paste here, so I created a repo here: https://github.com/seelabs/pasta-debugging/tree/main. "source.cpp" is here: https://github.com/seelabs/pasta-debugging/blob/main/source.cpp and the gdb transcript is here: https://github.com/seelabs/pasta-debugging/blob/main/gdb.txt with both bt full and bt. I'll paste bt here since it's small:

#0  HandleUnkownTypeAttrAsAnnotateTypeAttr (State=..., CurType=..., PA=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaType.cpp:8299
#1  0x0000555559ed322d in processTypeAttrs (state=..., type=..., TAL=TAL_DeclName, attrs=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaType.cpp:8413
#2  0x0000555559ec608b in GetFullTypeForDeclarator (state=..., declSpecType=..., TInfo=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaType.cpp:5810
#3  0x0000555559ec6d41 in clang::Sema::GetTypeForDeclarator (this=0x5555a18ddbf0, D=..., S=0x55556d734890) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaType.cpp:5935
#4  0x000055555909d929 in clang::Sema::HandleDeclarator (this=0x5555a18ddbf0, S=0x55556d734890, D=..., TemplateParamLists=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaDecl.cpp:6304
#5  0x00005555591ed58d in clang::Sema::ActOnCXXMemberDeclarator (this=0x5555a18ddbf0, S=0x55556d734890, AS=clang::AS_public, D=..., TemplateParameterLists=..., BW=0x0, VS=..., InitStyle=clang::ICIS_NoInit) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaDeclCXX.cpp:3510
#6  0x00005555589d2f3c in clang::Parser::ParseCXXInlineMethodDef (this=0x55559f6c1e60, AS=clang::AS_public, AccessAttrs=..., D=..., TemplateInfo=..., VS=..., PureSpecLoc=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseCXXInlineMethods.cpp:42
#7  0x0000555558a18a29 in clang::Parser::ParseCXXClassMemberDeclaration (this=0x55559f6c1e60, AS=clang::AS_public, AccessAttrs=..., TemplateInfo=..., TemplateDiags=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:2956
#8  0x0000555558a1ac52 in clang::Parser::ParseCXXClassMemberDeclarationWithPragmas (this=0x55559f6c1e60, AS=@0x7fffffff8598: clang::AS_public, AccessAttrs=..., TagType=clang::TST_struct, TagDecl=0x5555a64f2098) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:3391
#9  0x0000555558a1baee in clang::Parser::ParseCXXMemberSpecification (this=0x55559f6c1e60, RecordLoc=..., AttrFixitLoc=..., Attrs=..., TagType=25, TagDecl=0x5555a64f2098) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:3595
#10 0x0000555558a1506a in clang::Parser::ParseClassSpecifier (this=0x55559f6c1e60, TagTokKind=clang::tok::kw_struct, StartLoc=..., DS=..., TemplateInfo=..., AS=clang::AS_private, EnteringContext=true, DSC=clang::Parser::DeclSpecContext::DSC_class, Attributes=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:2107
#11 0x00005555589ee713 in clang::Parser::ParseDeclarationSpecifiers (this=0x55559f6c1e60, DS=..., TemplateInfo=..., AS=clang::AS_private, DSContext=clang::Parser::DeclSpecContext::DSC_class, LateAttrs=0x7fffffff9540, AllowImplicitTypename=clang::ImplicitTypenameContext::Yes) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDecl.cpp:4220
#12 0x00005555589cc6cb in clang::Parser::ParseDeclarationSpecifiers (this=0x55559f6c1e60, DS=..., TemplateInfo=..., AS=clang::AS_private, DSC=clang::Parser::DeclSpecContext::DSC_class, LateAttrs=0x7fffffff9540) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/include/clang/Parse/Parser.h:2421
#13 0x0000555558a18209 in clang::Parser::ParseCXXClassMemberDeclaration (this=0x55559f6c1e60, AS=clang::AS_private, AccessAttrs=..., TemplateInfo=..., TemplateDiags=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:2803
#14 0x0000555558a1ac52 in clang::Parser::ParseCXXClassMemberDeclarationWithPragmas (this=0x55559f6c1e60, AS=@0x7fffffffa8c8: clang::AS_private, AccessAttrs=..., TagType=clang::TST_class, TagDecl=0x5555a64eae48) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:3391
#15 0x0000555558a1baee in clang::Parser::ParseCXXMemberSpecification (this=0x55559f6c1e60, RecordLoc=..., AttrFixitLoc=..., Attrs=..., TagType=26, TagDecl=0x5555a64eae48) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:3595
#16 0x0000555558a1506a in clang::Parser::ParseClassSpecifier (this=0x55559f6c1e60, TagTokKind=clang::tok::kw_class, StartLoc=..., DS=..., TemplateInfo=..., AS=clang::AS_none, EnteringContext=true, DSC=clang::Parser::DeclSpecContext::DSC_top_level, Attributes=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:2107
#17 0x00005555589ee713 in clang::Parser::ParseDeclarationSpecifiers (this=0x55559f6c1e60, DS=..., TemplateInfo=..., AS=clang::AS_none, DSContext=clang::Parser::DeclSpecContext::DSC_top_level, LateAttrs=0x0, AllowImplicitTypename=clang::ImplicitTypenameContext::Yes) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDecl.cpp:4220
#18 0x00005555589cc6cb in clang::Parser::ParseDeclarationSpecifiers (this=0x55559f6c1e60, DS=..., TemplateInfo=..., AS=clang::AS_none, DSC=clang::Parser::DeclSpecContext::DSC_top_level, LateAttrs=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/include/clang/Parse/Parser.h:2421
#19 0x00005555589c11f8 in clang::Parser::ParseDeclOrFunctionDefInternal (this=0x55559f6c1e60, Attrs=..., DeclSpecAttrs=..., DS=..., AS=clang::AS_none) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/Parser.cpp:1119
#20 0x00005555589c18e6 in clang::Parser::ParseDeclarationOrFunctionDefinition (this=0x55559f6c1e60, Attrs=..., DeclSpecAttrs=..., DS=0x0, AS=clang::AS_none) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/Parser.cpp:1226
#21 0x00005555589c0c1f in clang::Parser::ParseExternalDeclaration (this=0x55559f6c1e60, Attrs=..., DeclSpecAttrs=..., DS=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/Parser.cpp:1041
#22 0x0000555558a0cfe8 in clang::Parser::ParseInnerNamespace (this=0x55559f6c1e60, InnerNSs=..., index=0, InlineLoc=..., attrs=..., Tracker=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:261
#23 0x0000555558a0cc6e in clang::Parser::ParseNamespace (this=0x55559f6c1e60, Context=clang::DeclaratorContext::File, DeclEnd=..., InlineLoc=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:238
#24 0x00005555589e4dad in clang::Parser::ParseDeclaration (this=0x55559f6c1e60, Context=clang::DeclaratorContext::File, DeclEnd=..., DeclAttrs=..., DeclSpecAttrs=..., DeclSpecStart=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDecl.cpp:1807
#25 0x00005555589c070a in clang::Parser::ParseExternalDeclaration (this=0x55559f6c1e60, Attrs=..., DeclSpecAttrs=..., DS=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/Parser.cpp:957
#26 0x00005555589bfa16 in clang::Parser::ParseTopLevelDecl (this=0x55559f6c1e60, Result=..., ImportState=@0x7fffffffc6c8: clang::Sema::ModuleImportState::NotACXX20Module) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/Parser.cpp:743
#27 0x00005555562df830 in pasta::CompileJob::Run[abi:cxx11]() const (this=0x555561541be0) at /home/swd/projs/common/pasta/lib/Compile/Run.cpp:425
#28 0x0000555555ef7b59 in main (argc=81, argv=0x7fffffffcd38) at /home/swd/projs/common/pasta/bin/DumpAST/Main.cpp:135

I also looked at source.cpp around line 1716508. Here's what appears there:

struct
_Stop_state_ref
{
_Stop_state_ref
(
)
=
default
;
[
[
__gnu__
::
__access__
(
__none__
,
2
)
]
]

seelabs avatar Aug 08 '23 14:08 seelabs

@pgoodman I found a simple way to reproduce the error.

  1. Create a file called "repo.cpp" with one line #include <stop_token>
  2. run ./dump-ast g++ -std=c++20 repo.cpp This will hit the assert above
  3. run ./dump-ast g++ repo.cpp This will work as expected (note: no `-std=c++20)

So the culprit is -std=c++20.

I'll note that the problem is not specific to stop_token. Including just optional will also exhibit this behavior, but a simple include like cstdint will run as expected. (I had tested using stop_token because it was the header implicated in the above source.cpp)

seelabs avatar Aug 08 '23 15:08 seelabs

Okay this is a bug in one of our patches: HandleUnkownTypeAttrAsAnnotateTypeAttr.

We've made this patch with the intent of implicitly converting things like __attribute__((hello_world("a"))) into __attribute__((annotate("hello_world", "a"))), though there are clearly some corner cases that are triggering assertions. I don't have the spare hours right now to fix this bug in our patches. Can you try mitigating it with the following change: Change this true to false: https://github.com/trailofbits/pasta/blob/master/lib/Compile/Run.cpp#L242-L244

pgoodman avatar Aug 09 '23 17:08 pgoodman

Thanks! I made the suggested change and that successfully ran against the minimal repro. I then tried against my real codebase and got:

A clang diagnostic or uncompilable error was produced when trying to get an AST due to error: /usr/lib/gcc/x86_64-pc-linux-gnu/13.1.1/include/xmmintrin.h:136:19 use of undeclared identifier '__builtin_ia32_addss'

I don't know if that's related to disabling the "unknown attribute" patch or not. (I know you don't have time to dig into this now, I just thought I'd report it anyway).

seelabs avatar Aug 10 '23 00:08 seelabs

The missing builtin looks to be a GCC-specific intrinsic that Clang doesn't seem to have. There are a number of others that are related, and could be harvested from this list. Fixing this involves adding entries into this file. This mirrors how Clang adds them, but we do it retroactively and in a slightly hacky way that generally gets the job done without having to patch Clang.

Could you try making the following additions to our BuiltinsX86.h file, and testing them out? If they work, please submit a PR :-)

TARGET_BUILTIN("__builtin_ia32_comieq", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_comineq", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_comilt", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_comile", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_comigt", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_comige", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomieq", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomineq", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomilt", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomile", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomigt", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomige", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_addps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_subps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_mulps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_divps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_addss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_subss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_mulss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_divss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpeqps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpltps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpleps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpgtps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpgeps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpunordps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpneqps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpnltps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpnleps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpngtps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpngeps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpordps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpeqss", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpltss", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpless", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpunordss", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpneqss", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpnlts", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpnless", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpordss", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_maxps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_maxss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_minps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_minss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_andps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_andnps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_orps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_xorps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_movss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_movhlps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_movlhps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_unpckhps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_unpcklps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvtpi2ps", "V4fV4fV2i", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvtsi2ss", "V4fV4f, i", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvtps2pi", "V2iV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvtss2si", "iV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvttps2pi", "V2iV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvttss2si", "iV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_rcpps", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_rsqrtps", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_sqrtps", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_rcpss", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_rsqrtss", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_sqrtss", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_shufps", "V4fV4fV4f, i", "", "sse")
TARGET_BUILTIN("__builtin_ia32_movntps", "vf*V4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_movmskps", "iV4f", "", "sse")

pgoodman avatar Aug 10 '23 04:08 pgoodman

I seem to be getting the same error as before. I modified the file pasta/lib/Compile/BuiltinsX86.h by adding the code above, recompiled, and re-ran. I also did a "make clean" and recompiled to double check. (as a dummy-check, here's my edited file as a gist: https://gist.github.com/seelabs/9a63d38260caf1ddce7bf7cdf043f8ae)

seelabs avatar Aug 10 '23 05:08 seelabs

Oh I'm a dummy. I shouldn't have wrapped all those builtin names with double quotes. Can you try with this?

TARGET_BUILTIN(__builtin_ia32_comieq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comineq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comilt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comile, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comigt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comige, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomieq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomineq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomilt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomile, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomigt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomige, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_addps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_subps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_mulps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_divps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_addss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_subss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_mulss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_divss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpeqps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpltps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpleps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpgtps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpgeps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpunordps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpneqps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnltps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnleps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpngtps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpngeps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpordps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpeqss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpltss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpless, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpunordss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpneqss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnlts, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnless, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpordss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_maxps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_maxss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_minps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_minss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_andps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_andnps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_orps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_xorps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movhlps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movlhps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_unpckhps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_unpcklps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtpi2ps, "V4fV4fV2i", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtsi2ss, "V4fV4f, i", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtps2pi, "V2iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtss2si, "iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvttps2pi, "V2iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvttss2si, "iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rcpps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rsqrtps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_sqrtps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rcpss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rsqrtss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_sqrtss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_shufps, "V4fV4fV4f, i", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movntps, "vf*V4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movmskps, "iV4f", "", "sse")

pgoodman avatar Aug 10 '23 16:08 pgoodman

Still no go, but a different error:

Unknown builtin type letter!
UNREACHABLE executed at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/AST/ASTContext.cpp:11250!

I went into gdb and looked at the offending Str parameter to DecodeTypeFromStr. It was: ", i". So it's choking on the ,. I don't have time to dig into this right now, but I'll try to look later and see if I can locate the offending code.

seelabs avatar Aug 10 '23 19:08 seelabs

From our dumped source.cpp, it's pointing to this as the culprit:

extern
__inline
__m128
__attribute__
(
(
__gnu_inline__
,
__always_inline__
,
__artificial__
)
)
_mm_cvtsi32_ss
(
__m128
__A
,
int
__B
)
{
return
(
__m128
)
__builtin_ia32_cvtsi2ss
(
(
__v4sf
)
__A
,
__B
)
;
}

The actual line it's pointing at is the one with __builtin_ia32_cvtsi2ss, but given the value of Str I'm guessing we should be looking at:

__m128
__A
,
int

seelabs avatar Aug 10 '23 21:08 seelabs

Try this?

TARGET_BUILTIN(__builtin_ia32_comieq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comineq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comilt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comile, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comigt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comige, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomieq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomineq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomilt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomile, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomigt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomige, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_addps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_subps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_mulps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_divps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_addss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_subss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_mulss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_divss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpeqps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpltps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpleps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpgtps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpgeps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpunordps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpneqps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnltps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnleps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpngtps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpngeps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpordps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpeqss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpltss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpless, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpunordss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpneqss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnlts, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnless, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpordss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_maxps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_maxss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_minps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_minss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_andps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_andnps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_orps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_xorps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movhlps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movlhps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_unpckhps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_unpcklps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtpi2ps, "V4fV4fV2i", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtsi2ss, "V4fV4fi", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtps2pi, "V2iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtss2si, "iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvttps2pi, "V2iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvttss2si, "iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rcpps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rsqrtps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_sqrtps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rcpss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rsqrtss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_sqrtss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_shufps, "V4fV4fV4fi", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movntps, "vf*V4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movmskps, "iV4f", "", "sse")

pgoodman avatar Aug 10 '23 22:08 pgoodman

No go, but a different error:

A clang diagnostic or uncompilable error was produced when trying to get an AST due to error: /usr/lib/gcc/x86_64-pc-linux-gnu/13.1.1/include/xmmintrin.h:624:19 use of undeclared identifier '__builtin_ia32_cvtsi642ss'

seelabs avatar Aug 11 '23 03:08 seelabs

Getting closer :-) Seems like we're missing that intrinsic too.

pgoodman avatar Aug 11 '23 16:08 pgoodman

Just a quick update: I'm attempting to add the needed intrinsics. I'm having trouble finding the documentation I need for these missing functions (the gcc manual doesn't seem to have them) and also the correct strings for the types. I'll spend time after work to poke at this, but it may be a while before I'm able to make a patch (but hopefully I'll learn something along the way).

seelabs avatar Aug 14 '23 23:08 seelabs

Oh I'm making a script for this! I'll be bulk grabbing them from CBMC, which has a lot of forward declarations. It turns out there is some other stuff in the Python API that has required fixing in order for me to make the script, so it's taking me a bit of time :-)

pgoodman avatar Aug 15 '23 00:08 pgoodman

My in-progress work is in this branch.

pgoodman avatar Aug 15 '23 00:08 pgoodman

Here's what I mean from taking the declarations from CBMC. The idea is to auto-generate the TARGET_BUILTIN macros from the declarations in a few of these header files.

pgoodman avatar Aug 15 '23 00:08 pgoodman

@seelabs Can you try with this version of BuiltinsX86.h, and let me know if it makes any more progress? Thanks!

pgoodman avatar Aug 15 '23 18:08 pgoodman

Yep, progress, but it looks like we're still missing some:

A clang diagnostic or uncompilable error was produced when trying to get an AST due to error: /usr/lib/gcc/x86_64-pc-linux-gnu/13.1.1/include/xmmintrin.h:827:1 definition of builtin function '_mm_getcsr'

seelabs avatar Aug 15 '23 20:08 seelabs

Hi @seelabs, I've updated the previously linked BuiltinsX86.h file, can you try this new one?

pgoodman avatar Aug 16 '23 18:08 pgoodman

I still get an error, but a different one. It's now complaining about _m_empty

A clang diagnostic or uncompilable error was produced when trying to get an AST due to error: /usr/lib/gcc/x86_64-pc-linux-gnu/13.1.1/include/mmintrin.h:70:1 definition of builtin function '_m_empty'

seelabs avatar Aug 16 '23 19:08 seelabs

I may need to patch clang if I want to generically handle (re)definitions of possible intrinsics. This will take deeper investigation.

pgoodman avatar Aug 17 '23 02:08 pgoodman

Note to self: Try adding a check for FD->getBuiltinID() in this function from SemaDecl.cpp:

/// canRedefineFunction - checks if a function can be redefined. Currently,
/// only extern inline functions can be redefined, and even then only in
/// GNU89 mode.
static bool canRedefineFunction(const FunctionDecl *FD,
                                const LangOptions& LangOpts) {
  return ((FD->hasAttr<GNUInlineAttr>() || LangOpts.GNUInline) &&
          !LangOpts.CPlusPlus &&
          FD->isInlineSpecified() &&
          FD->getStorageClass() == SC_Extern);
}

pgoodman avatar Aug 17 '23 02:08 pgoodman

@pgoodman Just a quick note that I'll be traveling Thursday afternoon-Sunday and will be AFK.

seelabs avatar Aug 17 '23 03:08 seelabs

Alright I've recently been making progress on this issue. I've been moving our various clang patches into our pasta branch of trail-of-forks/llvm-project. This has been going on as I port things over to Clang 17.

pgoodman avatar Sep 22 '23 01:09 pgoodman

@seelabs Alright, the llvm17 branch of PASTA now works with #include <stop_token> when using g++ as the compiler. This branch of PASTA now has a much simpler build process, and left to its own devices, will build a vendored LLVM (with all of our requisite patches) and nanobind (for Python bindings). I hope to have it merged soon.

pgoodman avatar Sep 24 '23 03:09 pgoodman

@seelabs if you have the time, can you test the master branch?

pgoodman avatar Sep 25 '23 20:09 pgoodman

@pgoodman Nice! I'll test this after work today.

seelabs avatar Sep 26 '23 13:09 seelabs

@pgoodman I built the master branch and ran the following command (similar to what I ran before, but using print-ast instead of dump-ast, which I don't see anymore):

./print-ast /home/swd/apps/clang-latest/bin/clang++ -DBOOST_ASIO_DISABLE_HANDLER_TYPE_REQUIREMENTS -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DBOOST_BEAST_ALLOW_DEPRECATED -DBOOST_CONTAINER_FWD_BAD_DEQUE -DBOOST_COROUTINES_NO_DEPRECATION_WARNING -DBOOST_FILESYSTEM_DEPRECATED -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=\\\"/usr/bin/addr2line\\\" -DBOOST_STACKTRACE_USE_ADDR2LINE -DBOOST_STACKTRACE_USE_BACKTRACE -DBOOST_STACKTRACE_USE_NOOP -DCARES_STATICLIB -DDATE_HEADER_ONLY -DDEBUG -DENABLE_TESTS -DGIT_COMMIT_HASH=\\\"ed9064ce3f6de6da6b63417e59f3100c8e6c4c6d\\\" -DHAS_UNCAUGHT_EXCEPTIONS=1 -DOPENSSL_NO_SSL2 -D_DEBUG -I/home/swd/projs/ripple/mine/src/ed25519-donna -I/home/swd/projs/ripple/mine/src/secp256k1/include -I/home/swd/projs/ripple/mine/src -isystem /home/swd/.conan/data/boost/1.82.0/_/_/package/f81afbcfa21cf6994cf0e45c1b11229c01c767a7/include -isystem /home/swd/.conan/data/openssl/1.1.1u/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/.conan/data/libarchive/3.6.2/_/_/package/d9fea6983af2a77bcbe3f0b088cea5aef9308bd7/include -isystem /home/swd/.conan/data/lz4/1.9.3/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/projs/ripple/mine/cur_build/proto_gen_grpc -isystem /home/swd/.conan/data/grpc/1.50.1/_/_/package/d156c58fd512defbc1155e665684d7875e800ba9/include -isystem /home/swd/.conan/data/abseil/20220623.0/_/_/package/e84b6d643e7f97d33bf72700134618fbae8bbbf6/include -isystem /home/swd/.conan/data/c-ares/1.19.0/_/_/package/ed86cef4487c6b9ce327a03d24ced8142000e8b9/include -isystem /home/swd/.conan/data/re2/20230301/_/_/package/e84b6d643e7f97d33bf72700134618fbae8bbbf6/include -isystem /home/swd/.conan/data/zlib/1.2.13/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/.conan/data/protobuf/3.21.9/_/_/package/681b3b1dffc7e4dc6f5551cdb6e1fe56bdda3e85/include -isystem /home/swd/projs/ripple/mine/cur_build/proto_gen -isystem /home/swd/projs/ripple/mine/cur_build/proto_gen/src/ripple/proto -isystem /home/swd/.conan/data/soci/4.0.3/_/_/package/dc82e3681e64ed36c447531c9e4883a5c616a774/include -isystem /home/swd/.conan/data/bzip2/1.0.8/_/_/package/5e357164f23f7c4e38ebe87433827878dbe1a74c/include -isystem /home/swd/.conan/data/libbacktrace/cci.20210118/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/.conan/data/sqlite3/3.42.0/_/_/package/ae0e146c4c82ac1e46c279d15c964c80fedbdf5f/include -isystem /home/swd/.conan/data/nudb/2.0.8/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/include -isystem /home/swd/.conan/data/date/3.0.1/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/include -m64 -stdlib=libstdc++ -Wall -Wdeprecated -Wextra -Wno-unused-parameter -g -std=c++20 -fPIE -frtti -Wnon-virtual-dtor -Wno-sign-compare -Wno-char-subscripts -Wno-format -Wno-unused-local-typedefs -fstack-protector -o CMakeFiles/rippled.dir/src/ripple/app/ledger/impl/LedgerReplay.cpp.o -c /home/swd/projs/ripple/mine/src/ripple/app/ledger/impl/LedgerReplay.cpp

I get the following error:

A clang diagnostic or uncompilable error was produced when trying to get an AST due to error: /usr/lib/gcc/x86_64-pc-linux-gnu/13.2.1/include/xmmintrin.h:827:1 definition of builtin function '_mm_getcsr'

I'll try to play with this some more later. I know we ran into similar errors earlier (I did not patch master at all).

Edit: Just for grins, I tried to use the clang that past built (/home/swd/projs/common/pasta-clang/bin/clang++) and got the same error as above.

seelabs avatar Sep 26 '23 14:09 seelabs

Excellent. It seems I didn't sufficiently patch clang. I thought I had covered this "definition of builtin function" error but didn't. This gives me something to go on :-)

pgoodman avatar Sep 26 '23 15:09 pgoodman