ClangSharp icon indicating copy to clipboard operation
ClangSharp copied to clipboard

CXTranslationUnit.TryParse results in CXTranslationUnit with CX_DeclKind_ObjCPropertyImpl cursor while parsing Windows targeted files

Open hypeartist opened this issue 4 years ago • 4 comments

Here is the code (based on @xoofx setup approach):

			const string root = "cppast.input";
   
			var filesToParse = new List<string>
			{
				@"c:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\shared\windef.h",
				@"C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\Include\um\corerror.h",
				@"e:\BigRepos\runtime\src\coreclr\src\inc\corhdr.h",
				@"e:\BigRepos\runtime\src\coreclr\src\inc\corjit.h"
			};
   
			var sysIncludes = new List<string>
			{
				@"c:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\um\",
				@"c:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\shared\",
				@"c:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\ucrt\"
			};
   
			var includes = new List<string>
			{
				@"C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\Include\um\",
				@"e:\BigRepos\runtime\src\coreclr\src\inc\"
			};
   
			var defines = new List<string>
			{
				"_AMD64_=1",
				"TARGET_AMD64=1",
				"_MSC_VER=1920",
				"_WIN32=1",
				"_M_AMD64=100",
				"_M_X64=100",
				"_WIN64=1"
			};
   
			var arguments = new List<string>
			{
				"-Wno-pragma-once-outside-header",
				"-fms-extensions",
				"-fms-compatibility",
				"-fms-compatibility-version=19.20",
				"-dM",
				"-E",
				"-xc++",
				"--target=x86_64-pc-windows-msvc19.20",
				"-fparse-all-comments"
			};
			var normalizedIncludePaths = new List<string>();
			normalizedIncludePaths.AddRange(includes.Select(x => Path.Combine(Environment.CurrentDirectory, x)));
   
			var normalizedSystemIncludePaths = new List<string>();
			normalizedSystemIncludePaths.AddRange(sysIncludes.Select(x => Path.Combine(Environment.CurrentDirectory, x)));
   
			arguments.AddRange(normalizedIncludePaths.Select(x => $"-I{x}"));
			arguments.AddRange(normalizedSystemIncludePaths.Select(x => $"-isystem{x}"));
			arguments.AddRange(defines.Select(x => $"-D{x}"));
   
			var translationFlags = CXTranslationUnit_Flags.CXTranslationUnit_None;
            translationFlags |= CXTranslationUnit_Flags.CXTranslationUnit_SkipFunctionBodies;
            translationFlags |= CXTranslationUnit_Flags.CXTranslationUnit_IncludeAttributedTypes;
            translationFlags |= CXTranslationUnit_Flags.CXTranslationUnit_VisitImplicitAttributes;
            translationFlags |= CXTranslationUnit_Flags.CXTranslationUnit_DetailedPreprocessingRecord;
            
var createIndex = CXIndex.Create();
   
            var tempBuilder = new StringBuilder();
            foreach (var file in filesToParse)
            {
	            var filePath = Path.Combine(Environment.CurrentDirectory, file);
	            tempBuilder.AppendLine($"#include \"{filePath}\"");
			}
			var rootFileContent = tempBuilder.ToString();
			var rootFileContentBlob = Encoding.UTF8.GetBytes(rootFileContent);
			fixed (void* rootFileContentPtr = rootFileContentBlob)
			{
				var rootFileName = Marshal.StringToHGlobalAnsi(root);
				CXTranslationUnit.TryParse(createIndex, root, arguments.ToArray(), new[]
				{
					new CXUnsavedFile
					{
						Contents = (sbyte*) rootFileContentPtr,
						Filename = (sbyte*) rootFileName,
						Length = new UIntPtr((uint) rootFileContentBlob.Length)
					}
				}, translationFlags, out var translationUnit);
				var tu = TranslationUnit.GetOrCreate(translationUnit);
				using var outputStream = new MemoryStream();
				var generator = new PInvokeGenerator(null, s => outputStream);
				generator.GenerateBindings(tu);
			}

But strangely TryParse out-returns translationUnit which cursor has DeclKind = CX_DeclKind_ObjCPropertyImpl and then all goes boom here since CXCursorKind is CXCursor_TranslationUnit:

    public sealed class ObjCPropertyImplDecl : Decl
    {
        internal ObjCPropertyImplDecl(CXCursor handle) : base(handle, handle.Kind, CX_DeclKind.CX_DeclKind_ObjCPropertyImpl)
        {
            if ((handle.Kind != CXCursorKind.CXCursor_ObjCSynthesizeDecl) && (handle.Kind != CXCursorKind.CXCursor_ObjCDynamicDecl))
            {
                throw new ArgumentException(nameof(handle));
            }
        }
    }

Do I do something wrong or miss something?

hypeartist avatar Jun 28 '20 21:06 hypeartist

Could you clarify what version of libClang is being loaded? I'd guess there is a mismatch between what you have on the box and the version of ClangSharp you are referencing

tannergooding avatar Jun 29 '20 02:06 tannergooding

I'd guess there is a mismatch between what you have on the box and the version of ClangSharp you are referencing

That was it!!! For some reasons I've been referencing not the latest libclang.dll! After I have copied the one from c:\Users\hypeartist\.nuget\packages\libclang.runtime.win-x64\10.0.0\runtimes\win-x64\native\ the issue seems to be resolved and now cursor's DeclKind is CX_DeclKind_LastDecl.

Btw, I can't make a reference to libclang to being copied into bin folder so I have to copy it manually the path above.

hypeartist avatar Jun 29 '20 04:06 hypeartist

Some off-topic points:

  1. Names of nested types are represented as ParentTypeName::TypeName (which is incorrect, i guess), so I had to add return name.Replace("::", ".") in string GetTypeName(Cursor cursor, Cursor context, Type type, out string nativeTypeName) method as a quick-n-dirty workaround.

  2. Method parameters default value assignment seems to be kinda broken for pointer types. Now it checks (in bool VisitImplicitCastExpr(ImplicitCastExpr implicitCastExpr, IntegerLiteral integerLiteral)) like:

	        if (implicitCastExpr.Type is PointerType)
            {
                if (integerLiteral.Value.Equals("0"))
                {
                    // C# doesn't have implicit conversion from zero to a pointer
                    // so we will manually check and handle the most common case

                    _outputBuilder.Write("null");
                    return true;
                }

                return false;
            }

But it goes wrong if implicitCastExpr.Type is a typedef for a pointer type. So I had to change

if (implicitCastExpr.Type is PointerType)

into:

if (implicitCastExpr.Type.CanonicalType is PointerType)

to make it work as expected.

  1. Enum values that are references to another enum are not prefixed with owner's enum name:
    public enum CorElementType
    {
		...
        ELEMENT_TYPE_BOOLEAN = 0x02,
		...
    }

    public enum CorSerializationType
    {
        ...
        SERIALIZATION_TYPE_BOOLEAN = ELEMENT_TYPE_BOOLEAN
		...
    }

So I've ended up with "patching" void VisitDeclRefExpr(DeclRefExpr declRefExpr) with this:

        private void VisitDeclRefExpr(DeclRefExpr declRefExpr)
        {
            var name = GetRemappedCursorName(declRefExpr.Decl);
			// PATCH_START
            if (declRefExpr.Decl.Kind == CX_DeclKind.CX_DeclKind_EnumConstant && declRefExpr.Decl.DeclContext != declRefExpr.DeclContext)
            {
	            name = $"{((EnumDecl) declRefExpr.Decl.DeclContext).Name}.{name}";
            }
			// PATCH_END
            _outputBuilder.Write(EscapeAndStripName(name));
        }

Will be happy to know whether all those tweaks are really needed or did I miss something (again) I'd better to know )

hypeartist avatar Jun 29 '20 07:06 hypeartist

There are a few scenarios, especially as you get to more C++ oriented code, that aren't going to be correctly handled today. It's something that I typically fix as the issues are raised.

These all sound like issues that could be fairly easily resolved with tests added to ensure the behavior works as expected. In the case of the pointer type being a typedef, there are likely similar scenarios around attributed and elaborated types.

tannergooding avatar Jun 29 '20 14:06 tannergooding

Closing this as the original issue was addressed.

Newer versions of ClangSharp now validate the native library version loaded matches the expected version.

tannergooding avatar Sep 18 '22 16:09 tannergooding