Building Carla on Mac OSX

[ajax98]

Online I see that there only exist manual for installing and building Carla on Linux. Is it possible for installing and running it on Mac?

[nsubiron]

Hi @ajax98,

Yes, it is possible. However we don't have an automated build system for MacOS, so it would require some effort to make it work.

Please have a look at the progress at #103. Also it might be helpful to check out the documentation for How to build on Linux and the discussion for building on Windows at #21 as the process is basically the same.

Please share your progress here if you find any difficulties. It can be very helpful for other people trying to build on Mac, and eventually we might be able to make an automated system as for Linux.

---

There is a compilation error that is going to appear, you need to replace the block at "Source/Carla/SceneCaptureCamera.cpp" Line 27 by

static constexpr auto DEPTH_MAT_PATH =
#if PLATFORM_LINUX
    TEXT("Material'/Carla/PostProcessingMaterials/DepthEffectMaterial_GLSL.DepthEffectMaterial_GLSL'");
#else
    TEXT("Material'/Carla/PostProcessingMaterials/DepthEffectMaterial.DepthEffectMaterial'");
#endif

This is something we have to fix, but we'll have to test if the default depth material is working well on Mac. Help on this would be appreciated as we don't have here Mac machines to test it.

[nsubiron]

Check out https://github.com/carla-simulator/carla/issues/176#issuecomment-363372488

[thegleb]

I skimmed the couple threads I've seen here about running Carla on a Mac and I have modified all the files as mentioned, but I keep running into the same issue. The .ueproject seems to get generated but I'm missing a couple of .dylibs, e.g. UE4Editor-Carla.dylib and a second one. Any advice?

Output from xcode when trying to build the CarlaUE4 project:

Undefined symbols for architecture x86_64:
  "FRHICommandListBase::WaitForRHIThreadTasks()", referenced from:
      ASceneCaptureCamera::WritePixelsNonBlocking(float, FRHICommandListImmediate&) const in Module.Carla.cpp.o
  "_GRHICommandList", referenced from:
      ASceneCaptureCamera::WritePixelsNonBlocking(float, FRHICommandListImmediate&) const in Module.Carla.cpp.o
      ASceneCaptureCamera::WritePixels(float) const in Module.Carla.cpp.o
  "FRHICommandListBase::WaitForDispatch()", referenced from:
      ASceneCaptureCamera::WritePixelsNonBlocking(float, FRHICommandListImmediate&) const in Module.Carla.cpp.o
  "FRHICommandListExecutor::ExecuteList(FRHICommandListImmediate&)", referenced from:
      ASceneCaptureCamera::WritePixelsNonBlocking(float, FRHICommandListImmediate&) const in Module.Carla.cpp.o
  "FlushRenderingCommands()", referenced from:
      FServerGameController::Tick(float) in Module.Carla.cpp.o
  "_GIsRunningRHIInSeparateThread_InternalUseOnly", referenced from:
      ASceneCaptureCamera::WritePixelsNonBlocking(float, FRHICommandListImmediate&) const in Module.Carla.cpp.o
  "_GMainThreadBlockedOnRenderThread", referenced from:
      ASceneCaptureCamera::Tick(float) in Module.Carla.cpp.o
  "_GIsThreadedRendering", referenced from:
      ASceneCaptureCamera::Tick(float) in Module.Carla.cpp.o
      FRenderCommand::GetDesiredThread() in Module.Carla.cpp.o
  "_GMaxRHIShaderPlatform", referenced from:
      ASceneCaptureCamera::Tick(float) in Module.Carla.cpp.o
  "FRHICommandListBase::WaitForTasks(bool)", referenced from:
      ASceneCaptureCamera::WritePixelsNonBlocking(float, FRHICommandListImmediate&) const in Module.Carla.cpp.o
  "_GDynamicRHI", referenced from:
      ASceneCaptureCamera::WritePixelsNonBlocking(float, FRHICommandListImmediate&) const in Module.Carla.cpp.o
      ASceneCaptureCamera::WritePixels(float) const in Module.Carla.cpp.o
  "GetImmediateCommandList_ForRenderCommand()", referenced from:
      ASceneCaptureCamera::Tick(float) in Module.Carla.cpp.o
      TGraphTask<ASceneCaptureCamera::Tick(float)::EURCMacro_FWritePixelsNonBlocking>::ExecuteTask(TArray<FBaseGraphTask*, FDefaultAllocator>&, ENamedThreads::Type) in Module.Carla.cpp.o
      TGraphTask<ASceneCaptureCamera::Tick(float)::EURCMacro_FWritePixels>::ExecuteTask(TArray<FBaseGraphTask*, FDefaultAllocator>&, ENamedThreads::Type) in Module.Carla.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ERROR: UBT ERROR: Failed to produce item: /Users/gleb/dev/deep-learning-learning/carla/Unreal/CarlaUE4 4.18/Plugins/Carla/Binaries/Mac/UE4Editor-Carla.dylib
Total build time: 26.00 seconds (Local executor: 0.00 seconds)
Command /Users/gleb/UnrealEngine_4.18/Engine/Build/BatchFiles/Mac/Build.sh failed with exit code 5

When I open the project through UE4Editor, it complains about the lack of the dylibs and tries to build them, but fails every time as well.

[thegleb]

Update: got it to complete the build by modifying the Carla.build.cs further:

    PublicDependencyModuleNames.AddRange(
      new string[]
      {
        "Core",
        // ... add other public dependencies that you statically link with here ...
        "RHI", // added this
        "RenderCore" // and this
      }
      );

[analog-cbarber]

I got this working using Xcode 8.2.1, which is based on clang 3.9. I had to modify the scripts to use clang/ clang++ without the version suffix. I did run into one problem that I do not understand: the Makefile generated by the GenerateProjectFiles script is targeted at Linux instead of Mac :

BUILD = bash "$(UNREALROOTPATH)/Engine/Build/BatchFiles/Linux/Build.sh"
PROJECTBUILD = mono "$(UNREALROOTPATH)/Engine/Binaries/DotNET/UnrealBuildTool.exe"
...
UE4Editor:
         $(BUILD) UE4Editor Linux Development  $(ARGS)

When I manually changed Linux to Mac and add mono to my PATH, I was able to complete the build. I cannot figure out how to fix the Makefile generation. Invoking the GenerateProjectFiles.sh script with -Platforms=Mac doesn't do any good. Not sure what I did wrong.

In any case, if I modify the generated Makefile by replacing all instances of Linux with Mac and replacing modifying the PROJECTBUILD definition to be the following, then the build works:

MONO = "$(UNREALROOTPATH)/Engine/Binaries/ThirdParty/Mono/Mac/bin/mono"
PROJECTBUILD = $(MONO) "$(UNREALROOTPATH)/Engine/Binaries/DotNET/UnrealBuildTool.exe"

[analog-cbarber]

It appears that the -makefiles flag to the UnrealBuildTool does not properly support Mac targets. The -cmakefiles appears to do the right thing, but I have not actually tried doing a build yet. Since it is not clear what version of llvm/clang is used by later releases of Xcode, and the 0.9 release of CARLA is using clang 5.0, I suspect that using cmake will be preferable to Xcode.

[nsubiron]

@analog-cbarber Thanks for the report!

In Linux we use a very specific version of the compiler because we need to link against libc++ (instead of the usual libstdc++ for Linux). In Mac, I imagine we can go with the default compiler installed like we do on Windows.

And for the sh scripts, they have been written for Linux and only tested on Linux, we may need to adapt them.

[analog-cbarber]

That's good, so the Xcode build approach may still be ok? So the main risk would be if CARLA started to depend on C++ features in clang 5.0 that are not yet supported in the Xcode variant?

[nsubiron]

That's likely to happen, in the latest changes in networking code we use C++17 features, most of them already available in previous versions of clang with -std=c++1z but some may not.

[analog-cbarber]

OK. So either I need to figure out how to build properly using actual llvm/clang 5.0 or hope that Xcode supports the required features. It seems that Xcode 10 may be based on 5.0 (see Joky's comment on this thread: https://news.ycombinator.com/item?id=16545037)

[analog-cbarber]

According to https://clang.llvm.org/cxx_status.html, it appears that clang 4.0 should support most of the C++17 features. The only ones I see that are missing are constexpr lambda expressions (P0170R1) and template argument deduction for class templates (P0091R3 and P0512R0). If you aren't using either of those features, then I would expect clang++ 4.0 and Xcode 9.2 to work. That would be nice if true, since the official 4.19 release appears to be built with Xcode 9.2.

[nsubiron]

I don't think we use any of those features, up until recently all our code base was compiling with clang-3.9 and -std=c++14, it's going to be starting at CARLA 0.9.0 that we move to clang-5.0 and C++17. The C++17 standard has some nice features that may help performance, and hopefully soon will be fully supported by most compilers.

So for now shouldn't be any problems compiling. However, we won't be maintaining compatibility as we simply don't have enough people to maintain it.

[nsubiron]

Hey @analog-cbarber, so after your comments I've been digging in the UE4 toolchains, and they seem to be forcing C++14 in all platforms (as for UE4.19). I think it would be wise for us to restrict our libraries to C++14 too since some of the code may need to compile inside UE4 environment too. So I'm going to keep CARLA 0.9.0 with C++14 standard (still clang-5.0 by default as it's the recommended for UE4.19 in Linux).

I'm a bit sad about it cause I wanted to start using C++17 :smile: but at least it's good to keep compatibility since at this point almost every compiler supports C++14.

[analog-cbarber]

You could still probably get away with using C++17 features that are compatible with whatever runtime UE 4.19 is using. I am not sure which ones that would be...

FWIW, the changes I made to get the Mac build to work using Xcode 8.2.1 are here:

https://github.com/analog-garage/carla/commits/mac-build

I won't issue a pull request, because this is hackier than I would like and will require significant reworking for the 0.9 release in any case.

[edufford]

@analog-cbarber, I was able to get Mac command line setup/builds working using homebrew-installed llvm/clang 3.9 with some modifications to the Carla scripts and cmake setup file instead of installing another version of Xcode.

I think the main trick was to prevent the libc++ library conflict with Apple's built-in version by using the -nostdinc++ flag for using an alternate installation of libc++ based on the libcxx docs at: https://libcxx.llvm.org/docs/UsingLibcxx.html#getting-started

Getting all the directory changes correct for the Unreal Engine command line project build and launch was also a little tricky, but seems to work. My Unreal Engine is installed from the Epic Games Launcher instead of from source.

You can check out my modifications for this from the current Carla master at: https://github.com/edufford/carla/commit/78e9e1919d30f7be84a3e47bc6cf053eca664a68

My environment (for reference):

• macOS High Sierra 10.13.5
• Xcode 9.4.1
• Clang 3.9.1 (installed by homebrew llvm-3.9)
• UE Editor 4.18.3 (installed from Mac Epic Games Launcher)

[edufford]

Also, for installing the Mac build dependencies, I used:

• LLVM/Clang 3.9 brew tap llvm-hs/homebrew-llvm brew install llvm-3.9

• tac command by symbolic link brew install coreutils ln -s /usr/local/bin/gtac /usr/local/bin/tac

• Others brew install cmake autoconf automake ninja libtool wget

• Python 3.6 (installed by Conda) Requests package pip install requests

When building the plugins in Unreal Engine, I also had to downgrade errors for unused lambda capture following https://answers.unrealengine.com/questions/779733/xcode-build-errors-lambda-capturenot-used.html:

1. Modify Engine/Source/Programs/UnrealBuildTool/Platform/MacToolChain.cs to add:

   Result += " -Wno-unused-lambda-capture";
   

   after the existing line:

   Result += " -Wno-unused-value";
   

2. Run xbuild UnrealBuildTool.csproj (deprecated) or msbuild UnrealBuildTool.csproj from the Engine/Source/Programs/UnrealBuildTool/ directory

[raskolnikov-reborn]

I get the "UE4Editor-Carla.dylib missing" along with one more. Rebuilding fails and it says try from source manually. Any help with this?

[edufford]

The missing dylib messages is normal, just needs the UE rebuild to work. Are you using the latest Carla 0.9.0? It was updated for Unreal Engine 4.19 instead of 4.18 so there may be new UE rebuild issues. I haven’t had a chance to try it with UE 4.19 yet.

[analog-cbarber]

I got a Mac build working based on the current master branch (i.e. 0.9.0) using Xcode 9.2 and the standard UE 4.19 download, so there is no need to build libc++ or the Unreal Engine.

Some changes were required.

I have pushed what I have so far on the mac-build-0.9 branch of our fork.

See https://github.com/analog-garage/carla/blob/mac-build-0.9/Docs/how_to_build_on_mac.md

[edufford]

@analog-cbarber Nice work!

I was able to use your fork's mac-build-0.9 branch to build Carla 0.9 with Xcode 9.4.1 as well, after modifying the Xcode version check in the Util/BuildTools/Setup.command file.

Copying the GenerateProjectFiles.sh to the UE root folder didn't work for me though, so instead I was able to modify the Util/BuildTools/BuildCarlaUE4.command file to call the built in Mac GenerateProjectFiles.sh that comes with the UE engine install package in place:

Util/BuildTools/BuildCarlaUE4.command

log "Generate Unreal project files."

#if [[ ! -f ${UE4_ROOT}/GenerateProjectFiles.sh ]]; then
#  fatal_error "No GenerateProjectFiles.sh in ${UE4_ROOT}. You can copy this from UnrealEngine source tree."
#fi

#${UE4_ROOT}/GenerateProjectFiles.sh -project="${PWD}/CarlaUE4.uproject" -game -engine -makefiles

# HACK! This generates wrong targets on the Mac! Just replace Linux with Mac everywhere:
#sed -i .original -e "s/Linux/Mac/g" Makefile

# GenerateProjectFiles.sh requires to change directory to location of UE engine Mac batch files
pushd "${UE4_ROOT}/Engine/Build/BatchFiles/Mac/" >/dev/null
./GenerateProjectFiles.sh -project="${CARLAUE4_ROOT_FOLDER}/CarlaUE4.uproject" -game -engine -makefiles
popd >/dev/null

The Xcode project workspace Unreal/CarlaUE4/CarlaUE4.xcworkspace that this generates worked fine to Build & Run the project and launch the UE Editor. It would be great if your how_to_build_on_mac.md doc also included some info about using this file to build/run the project too.

For the command line UE project build though, instead of using make with the Linux->Mac swapped UE Makefile, I had to modify the Util/BuildTools/BuildCarlaUE4.command file to use UE's Mac Build.sh script directly and add -NoUBTMakefiles to prevent an error about invalid makefile contents from the UE Build Tool:

Util/BuildTools/BuildCarlaUE4.command

log "Build CarlaUE4 project."
#make CarlaUE4Editor

# Build.sh requires to change directory to location of UE engine root
pushd "${UE4_ROOT}" >/dev/null
./Engine/Build/BatchFiles/Mac/Build.sh UE4Editor Mac Development -NoUBTMakefiles
popd >/dev/null

For Mac, Xcode is probably the way to go anyways as long as the AppleClang version is still compatible. Thanks again for posting your work!

[analog-cbarber]

Did you copy GenerateProjectFiles.sh from the root of the Unreal Engine source tree? I think that should work. I would have checked in a copy of the file I used if not for the conflicting copyright.

On Aug 20, 2018 3:29 PM, Effendi Dufford [email protected] wrote:

@analog-cbarberhttps://github.com/analog-cbarber Nice work!

I was able to use your fork's mac-build-0.9 branch to build Carla 0.9 with Xcode 9.4.1 as well, after modifying the Xcode version check in the Util/BuildTools/Setup.command file.

Copying the GenerateProjectFiles.sh to the UE root folder didn't work for me though, so instead I was able to modify the Util/BuildTools/BuildCarlaUE4.command file to call the built in Mac GenerateProjectFiles.sh that comes with the UE engine install package in place:

Util/BuildTools/BuildCarlaUE4.command

log "Generate Unreal project files."

#if [[ ! -f ${UE4_ROOT}/GenerateProjectFiles.sh ]]; then

fatal_error "No GenerateProjectFiles.sh in ${UE4_ROOT}. You can copy this from UnrealEngine source tree."

#fi

#${UE4_ROOT}/GenerateProjectFiles.sh -project="${PWD}/CarlaUE4.uproject" -game -engine -makefiles

HACK! This generates wrong targets on the Mac! Just replace Linux with Mac everywhere:

#sed -i .original -e "s/Linux/Mac/g" Makefile

GenerateProjectFiles.sh requires to change directory to location of UE engine Mac batch files

pushd "${UE4_ROOT}/Engine/Build/BatchFiles/Mac/" >/dev/null ./GenerateProjectFiles.sh -project="${CARLAUE4_ROOT_FOLDER}/CarlaUE4.uproject" -game -engine -makefiles popd >/dev/null

The Xcode project workspace Unreal/CarlaUE4/CarlaUE4.xcworkspace that this generates worked fine to Build & Run the project and launch the UE Editor. It would be great if your how_to_build_on_mac.md doc also included some info about using this file to build/run the project too.

For the command line UE project build though, instead of using make with the Linux->Mac swapped UE Makefile, I had to modify the Util/BuildTools/BuildCarlaUE4.command file to use UE's Mac Build.sh script directly and add -NoUBTMakefiles to prevent an error about invalid makefile contents from the UE Build Tool:

Util/BuildTools/BuildCarlaUE4.command

log "Build CarlaUE4 project." #make CarlaUE4Editor

Build.sh requires to change directory to location of UE engine root

pushd "${UE4_ROOT}" >/dev/null ./Engine/Build/BatchFiles/Mac/Build.sh UE4Editor Mac Development -NoUBTMakefiles popd >/dev/null

For Mac, Xcode is probably the way to go anyways as long as the AppleClang version is still compatible. Thanks again for posting your work!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/carla-simulator/carla/issues/150#issuecomment-414434501, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHEKc4MWwqyif8QWydT_yWu4eM4nD6cjks5uSw3-gaJpZM4Rj_40.

[analog-cbarber]

Yes, it should be possible to build everything using Xcode and plan to add am option for that if not make it the default on the Mac.

On Aug 20, 2018 3:29 PM, Effendi Dufford [email protected] wrote:

@analog-cbarberhttps://github.com/analog-cbarber Nice work!

I was able to use your fork's mac-build-0.9 branch to build Carla 0.9 with Xcode 9.4.1 as well, after modifying the Xcode version check in the Util/BuildTools/Setup.command file.

Copying the GenerateProjectFiles.sh to the UE root folder didn't work for me though, so instead I was able to modify the Util/BuildTools/BuildCarlaUE4.command file to call the built in Mac GenerateProjectFiles.sh that comes with the UE engine install package in place:

Util/BuildTools/BuildCarlaUE4.command

log "Generate Unreal project files."

#if [[ ! -f ${UE4_ROOT}/GenerateProjectFiles.sh ]]; then

fatal_error "No GenerateProjectFiles.sh in ${UE4_ROOT}. You can copy this from UnrealEngine source tree."

#fi

#${UE4_ROOT}/GenerateProjectFiles.sh -project="${PWD}/CarlaUE4.uproject" -game -engine -makefiles

HACK! This generates wrong targets on the Mac! Just replace Linux with Mac everywhere:

#sed -i .original -e "s/Linux/Mac/g" Makefile

GenerateProjectFiles.sh requires to change directory to location of UE engine Mac batch files

pushd "${UE4_ROOT}/Engine/Build/BatchFiles/Mac/" >/dev/null ./GenerateProjectFiles.sh -project="${CARLAUE4_ROOT_FOLDER}/CarlaUE4.uproject" -game -engine -makefiles popd >/dev/null

The Xcode project workspace Unreal/CarlaUE4/CarlaUE4.xcworkspace that this generates worked fine to Build & Run the project and launch the UE Editor. It would be great if your how_to_build_on_mac.md doc also included some info about using this file to build/run the project too.

For the command line UE project build though, instead of using make with the Linux->Mac swapped UE Makefile, I had to modify the Util/BuildTools/BuildCarlaUE4.command file to use UE's Mac Build.sh script directly and add -NoUBTMakefiles to prevent an error about invalid makefile contents from the UE Build Tool:

Util/BuildTools/BuildCarlaUE4.command

log "Build CarlaUE4 project." #make CarlaUE4Editor

Build.sh requires to change directory to location of UE engine root

pushd "${UE4_ROOT}" >/dev/null ./Engine/Build/BatchFiles/Mac/Build.sh UE4Editor Mac Development -NoUBTMakefiles popd >/dev/null

For Mac, Xcode is probably the way to go anyways as long as the AppleClang version is still compatible. Thanks again for posting your work!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/carla-simulator/carla/issues/150#issuecomment-414434501, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHEKc4MWwqyif8QWydT_yWu4eM4nD6cjks5uSw3-gaJpZM4Rj_40.

[edufford]

Ah, I was looking for the GenerateProjectFiles.sh file in the /UE_4.19/Engine/Source/ folder since I installed it from the Epic Games Launcher instead of from the full source package, so I could only find the file in the /UE_4.19/Engine/Build/BatchFiles/Mac/ folder and copying that one over didn't work.

It did work when I downloaded the GenerateProjectFiles.sh file from Epic's source repo as you linked, but it looks like this script is basically just running:

cd Engine/Build/BatchFiles/Mac
sh ./GenerateLLDBInit.sh
sh ./GenerateProjectFiles.sh $@

So it seems that if the BuildCarlaUE4.command file just manually ran these scripts directly, similar to my modification, then there would be no need for copying this file over from the main engine source.

[danxuhk]

when run python example.py, it shows 'AttributeError: 'module' object has no attribute 'Client' 'in Traceback (most recent call last): File "example.py", line 114, in main(add_a_camera=False, enable_autopilot=True) File "example.py", line 43, in main client = carla.Client('localhost', 2000) Any idea on this issue?

[analog-cbarber]

Works for me. Are you using python 3? I don't think that works yet.

[danxuhk]

I am using python 2.7. After the compilation, I copied the folder 'carla' under PythonClient to PythonAPI, and then run python example.py. The issue comes out. Actually which .py file under carla defines the Client class? I am not able to find a definition from there.

[analog-cbarber]

Not sure where you got the idea to copy that directory. The new API is mostly C++ now. You need to install the egg in PythonAPI/dist into your python environment using easy_install.

On Aug 23, 2018 1:28 PM, Dan Xu [email protected] wrote:

I am using python 2.7. After the compilation, I copied the folder 'carla' under PythonClient to PythonAPI, and then run python example.py. The issue comes out. Actually which .py file under carla defines the Client class? I am not able to find a definition from there.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/carla-simulator/carla/issues/150#issuecomment-415502154, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHEKcxSBsFALXRbVte3Y4BGsMj0bbVvdks5uTuYkgaJpZM4Rj_40.

[analog-cbarber]

I pushed some changes to my fork that allow later versions of Xcode and use Xcode by default to build everything except boost.

[danxuhk]

The example.py can be run successfully, while I run python client_example.py, another issue of failed to read data comes out: INFO: listening to server localhost:2000 CarlaClient connected ERROR: (localhost:2000) failed to read data: timed out CarlaClient connected ERROR: (localhost:2000) failed to read data: timed out CarlaClient connected ERROR: (localhost:2000) failed to read data: timed out CarlaClient connected ERROR: (localhost:2000) failed to read data: timed out

Have you ever encountered something like this?

[analog-cbarber]

I haven't seen that. Are you sure you don't have another program using port 2000?