Optimize WebRTC build
I tried to build commit e20a79928 on Windows 10 with VS2022 x64 and got the following error:
[0:27:28] src/third_party
src/buildtools/win-format:565cab9c66d61360c27c7d4df5defe1a78ab56d3 (ERROR)
----------------------------------------
[0:01:56] Started.
----------------------------------------
9>CustomBuild:
cipher_aes_gcm_siv_polyval.c
18>CustomBuild:
Traceback (most recent call last):
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\metrics.py", line 302, in print_notice_and_exit
yield
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gclient.py", line 4706, in <module>
sys.exit(main(sys.argv[1:]))
^^^^^^^^^^^^^^^^^^
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gclient.py", line 4692, in main
return dispatcher.execute(OptionParser(), argv)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\subcommand.py", line 262, in execute
return command(parser, args[1:])
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gclient.py", line 4028, in CMDsync
ret = client.RunOnDeps('update', args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gclient.py", line 2450, in RunOnDeps
work_queue.flush(revision_overrides,
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gclient_utils.py", line 1045, in flush
reraise(e[0], e[1], e[2])
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gclient_utils.py", line 54, in reraise
raise value
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gclient_utils.py", line 1124, in run
self.item.run(*self.args, **self.kwargs)
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gclient.py", line 2746, in run
self.DownloadGoogleStorage()
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gclient.py", line 2854, in DownloadGoogleStorage
raise Exception(f'{code}: {err}')
Exception: 1: Traceback (most recent call last):
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gsutil.py", line 367, in <module>
sys.exit(main())
^^^^^^
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gsutil.py", line 363, in main
return run_gsutil(args.target, args.args, clean=args.clean)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gsutil.py", line 236, in run_gsutil
gsutil_bin = ensure_gsutil(VERSION, target, clean)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools-prefix\src\depot-tools\gsutil.py", line 120, in ensure_gsutil
target_zip.extractall(download_dir)
File "C:\Users\bjoer\AppData\Local\vpython-root.0\store\cpython+gdee0q11ohi0s4eqh0uqprvov4\contents\bin\Lib\zipfile.py", line 1701, in extractall
self._extract_member(zipinfo, path, pwd)
File "C:\Users\bjoer\AppData\Local\vpython-root.0\store\cpython+gdee0q11ohi0s4eqh0uqprvov4\contents\bin\Lib\zipfile.py", line 1755, in _extract_member
open(targetpath, "wb") as target:
^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\tt5dist\\TeamTalk5_x64\\TeamTalk5\\Build\\build-win64\\Library\\TeamTalkLib\\build\\webrtc\\depot-tools-prefix\\src\\depot-tools\\external_bin\\gsutil\\twhef23lj\\d\\
gsutil\\third_party\\google-auth-library-python\\tests\\data\\impersonated_service_account_with_quota_project.json'
Running: 'C:\Users\bjoer\AppData\Local\vpython-root.0\store\python_venv-jobqra5cp6rd3n3ialv669q9o8\contents\Scripts\python3.exe' 'C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools
-prefix\src\depot-tools\gclient.py' root
Running: 'C:\Users\bjoer\AppData\Local\vpython-root.0\store\python_venv-jobqra5cp6rd3n3ialv669q9o8\contents\Scripts\python3.exe' 'C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools
-prefix\src\depot-tools\gclient.py' config --spec 'solutions = [
{
"name": "src",
"url": "https://webrtc.googlesource.com/src.git",
"deps_file": "DEPS",
"managed": False,
"custom_deps": {},
},
]
'
Running: 'C:\Users\bjoer\AppData\Local\vpython-root.0\store\python_venv-jobqra5cp6rd3n3ialv669q9o8\contents\Scripts\python3.exe' 'C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\Library\TeamTalkLib\build\webrtc\depot-tools
-prefix\src\depot-tools\gclient.py' sync --nohooks --with_branch_heads
Subprocess failed with return code 1.
18>C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\CMakeFiles\055eb5c207f5df
d6a6d0430f953a0311\webrtc-src-mkdir.rule;C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\CMakeFiles\055eb5c207f5dfd6a6d0430f953a0311\webrtc-src-download.rule;C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\CMakeFiles\05
5eb5c207f5dfd6a6d0430f953a0311\webrtc-src-update.rule;C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\CMakeFiles\055eb5c207f5dfd6a6d0430f953a0311\webrtc-src-patch.rule;C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\CMa
keFiles\055eb5c207f5dfd6a6d0430f953a0311\webrtc-src-configure.rule;C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\CMakeFiles\055eb5c207f5dfd6a6d0430f953a0311\webrtc-src-build.rule;C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\bu
ild-win64\CMakeFiles\055eb5c207f5dfd6a6d0430f953a0311\webrtc-src-install.rule;C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build\build-win64\CMakeFiles\65c21a2c36cf62ac6bff0ee693c745dd\webrtc-src-complete.rule;C:\tt5dist\TeamTalk5_x64\Team
Talk5\Build\build-win64\CMakeFiles\42229b31660bfaba53a89ead87eb426e\webrtc-src.rule;C:\tt5dist\TeamTalk5_x64\TeamTalk5\Library\TeamTalkLib\build\webrtc\CMakeLists.txt' exited with code 1. [C:\tt5dist\TeamTalk5_x64\TeamTalk5\Build
\build-win64\Library\TeamTalkLib\build\webrtc\webrtc-src.vcxproj]
PATH length exceeds 260 characters. Try again with a shorter path.
Hm, on macOS I get the following error:
[149/489] Performing configure step for 'depot-tools'
Traceback (most recent call last):
File "/Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/gsutil.py", line 367, in <module>
sys.exit(main())
~~~~^^
File "/Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/gsutil.py", line 363, in main
return run_gsutil(args.target, args.args, clean=args.clean)
File "/Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/gsutil.py", line 319, in run_gsutil
return luci_context(cmd).returncode
~~~~~~~~~~~~^^^^^
File "/Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/gsutil.py", line 167, in luci_context
p = _luci_auth_cmd('context', wrapped_cmds=cmd)
File "/Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/gsutil.py", line 192, in _luci_auth_cmd
return _run_subprocess(cmd, interactive)
File "/Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/gsutil.py", line 206, in _run_subprocess
return subprocess.run(cmd, **kwargs)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
@bear101, Please retry the build.
Notice DEPOTTOOLS_EXPORT is set after ExternalProject_Add
[49/489] Performing configure step for 'depot-tools'
FAILED: [code=2] Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools-stamp/depot-tools-configure /Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools-stamp/depot-tools-configure
cd /Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools-build && && /Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/ensure_bootstrap && /usr/local/bin/cmake -E touch /Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools-stamp/depot-tools-configure
/bin/sh: -c: line 0: syntax error near unexpected token `&&'
/bin/sh: -c: line 0: `cd /Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools-build && && /Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/ensure_bootstrap && /usr/local/bin/cmake -E touch /Volumes/BEAR-1/MacOS/TeamTalk5/Build/build-mac/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools-stamp/depot-tools-configure'
Now I'm trying to build WebRTC for Android:
[0:37:16] Still working on:
[0:37:16] src/third_party/tflite/src
[0:37:19] Still working on:
[0:37:19] src/third_party/tflite/src
Warnings:
git update is recommended.
Installed git version is 2.43.0;
depot_tools recommends version 2.46.0 or later.
Disable this warning by setting the GCLIENT_SUPPRESS_GIT_VERSION_WARNING
environment variable to 1.
Running: gclient root
Running: gclient config --spec 'solutions = [
{
"name": "src",
"url": "https://webrtc.googlesource.com/src.git",
"deps_file": "DEPS",
"managed": False,
"custom_deps": {},
},
]
target_os = ["android", "unix"]
'
Running: gclient sync --nohooks --with_branch_heads
Running: git config --add remote.origin.fetch '+refs/tags/*:refs/tags/*'
Running: git config diff.ignoreSubmodules dirty
Previous HEAD position was f672636abf Privatize signals in async_packet_socket.h
HEAD is now at 060ae594b3 Modify Demo for Linux
src/buildtools/linux64-format:b42097ca924d1f1736a5a7806068fed9d7345eb4 (ERROR)
----------------------------------------
[0:00:03] Started.
----------------------------------------
Traceback (most recent call last):
File "/BearWare/TeamTalk5/Build/build-android-armeabi-v7a/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/metrics.py", line 302, in print_notice_and_exit
yield
File "/BearWare/TeamTalk5/Build/build-android-armeabi-v7a/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/gclient.py", line 4706, in <module>
sys.exit(main(sys.argv[1:]))
^^^^^^^^^^^^^^^^^^
File "/BearWare/TeamTalk5/Build/build-android-armeabi-v7a/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/gclient.py", line 4692, in main
return dispatcher.execute(OptionParser(), argv)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/BearWare/TeamTalk5/Build/build-android-armeabi-v7a/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/subcommand.py", line 262, in execute
return command(parser, args[1:])
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/BearWare/TeamTalk5/Build/build-android-armeabi-v7a/Library/TeamTalkLib/build/webrtc/depot-tools-prefix/src/depot-tools/gclient.py", line 4028, in CMDsync
ret = client.RunOnDeps('update', args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
There's no exact error message. I'm not experiencing this issue. Could you please send me the logs further down?
Tried again and now I get this error:
[0:29:33] Still working on:
[0:29:33] src/third_party
87>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "--no-tags"' in /root/webrtc/Android/src/third_party/jsoncpp/source failed; will retry after a short nap...
87>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "--no-tags"' in /root/webrtc/Android/src/third_party/jsoncpp/source failed; will retry after a short nap...
87>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "clone" "--no-checkout" "--progress" "https://chromium.googlesource.com/external/github.com/open-source-parsers/jsoncpp.git" "/root/webrtc/Android/src/third_party/jsoncpp/_gclient_source_a_ivv2h8"' in /root/webrtc/Android failed; will retry after a short nap...
97>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "--no-tags"' in /root/webrtc/Android/src/third_party/libpfm4/src failed; will retry after a short nap...
99>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "--no-tags"' in /root/webrtc/Android/src/third_party/libunwind/src failed; will retry after a short nap...
97>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "--no-tags"' in /root/webrtc/Android/src/third_party/libpfm4/src failed; will retry after a short nap...
99>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "--no-tags"' in /root/webrtc/Android/src/third_party/libunwind/src failed; will retry after a short nap...
99>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "clone" "--no-checkout" "--progress" "https://chromium.googlesource.com/external/github.com/llvm/llvm-project/libunwind.git" "/root/webrtc/Android/src/third_party/libunwind/_gclient_src_gyu7q6nq"' in /root/webrtc/Android failed; will retry after a short nap...
99>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "fetch" "origin" "--no-tags"' in /root/webrtc/Android/src/third_party/libunwind/src failed; will retry after a short nap...
[0:35:52] Still working on:
[0:35:52] src/third_party/tflite/src
[0:36:02] Still working on:
[0:36:02] src/third_party/tflite/src
[0:36:12] Still working on:
[0:36:12] src/third_party/tflite/src
[0:36:22] Still working on:
[0:36:22] src/third_party/tflite/src
[0:36:32] Still working on:
[0:36:32] src/third_party/tflite/src
[0:36:42] Still working on:
[0:36:42] src/third_party/tflite/src
[0:36:52] Still working on:
[0:36:52] src/third_party/tflite/src
[0:37:02] Still working on:
[0:37:02] src/third_party/tflite/src
[0:37:12] Still working on:
[0:37:12] src/third_party/tflite/src
[0:37:12] Still working on:
[0:37:12] src/third_party/tflite/src
Warnings:
Conflicting directory /root/webrtc/Android/src/third_party/jsoncpp/source moved to /root/webrtc/Android/_bad_scm/src/third_party/jsoncpp/sourceyevgr9t2.
Conflicting directory /root/webrtc/Android/src/third_party/libpfm4/src moved to /root/webrtc/Android/_bad_scm/src/third_party/libpfm4/src77bc6tch.
Conflicting directory /root/webrtc/Android/src/third_party/libunwind/src moved to /root/webrtc/Android/_bad_scm/src/third_party/libunwind/srczuz26oas.
git update is recommended.
Installed git version is 2.43.0;
depot_tools recommends version 2.46.0 or later.
Disable this warning by setting the GCLIENT_SUPPRESS_GIT_VERSION_WARNING
environment variable to 1.
Running: gclient root
Running: gclient config --spec 'solutions = [
{
"name": "src",
"url": "https://webrtc.googlesource.com/src.git",
"deps_file": "DEPS",
"managed": False,
"custom_deps": {},
},
]
target_os = ["android", "unix"]
'
Running: gclient sync --nohooks --with_branch_heads
Running: git config --add remote.origin.fetch '+refs/tags/*:refs/tags/*'
Running: git config diff.ignoreSubmodules dirty
Previous HEAD position was 9af1a88de4 Roll chromium_revision 6612b939a3..1bc86ef67b (1547055:1547199)
HEAD is now at 060ae594b3 Modify Demo for Linux
src/testing (ERROR)
----------------------------------------
[0:00:00] Started.
[0:00:00] Finished running: git rev-list -n 1 HEAD
[0:00:00] Finished running: git rev-parse --abbrev-ref=strict HEAD
----------------------------------------
Error: 6>
6> ____ src/testing at eac4c18f749488d7232af275405e0e1b67d84cb3
6> You have uncommitted changes.
6> cd into src/testing, run git status to see changes,
6> and commit, stash, or reset.
ninja: build stopped: subcommand failed.
That seems to be your computer's problem. Something is wrong.
@bear101, your change is causing a problem.
Originally, I could build without any issues by deleting the directory created by webrtc-src and without clearing the build cache. However, since depot_tools is also gone, I now have to delete the Library/TeamTalkLib/build/webrtc directory itself from the build output to rebuild properly, which is a bit of a hassle.
Do you have any solutions in mind?
By the way, the problem you're experiencing above doesn't seem to be happening to me, unfortunately. How did you build it?