Running Eden filesystem -- nearly got it working?

[MatthewSteel]

Hi folks, I know the Eden Fuse filesystem isn't supported but I'm trying to get it running anyway. I think I got pretty close, and wonder if you could help me figure out how to get the rest of the way there. (If we can get the rest of the way there, I wouldn't mind contributing some changes to make it a bit easier.)

Things I did:

1. Installed a few things on my machine (pkg-config, cargo),
2. Checked out edencommon and the sapling repo and ran cp -R edencommon/eden/common sapling/eden. Feels like a bad idea, probably I was meant to do this in some automated way?
3. (In some order, can't remember)
   • Ran python3 ./build/fbcode_builder/getdeps.py --allow-system-packages build eden,
   • ran python3 ./build/fbcode_builder/getdeps.py build fbthrift,
   • ran cd eden/scm && make oss,
   • added "filelock" to these deps,
4. Using bd=/tmp/long_path_to_build_dir ran ./make-client.py $bd/installed/fb303 --thrift-compiler=$bd/installed/fbthrift/bin/thrift1 --thrift-py=$bd/repos/github.com-facebook-fbthrift.git/thrift/lib/py which output eden.zip,
5. Tried ./eden.zip start but got "can't open shared library" errors, and couldn't fix with LD_LIBRARY_PATH so I ran $bd/installed/eden/bin/edenfs directly. It "exited" immediately but pgrep eden shows a couple of running processes, nice. Also some promising stuff in /home/matt.steel/.eden/logs/edenfs.log, dunno when that appeared.
6. Ran mkdir /home/matt.steel/eden2 && ./eden.zip clone . /home/matt.steel/eden2 -r main but got an error

edenfs daemon is not currently running. Starting...
error: unable to find edenfs executable

Running pgrep -a eden again I get output

cr sapling$ pgrep -a eden
373836 edenfs_privhelper --privhelper_uid=1001 --privhelper_gid=1002 --privhelper_fd=5
373843 ./installed/eden/bin/edenfs --foreground --logPath /home/matt.steel/.eden/logs/edenfs.log --privhelper_fd 6 --startupLoggerFd 8

so I'm a bit stumped. Feels very close, though. Any guidance you can offer would be very much appreciated.

[MichaelCuevas]

Hi Matthew! I work on the EdenFS team at Meta. We think it's really awesome that you're trying to get open source Eden running (you may be the first to do so). As a disclaimer, we haven't even tried to do this ourselves, so you're venturing into unknown territory.

The error that you're running into is likely caused by assumptions we make about Eden's installation location. We have a few components that we build/install. The first is the daemon, which you have installed as at ./installed/eden/bin/edenfs. The second is the privhelper, which you can see in the pgrep output as edenfs_privhelper. This is used by the daemon to perform privileged operations such as mount and unmount. Finally, we have the CLI which you're running as ./eden.zip. We refer to this as edenfsctl.

During startup, we assume that these are all installed near each other. In your case, they're not, so eden.zip clone (which calls edenfsctl start) cannot find the daemon in the expected location. To remedy this, you can pass in the --daemon-binary argument to edenfsctl commands to inform the CLI where the daemon is located.

This should get you a little further along in the startup process, but we expect that you will be blocked by other issues pretty soon after. EdenFS currently hardcodes certain server assumptions, and building/running the server component isn't a supported open source workflow. There are several tech-debt blockers that currently make this impossible. My teammate will add some additional information about these blockers in a separate reply.

By the way, could you please paste the contents of your edenfs.log here? I'm interested in seeing what's written there :)

Edit: Turns out there's at least 1 other person trying to get EdenFS OSS running. Alex is a former coworker of ours and seems to be quite close to getting things in a working state.

[quark-zju]

Regarding on the Sapling/EdenFS integration: Getting the Thrift part working is the first step. The next major blocker is that the EdenFS -> Sapling integration does not use abstractions to support Git repos. If we want to lazily fetch objects from the Git server, there are extra things to implement.

We do plan to make it work at some point. But it's not a trivial change so it'll take a while.

[MatthewSteel]

We refer to this as edenfsctl.

Oh hmm, I might have had one of those built near the daemon but didn't think to try running it. I'll take another look when I get a moment. Maybe a "nearby" client built at the same time would be better able to phone home.

EdenFS currently hardcodes certain server assumptions, and building/running the server component isn't a supported open source workflow

Is there a workflow where EdenFS just provides a "view" into a local .git database? Runs git cat-file when it needs to read a file, inserts blobs/trees/commits into the git database when committing work? "Cheaper checkout" is as much or more of a draw for me than "lazy fetch". (Doesn't sound like it'd be super useful if you've got your own server, but for some reason I got the impression that sort of thing worked while reading some of the code.)

By the way, could you please paste the contents of your edenfs.log here? I'm interested in seeing what's written there :)

Looks like the server set up its datastore, seems content but lonely:

V0722 20:07:54.706594 373843 StartupLogger.cpp:110] Starting edenfs 20230722-021436, pid 373843, session_id 4278158960
V0722 20:07:54.711975 373843 EdenServer.cpp:1084] Creating local RocksDB store...
V0722 20:07:54.712102 373843 RocksDbLocalStore.cpp:282] Making a new RockDB localstore ( 0x5567f5b6e940 ). debug information for T136469251.
V0722 20:07:54.712107 373843 RocksDbLocalStore.cpp:298] Making a new RockDB localstore ( 0x5567f5b6e890 ) . debug information for T136469251.
V0722 20:07:54.712123 373843 EdenServer.cpp:1093] Created RocksDB store in 0 seconds.
V0722 20:07:54.768514 373843 StartupLogger.cpp:110] Opening local store...
V0722 20:07:54.768567 373843 RocksDbLocalStore.cpp:303] Opening Rocksdb localstore ( 0x5567f5b6e890 ) . debug information for T136469251.
V0722 20:07:54.892559 373843 RocksDbLocalStore.cpp:323] RocksDB opened, computing statistics ...
V0722 20:07:54.893096 373843 RocksDbLocalStore.cpp:325] RocksDB opened, clearing out old data ...
V0722 20:07:54.893168 373843 RocksDbLocalStore.cpp:391] clearing column family "blobsize"
V0722 20:07:54.893614 373843 RocksDbLocalStore.cpp:429] compacting column family "blobsize"
V0722 20:07:54.907442 373843 RocksDbLocalStore.cpp:391] clearing column family "scsproxyhash"
V0722 20:07:54.907664 373843 RocksDbLocalStore.cpp:429] compacting column family "scsproxyhash"
V0722 20:07:54.919810 373843 RocksDbLocalStore.cpp:391] clearing column family "treemeta"
V0722 20:07:54.920014 373843 RocksDbLocalStore.cpp:429] compacting column family "treemeta"
V0722 20:07:54.932358 373843 RocksDbLocalStore.cpp:391] clearing column family "recasdigestproxyhash"
V0722 20:07:54.932542 373843 RocksDbLocalStore.cpp:429] compacting column family "recasdigestproxyhash"
V0722 20:07:54.945131 373843 RocksDbLocalStore.cpp:327] RocksDB setup complete. ( 0x5567f5b6e890 ) . debug information for T136469251.
V0722 20:07:54.945178 373843 StartupLogger.cpp:110] Opened local store in 0.176 seconds.
V0722 20:07:54.945284 373843 StartupLogger.cpp:110] No mount points currently configured.
I0722 20:07:54.945385 373843 StartupLogger.cpp:110] Started EdenFS (pid 373843, session_id 4278158960) in 0s
I0722 20:07:54.945407 373843 StartupLogger.cpp:110] Logs available at /home/matt.steel/.eden/logs/edenfs.log

Anyway, thanks for the info, all the best.

[ahornby]

Edit: Turns out there's at least 1 other person trying to get EdenFS OSS running. Alex is a former coworker of ours and seems to be quite close to getting things in a working state.

@MichaelCuevas thanks for the shout out :) stack to https://github.com/facebook/sapling/pull/695 gets the eden and mononoke OSS build CI working, planning to look at enabling edenfs OSS tests when I get a chance

[shreyas2077]

I am trying to build eden from edencommon source code but hitting issues with dependencies. there seems to be lot of cmake files missing for them. Please advise on how to proceed further @MatthewSteel @MichaelCuevas

C02G8B4EMD6R:edencommon$ python3 ./build/fbcode_builder/getdeps.py --allow-system-packages build eden
Building on {distro=None, distro_vers=None, fb=off, fbsource=off, os=darwin, shared_libs=off, test=on}
Assessing lmdb...
Assessing osxfuse...
Assessing blake3...
Building blake3...
Traceback (most recent call last):
  File "/home/edencommon/./build/fbcode_builder/getdeps.py", line 1358, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/edencommon/./build/fbcode_builder/getdeps.py", line 1341, in main
    return args.func(args)
           ^^^^^^^^^^^^^^^
  File "/home/edencommon/./build/fbcode_builder/getdeps.py", line 108, in run
    self.run_project_cmd(args, loader, manifest)
  File "/home/edencommon/./build/fbcode_builder/getdeps.py", line 640, in run_project_cmd
    builder.build(install_dirs, reconfigure=reconfigure)
  File "/home/edencommon/build/fbcode_builder/getdeps/builder.py", line 137, in build
    self._build(install_dirs=install_dirs, reconfigure=reconfigure)
  File "/home/edencommon/build/fbcode_builder/getdeps/builder.py", line 697, in _build
    raise Exception("Failed to find CMake")
Exception: Failed to find CMake

[shreyas2077]

I get the above error for almost all dependencies. Has something changed? I am trying to see if changing builder = make to nop in dependency manifest file would help in the setup but not sure if it's the right thing to do. Appreciate any help

[quark-zju]

File "/home/edencommon/build/fbcode_builder/getdeps/builder.py", line 697, in _build raise Exception("Failed to find CMake") Exception: Failed to find CMake

If you read related code, it happens when "cmake" is not found in "PATH". It seems you need to install cmake (which is a common third-party software) first.

changing builder = make to nop

This is not the right thing to do.

Our GitHub workflow includes rules to build edenfs. The build is indeed recently broken (unrelated to cmake). The build was green one week ago.

While you might get edenfs built and running, note it is still not yet a supported use-case at present.

[shreyas2077]

@quark-zju Thanks for the response. I had to re-install cmake(something went wrong with it). However I hit another error while building eden

Output

ld: warning: ignoring duplicate libraries: 'eden/fs/nfs/libeden_nfs_nfsd_rpc.a'
[64/451] Building Rust crate 'backingstore'...
FAILED: eden/scm/lib/backingstore/CMakeFiles/rust_backingstore.cargo eden/scm/lib/backingstore/debug/libbackingstore.a eden/scm/lib/backingstore/release/libbackingstore.a /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/eden/scm/lib/backingstore/CMakeFiles/rust_backingstore.cargo /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/eden/scm/lib/backingstore/debug/libbackingstore.a /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/eden/scm/lib/backingstore/release/libbackingstore.a
cd /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/eden/scm/lib/backingstore && /usr/local/Cellar/cmake/3.27.9/bin/cmake -E remove -f /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/eden/scm/lib/backingstore/Cargo.lock && /usr/local/Cellar/cmake/3.27.9/bin/cmake -E env CARGO_TARGET_DIR=/private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/eden/scm/lib/backingstore /Users/bytedance/.cargo/bin/cargo build --release -p backingstore
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package:   /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/eden/scm/exec/hgmain/Cargo.toml
workspace: /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/eden/scm/Cargo.toml
warning: virtual workspace defaulting to `resolver = "1"` despite one or more workspace members being on edition 2021 which implies `resolver = "2"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
note: for more details see https://doc.rust-lang.org/cargo/reference/resolver.html#resolver-versions
warning: /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/configerator/structs/scm/hg/hgclientconf/Cargo.toml: version requirement `0.0.1+unstable` for dependency `fbthrift` includes semver metadata which will be ignored, removing the metadata is recommended to avoid confusion
warning: /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/eden/scm/lib/thrift-types/Cargo.toml: version requirement `0.0.1+unstable` for dependency `fbthrift` includes semver metadata which will be ignored, removing the metadata is recommended to avoid confusion
warning: /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/eden/scm/lib/zstdelta/Cargo.toml: version requirement `2.0.9+zstd.1.5.5` for dependency `zstd-sys` includes semver metadata which will be ignored, removing the metadata is recommended to avoid confusion
warning: /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/eden/fs/service/Cargo.toml: version requirement `0.0.1+unstable` for dependency `fbthrift` includes semver metadata which will be ignored, removing the metadata is recommended to avoid confusion
warning: /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/eden/fs/config/Cargo.toml: version requirement `0.0.1+unstable` for dependency `fbthrift` includes semver metadata which will be ignored, removing the metadata is recommended to avoid confusion
    Blocking waiting for file lock on package cache
Subprocess killed
[65/451] Installing cxxbridge (version 1.0.110)
FAILED: cxxbridge_v1.0.110/bin/cxxbridge /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/cxxbridge_v1.0.110/bin/cxxbridge
cd /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/eden/scm/lib/edenfs_ffi && /usr/local/Cellar/cmake/3.27.9/bin/cmake -E make_directory /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/cxxbridge_v1.0.110 && /usr/local/Cellar/cmake/3.27.9/bin/cmake -E remove -f /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/eden/scm/lib/edenfs_ffi/Cargo.lock && /usr/local/Cellar/cmake/3.27.9/bin/cmake -E env /Users/bytedance/.cargo/bin/cargo install cxxbridge-cmd --version 1.0.110 --root /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/cxxbridge_v1.0.110 --quiet && /usr/local/Cellar/cmake/3.27.9/bin/cmake -E remove -f /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/eden/scm/lib/edenfs_ffi/Cargo.lock
Subprocess killed
[66/451] Running utility command for integration_tests.GEN_PY_EXE
FAILED: eden/integration/CMakeFiles/integration_tests.GEN_PY_EXE.util eden/integration/integration_tests_tests.cmake /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/eden/integration/integration_tests_tests.cmake
cd /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/eden/integration && /usr/local/Cellar/cmake/3.27.9/bin/cmake -D TEST_TARGET=integration_tests -D TEST_INTERPRETER=/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11 -D TEST_ENV=CMAKE_SOURCE_DIR=/private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git -D TEST_EXECUTABLE=/private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/eden/integration/integration_tests -D TEST_WORKING_DIR=/private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden -D TEST_LIST=integration_tests_TESTS -D TEST_PREFIX=integration_tests:: -D TEST_PROPERTIES= -D CTEST_FILE=/private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/eden/integration/integration_tests_tests.cmake -P /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/build/fbcode_builder/CMake/FBPythonTestAddTests.cmake
/bin/sh: line 1: 93520 Killed: 9               /usr/local/Cellar/cmake/3.27.9/bin/cmake -D TEST_TARGET=integration_tests -D TEST_INTERPRETER=/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11 -D TEST_ENV=CMAKE_SOURCE_DIR=/private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git -D TEST_EXECUTABLE=/private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/eden/integration/integration_tests -D TEST_WORKING_DIR=/private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden -D TEST_LIST=integration_tests_TESTS -D TEST_PREFIX=integration_tests:: -D TEST_PROPERTIES= -D CTEST_FILE=/private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden/eden/integration/integration_tests_tests.cmake -P /private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/repos/github.com-facebook-sapling.git/build/fbcode_builder/CMake/FBPythonTestAddTests.cmake
Subprocess killed
CMake Error: Generator: execution of make failed. Make command was: /usr/local/opt/ninja/bin/ninja -j 12 install
Command '['/usr/local/opt/cmake/bin/cmake', '--build', '/private/var/folders/30/cmv9c_5j3mq_kthx63sb1t5c0000gn/T/fbcode_builder_getdeps-ZhomeedencommonZbuildZfbcode_builder/build/eden', '--target', 'install', '--config', 'Release', '-j', '12']' returned non-zero exit status 1.
!! Failed
Killed: 9