gocryptfs gocryptfs on macOS BigSur, Apple M1

gocryptfs on macOS BigSur, Apple M1

Open corelanc0d3r opened this issue 3 years ago • 33 comments

Hi,

I am trying to run 'gocryptfs' on my 2020 MacBook Air, M1 processor. Latest version of BigSur. I changed the Security Policy to allow kernel extension.

Setup: OSXFuse:

brew install osxfuse

gocryptfs

brew install gocryptfs

I was able to initialize a new encrypted folder with gocryptfs -init /Users/corelanc0d3r/CTest

So far so good.

When trying to mount the encrypted folder, I get this:

gocryptfs /Users/corelanc0d3r/CTest /Users/corelanc0d3r/mnt/test                                                                                     
Password: 
Decrypting master key
fuse.NewServer failed: exit status 255

Any suggestions on getting it to work? (Note: I was able to get it to work on a 2019 MacBook Pro, Intel processor, also running BigSur)

How can I help getting you the info you need to troubleshoot the problem?

thanks !

Mar 25 '21 08:03 corelanc0d3r

I would like to add the following thoughts:

I have installed osxfuse, as well as macFuse 4.1.0 (via https://github.com/osxfuse/osxfuse/releases). (The names are a bit confusing, as they both refer to 'osxfuse'). macFuse seems to support M1. I am not sure about osxfuse though.
It looks like M1/BigSur devices are quite picky about kernel extension... I am a bit of a newbie, so I don't know if there is a way to actually test/check if osxfuse is working or not. Any help would be greatly appreciated.
This may be a dumb idea, but would there be a way for gocryptfs to use macfuse instead of osxfuse? I see a reference to osxfuse in the error message. But perhaps macfuse already works?

fuse.NewServer failed: exit status 255
Maybe you should run: /Library/Filesystems/osxfuse.fs/Contents/Resources/load_osxfuse

thanks !

Mar 25 '21 10:03 corelanc0d3r

Issue being discussed here issue 15

Apr 20 '21 21:04 cocoonkid

I suggest the following:

(1) Uninstall osxfuse. You must have macfuse 4.x for this to work on M1. (2) Try and see if SSHFS works (download from https://osxfuse.github.io/ )

May 03 '21 17:05 rfjakob

Anyone have instructions on getting it to work with Macfuse on the M1?

May 21 '21 01:05 litori

@litori

Install Macfuse
Install go
Download / Clone this repository
Run build.bash which will produce the gocryptfs binary.

Disclaimer: Didn't test this on a M1 Macbook. But it should work.

May 21 '21 09:05 MarcelBochtler

Thanks! got it to work on the m1. Greatly appreciated

May 24 '21 15:05 litori

@litori

1. Install Macfuse

2. Install go

3. Download / Clone this repository

4. Run [build.bash](https://github.com/rfjakob/gocryptfs/blob/master/build.bash) which will produce the gocryptfs binary.

Disclaimer: Didn't test this on a M1 Macbook. But it should work.

M1 MacBook Air with Big Sur gets the following output:

# pkg-config --cflags -- libcrypto Package libcrypto was not found in the pkg-config search path. Perhaps you should add the directory containing libcrypto.pc' to the PKG_CONFIG_PATH environment variable No package 'libcrypto' found pkg-config: exit status 1`

May 24 '21 21:05 Ashaman-

@Ashaman- just try giving pkgconfig the proper paths and building again.

this worked for me:

export PKG_CONFIG_PATH="/opt/homebrew/opt/[email protected]/lib/pkgconfig"
export LDFLAGS="-L/opt/homebrew/opt/[email protected]/lib"
export CPPFLAGS="-I/opt/homebrew/opt/[email protected]/include"

May 25 '21 17:05 rawdigits

You can also just run

./build-without-openssl.bash

and skip the optional dependency on openssl. I think it does not give you a speed advantage on the Apple M1, but maybe someone who built with openssl can post their openssl -speed results?

May 25 '21 17:05 rfjakob

Ah, thanks for the info @rfjakob. ~Does the M1 not have any crypto acceleration offload capability?~

edit/update:

% gocryptfs -speed
gocryptfs v2.0-beta4-5-g09870bf; go-fuse v2.1.1-0.20210423170155-a90e1f463c3f => github.com/rfjakob/go-fuse/v2 v2.1.1-0.20210508151621-62c5aa1919a7; 2021-05-25 go1.16.3 darwin/arm64
AES-GCM-256-OpenSSL 	1627.09 MB/s	(selected in auto mode)
AES-GCM-256-Go      	3746.85 MB/s	
AES-SIV-512-Go      	 452.57 MB/s	
XChaCha20-Poly1305-Go	 747.43 MB/s	(benchmark only, not selectable yet)

May 25 '21 17:05 rawdigits

Wow! Added to https://github.com/rfjakob/gocryptfs/wiki/CPU-Benchmarks#apple-m1-launch-q42020 , thanks!

Looks like

the M1 does have a crypto accelerator
Go natively supports it
gocryptfs makes the wrong decision in auto-selecting between AES-GCM-256-OpenSSL and AES-GCM-256-Go

I suggest to always compile with

./build-without-openssl.bash

on the Apple M1. Using OpenSSL makes no sense here.

May 25 '21 19:05 rfjakob

(3) is fixed now via 1d2ac1e589209be96f8ee5fbba9a18e7e2c5fb04

May 26 '21 07:05 rfjakob

@litori

Install Macfuse

Install go

Download / Clone this repository

Run build.bash which will produce the gocryptfs binary.

Disclaimer: Didn't test this on a M1 Macbook. But it should work.

I followed these steps on an Intel MacBook and now my gocryptfs is entirely broken.

I installed the latest version of macFUSE (4.1.2), cloned the gocryptfs repository, checked out the latest stable tag (v2.0.1), built gocryptfs and mounted an encrypted directory.

Bad mistake.

Now the system.log gets spammed with "go-fuse: using reserved ID 1 for inode number" and then the mounted folder disappears and becomes unaccessible. I have to forcefully unmount it via sudo diskutil umount force /path/.

When I look at the encrypted folder, I see a lot of files with 0 Bytes in size. When I delete these files from the encrypted folder, the error message disappears for some time, until it arises again after mounting and opening a directory.

I also cannot mount the file system as read-only anymore, as this results now in the error message

fs.Mount failed: read-only file system
Maybe you should run: /Library/Filesystems/osxfuse.fs/Contents/Resources/load_osxfuse

Running the suggested line does nothing.

Now my entire setup is broken and gocryptfs does not work anymore.

How can I mitigate this?

Edit: I also tried older versions of macFUSE, right now I have FUSE 3.11.2, but to no avail.

Edit2: I just discovered, that it happens whenever there's a write access to the mounted volume. When I open the volume in the Finder app, it tries to create a file called .DS_Store and fails. The same happens when I use the Terminal and touch /path/test.txt. The result is

gocryptfs[3437]: go-fuse: rx 6: ACCESS n1 {u=501 g=20 x} 
gocryptfs[3437]: go-fuse: tx 6:     OK
gocryptfs[3437]: go-fuse: rx 8: LOOKUP n1 ["test.txt"] 9b
gocryptfs[3437]: go-fuse: using reserved ID 1 for inode number

Jul 14 '21 15:07 ghost

So, I started debugging a bit and got the following: With FUSE 3.11.2 and go1.16.3 darwin/amd64, gocryptfs v1.8.0 works fine, but v2.0.1 does not.

So I started git bisect with v1.8.0 as good and v2.0.1 as bad. My test setup:

Mounting an encrypted directory
Creating a file via touch /directory/test.txt
Watching the syslog for error messages.

The problem was, that a few commits between the tagged ones mentioned above did not compile, and a few others did not let me mount the directory (error message: "fs.Mount failed: operation not permitted") so I cannot say whether it was the same error, but:

Everything up to 3b61244b72f74a25651d4ba184ac7cc62c937db0 works as expected. After that I get either error messages when mounting, or mounting succeeds and a write access triggers "go-fuse: using reserved ID 1 for inode number".

I'm not sure whether this is enough information. Just tell me, if you need more or if I should debug something differently.

Maybe I should open a new issue?

Jul 15 '21 07:07 ghost

Could you test if the loopback example works? Just run "go build" here and you get the loopback binary:

https://github.com/hanwen/go-fuse/tree/master/example/loopback

I wonder if the go-fuse v2 library is broken on macos

Jul 15 '21 07:07 rfjakob

Could you test if the loopback example works? Just run "go build" here and you get the loopback binary:

https://github.com/hanwen/go-fuse/tree/master/example/loopback

I wonder if the go-fuse v2 library is broken on macos

That works just fine.

I also discovered that everything works if I just create and initialize a new directory, and then mount that. At least I wasn't able to trigger any error with a short test.

The "go-fuse: using reserved ID 1 for inode number" message only shows up for directories that already existed in the past.

Up until yesterday I used gocryptfs from homebrew (1.8.0), but it didn't receive updates anymore, so I followed this thread. Also up until yesterday I had Boxcryptor installed, which brought FUSE 3.11 with it. This combination worked without problems and I created the gocryptfs-directories under this configuration.

Then I uninstalled Boxcryptor and FUSE 3.11, installed macFUSE 4, cloned the gocryptfs-repository and then the problems started (apparently only with the legacy directories).

Maybe I need to reinstall macOS. -.- But so far gocryptfs 1.8.0 and FUSE 3.11.2 works just fine.

Jul 15 '21 08:07 ghost

Reinstalling macOS won't be necessary I hope :)

(1) Before the using reserved ID message, you should get a panic message and details about all running goroutines. Could you check if you have that and post it?

(2) Can you show a ls -li of these 0-byte files?

Jul 15 '21 10:07 rfjakob

Reinstalling macOS won't be necessary I hope :)

(1) Before the using reserved ID message, you should get a panic message and details about all running goroutines. Could you check if you have that and post it?

Unfortunately there are no error messages or panics in the system.log. But running gocryptfs -d -fusedebug -fg ... gives proper output:

I have

gocryptfs v2.0.1.HEAD without_openssl; go-fuse v2.1.1-0.20210508151621-62c5aa1919a7; 2021-07-15 go1.16.3 darwin/amd64

I mount the directory and directly execute touch /path/test.txt. Result from the command

touch: plain/test.txt: Socket is not connected

Result from gocryptfs

14:01:40.319481 rx 3: ACCESS n1 {u=501 g=20 x} 
14:01:40.319624 tx 3:     OK
14:01:40.319781 rx 5: LOOKUP n1 ["test.txt"] 9b
14:01:40.319858 tx 5:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 0 0:0 A 0.000000 M 0.000000 C 0.000000}}
14:01:40.319929 rx 4: LOOKUP n1 ["test.txt"] 9b
14:01:40.319970 tx 4:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 0 0:0 A 0.000000 M 0.000000 C 0.000000}}
14:01:40.320047 rx 2: GETATTR n1  
14:01:40.320169 tx 2:     OK, {tA=1s {M040777 SZ=131072 L=1 501:20 256 0:0 A 1615584846.000000 M 1623418700.390000 C 1623418700.390000}}
14:01:40.320281 rx 3: ACCESS n1 {u=501 g=20 w} 
14:01:40.320345 tx 3:     OK
14:01:40.320430 rx 5: CREATE n1 {0100644 [EXCL,RDWR,CREAT]} ["test.txt"] 9b
14:01:40.320603 using reserved ID 1 for inode number
panic: using reserved ID 1 for inode number

goroutine 226 [running]:
log.Panicf(0x12825f3, 0x25, 0xc00005cc70, 0x1, 0x1)
    log/log.go:361 +0xc5
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).newInodeUnlocked(0xc00016c140, 0x12c3358, 0xc000094300, 0xc0000081ff, 0x1, 0x1, 0x123e200, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:95 +0xf4
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).newInode(0xc00016c140, 0x0, 0x0, 0x12c3358, 0xc000094300, 0x81ff, 0x1, 0x1, 0x1000000, 0xc00007c750)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:133 +0x74
github.com/hanwen/go-fuse/v2/fs.(*Inode).newInode(...)
    github.com/hanwen/go-fuse/[email protected]/fs/inode.go:367
github.com/hanwen/go-fuse/v2/fs.(*Inode).NewInode(...)
    github.com/hanwen/go-fuse/[email protected]/fs/inode.go:363
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*Node).newChild(0xc00016e000, 0x0, 0x0, 0xc00007c750, 0xc004218108, 0xc00007c750)
    github.com/rfjakob/gocryptfs/internal/fusefrontend/node_helpers.go:176 +0x11c
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*Node).Create(0xc00016e000, 0x0, 0x0, 0xc00008e438, 0x8, 0x81a400000a02, 0xc004218108, 0x0, 0x1267dc0, 0xc000112150, ...)
    github.com/rfjakob/gocryptfs/internal/fusefrontend/node_open_create.go:106 +0x4e8
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).Create(0xc00016c140, 0xc004216000, 0xc0042181c0, 0xc00008e438, 0x8, 0xc004218108, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:444 +0x12c
github.com/hanwen/go-fuse/v2/fuse.doCreate(0xc000170000, 0xc004218000)
    github.com/hanwen/go-fuse/[email protected]/fuse/opcode.go:159 +0x8f
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000170000, 0xc004218000, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fuse/server.go:483 +0x2b6
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).loop
    github.com/hanwen/go-fuse/[email protected]/fuse/server.go:454 +0x185

(2) Can you show a ls -li of these 0-byte files?

🤦 I just realized, that these are the files that I created with touch, thus of course they are empty. These are also some of macOS's .DS_Store-files, which are placed there by the Finder just in case, but are often just empty. So, these files being empty is expected.

I repeated the above, but instead of touch /path/test.txt, I used

echo "Test" > ./plain/test-with-data.txt

Output from the command:

-bash: ./plain/test-with-data.txt: Socket is not connected

Output from gocryptfs

13:59:36.460897 rx 4: ACCESS n1 {u=501 g=20 x} 
13:59:36.461096 tx 4:     OK
13:59:36.461225 rx 5: LOOKUP n1 ["test-with-data.txt"] 19b
13:59:36.461323 tx 5:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 0 0:0 A 0.000000 M 0.000000 C 0.000000}}
13:59:36.461437 rx 2: GETATTR n1  
13:59:36.461593 tx 2:     OK, {tA=1s {M040777 SZ=131072 L=1 501:20 256 0:0 A 1615584846.000000 M 1623418700.390000 C 1623418700.390000}}
13:59:36.461699 rx 3: ACCESS n1 {u=501 g=20 w} 
13:59:36.461775 tx 3:     OK
13:59:36.461850 rx 4: CREATE n1 {0100644 [RDWR,CREAT,EXCL]} ["test-with-data.txt"] 19b
13:59:36.462031 using reserved ID 1 for inode number
panic: using reserved ID 1 for inode number

goroutine 204 [running]:
log.Panicf(0x12825f3, 0x25, 0xc00005ec70, 0x1, 0x1)
    log/log.go:361 +0xc5
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).newInodeUnlocked(0xc0001b8140, 0x12c3358, 0xc000098180, 0xc0000081ff, 0x1, 0x1, 0x123e200, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:95 +0xf4
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).newInode(0xc0001b8140, 0x0, 0x0, 0x12c3358, 0xc000098180, 0x81ff, 0x1, 0x1, 0x1000000, 0xc000074510)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:133 +0x74
github.com/hanwen/go-fuse/v2/fs.(*Inode).newInode(...)
    github.com/hanwen/go-fuse/[email protected]/fs/inode.go:367
github.com/hanwen/go-fuse/v2/fs.(*Inode).NewInode(...)
    github.com/hanwen/go-fuse/[email protected]/fs/inode.go:363
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*Node).newChild(0xc0001ba000, 0x0, 0x0, 0xc000074510, 0xc000102a08, 0xc000074510)
    github.com/rfjakob/gocryptfs/internal/fusefrontend/node_helpers.go:176 +0x11c
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*Node).Create(0xc0001ba000, 0x0, 0x0, 0xc00008e150, 0x12, 0x81a400000a02, 0xc000102a08, 0x0, 0x1267dc0, 0xc0001a0000, ...)
    github.com/rfjakob/gocryptfs/internal/fusefrontend/node_open_create.go:106 +0x4e8
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).Create(0xc0001b8140, 0xc0000562a0, 0xc000102ac0, 0xc00008e150, 0x12, 0xc000102a08, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:444 +0x12c
github.com/hanwen/go-fuse/v2/fuse.doCreate(0xc0001bc000, 0xc000102900)
    github.com/hanwen/go-fuse/[email protected]/fuse/opcode.go:159 +0x8f
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc0001bc000, 0xc000102900, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fuse/server.go:483 +0x2b6
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).loop
    github.com/hanwen/go-fuse/[email protected]/fuse/server.go:454 +0x185

Afterwards, I have two new, encrypted but empty, files in the outer volume:

$> ls -li BGn1TSv6XWBjolVwnqjmLA smglExWHwVmyBDiB8ozcFZOIVFlfpykpLj4hpeYj7Xk
1 -rwxrwxrwx  1 user  group  0 15 Jul 14:01 BGn1TSv6XWBjolVwnqjmLA
1 -rwxrwxrwx  1 user  group  0 15 Jul 13:59 smglExWHwVmyBDiB8ozcFZOIVFlfpykpLj4hpeYj7Xk

Both are empty, the one with the shorter name is expected to be, though.

Jul 15 '21 12:07 ghost

Ok, thanks, looks like I broke file creation on macos.

Jul 15 '21 12:07 rfjakob

Well, people who still use Homebrew are still at version 1.8.0 and FUSE 3.11, a combination that seems to work fine.

I wonder how these M1 people get it done with the latest version and macFUSE 4? 🤔

Can this be caused by this old version of FUSE? I tried macFUSE 4 yesterday for some time, but couldn't even compile gocryptfs v1.8.0, as it was complaining about something FUSE-related.

Jul 15 '21 12:07 ghost

My guess at the moment is that gocryptfs v2.x is completely broken on macos (with all macfuse versions).

However, macfuse should have no influence at all on compiling, what error do you get?

Jul 15 '21 13:07 rfjakob

My guess at the moment is that gocryptfs v2.x is completely broken on macos (with all macfuse versions).

That is not correct. I'm using it installed via macports on my Intel MBP from 2019 using macfuse version 4.1.2. I have no idea though if it works for M1 models.

Edit: Before adding it to macports, I was compiling it manually and this was also working (gocryptfs 2.x).

Jul 15 '21 13:07 MarcelBochtler

However, macfuse should have no influence at all on compiling, what error do you get?

I don't remember the specific message; Maybe that was something else and I stopped too soon.

I'll test macFUSE 4 again, but for that I must first uninstall FUSE 3.11 and I need the drive right now. I'll report back tomorrow.

Jul 15 '21 13:07 ghost

Edit: Before adding it to macports, I was compiling it manually and this was also working (gocryptfs 2.x).

Maybe I should just reinstall my OS. :D Maybe the combination of Homebrew, Boxcryptor, and self-compiled / -installed stuff produced some relicts and now the wrong libraries get linked at the wrong place or something like this.

Jul 15 '21 13:07 ghost

I tested with macFUSE 4.1.2.

With gocryptfs v1.8.0.HEAD without_openssl; go-fuse v1.0.1-0.20190319092520-161a16484456; 2021-07-17 go1.16.3 darwin/amd64, trying to mount results in

Password: 
Decrypting master key
CryptoCore.Wipe: Only nil'ing stdlib refs
frontendArgs: {
    "Cipherdir": "/Volumes/T7-0-data1/backup-data",
    "PlaintextNames": false,
    "LongNames": true,
    "PreserveOwner": false,
    "ForceOwner": null,
    "ConfigCustom": false,
    "NoPrealloc": false,
    "SerializeReads": false,
    "ForceDecode": false,
    "Exclude": null,
    "ExcludeWildcard": null,
    "ExcludeFrom": null
}
fuse.NewServer failed: fork/exec /Library/Filesystems/osxfusefs.fs/Support/load_osxfusefs: no such file or directory
Maybe you should run: /Library/Filesystems/osxfuse.fs/Contents/Resources/load_osxfuse

With macFUSE 4 the file was move to /Library/Filesystems/macfuse.fs/Contents/Resources/load_macfuse.

With gocryptfs v2.0.1.HEAD without_openssl; go-fuse v2.1.1-0.20210508151621-62c5aa1919a7; 2021-07-17 go1.16.3 darwin/amd64, mounting the legacy directory and executing touch /path/test.txt results in

17:36:36.315318 rx 4: ACCESS n1 {u=501 g=20 x} 
17:36:36.315440 tx 4:     OK
17:36:36.315519 rx 3: LOOKUP n1 ["test.txt"] 9b
17:36:36.315584 tx 3:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 0 0:0 A 0.000000 M 0.000000 C 0.000000}}
17:36:36.315643 rx 2: LOOKUP n1 ["test.txt"] 9b
17:36:36.315683 tx 2:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 0 0:0 A 0.000000 M 0.000000 C 0.000000}}
17:36:36.315791 rx 5: ACCESS n1 {u=501 g=20 w} 
17:36:36.315857 tx 5:     OK
17:36:36.315904 rx 4: GETATTR n1  
17:36:36.315993 tx 4:     OK, {tA=1s {M040777 SZ=131072 L=1 501:20 256 0:0 A 1615584846.000000 M 1623418700.390000 C 1623418700.390000}}
17:36:36.316060 rx 3: CREATE n1 {0100644 [EXCL,RDWR,CREAT]} ["test.txt"] 9b
17:36:36.316207 using reserved ID 1 for inode number
panic: using reserved ID 1 for inode number

goroutine 25 [running]:
log.Panicf(0x12825f3, 0x25, 0xc004247c70, 0x1, 0x1)
    log/log.go:361 +0xc5
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).newInodeUnlocked(0xc00007c140, 0x12c3358, 0xc0000ac280, 0xc0000081ff, 0x1, 0x1, 0x123e200, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:95 +0xf4
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).newInode(0xc00007c140, 0x0, 0x0, 0x12c3358, 0xc0000ac280, 0x81ff, 0x1, 0x1, 0x1000000, 0xc000073050)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:133 +0x74
github.com/hanwen/go-fuse/v2/fs.(*Inode).newInode(...)
    github.com/hanwen/go-fuse/[email protected]/fs/inode.go:367
github.com/hanwen/go-fuse/v2/fs.(*Inode).NewInode(...)
    github.com/hanwen/go-fuse/[email protected]/fs/inode.go:363
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*Node).newChild(0xc00007e000, 0x0, 0x0, 0xc000073050, 0xc00420a588, 0xc000073050)
    github.com/rfjakob/gocryptfs/internal/fusefrontend/node_helpers.go:176 +0x11c
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*Node).Create(0xc00007e000, 0x0, 0x0, 0xc0001279b0, 0x8, 0x81a400000a02, 0xc00420a588, 0x0, 0x1267dc0, 0xc0000b0000, ...)
    github.com/rfjakob/gocryptfs/internal/fusefrontend/node_open_create.go:106 +0x4e8
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).Create(0xc00007c140, 0xc000128180, 0xc00420a640, 0xc0001279b0, 0x8, 0xc00420a588, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:444 +0x12c
github.com/hanwen/go-fuse/v2/fuse.doCreate(0xc000048000, 0xc00420a480)
    github.com/hanwen/go-fuse/[email protected]/fuse/opcode.go:159 +0x8f
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000048000, 0xc00420a480, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fuse/server.go:483 +0x2b6
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).loop
    github.com/hanwen/go-fuse/[email protected]/fuse/server.go:454 +0x185

So I'll just stick with gocryptfs 1.8.0 and FUSE 3.11 for a while.

Interestingly this only happens with my legacy directory, but not with newly created ones, no matter whether I create them with gocryptfs v1.8.0 or v2.0.1.

Maybe one of the files in that directory has this bad inode id and the old versions don't care about this? Can I check this somehow?

Jul 17 '21 15:07 ghost

this only happens with my legacy directory, but not with newly created ones, no matter whether I create them with gocryptfs v1.8.0 or v2.0.1.

This is very strange. Do old and new have their encrypted directories on the same filesystem?

Jul 19 '21 18:07 rfjakob

this only happens with my legacy directory, but not with newly created ones, no matter whether I create them with gocryptfs v1.8.0 or v2.0.1.

This is very strange. Do old and new have their encrypted directories on the same filesystem?

Ah, now we're getting closer. Everything works as expected when I create a new directory on my main disk (APFS).

The legacy directory that has this problem lives on an external SSD, which is formatted with exFAT. I initialized a new directory on that filesystem with gocryptfs v2.0.1 and tried to write "Test" into a new file. Same error as before:

18:05:11.310710 rx 2: ACCESS n1 {u=501 g=20 x} 
18:05:11.310885 tx 2:     OK
18:05:11.311014 rx 3: LOOKUP n1 ["test.txt"] 9b
18:05:11.311171 tx 3:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 0 0:0 A 0.000000 M 0.000000 C 0.000000}}
18:05:11.311291 rx 5: GETATTR n1  
18:05:11.311437 tx 5:     OK, {tA=1s {M040777 SZ=131072 L=1 501:20 256 0:0 A 1626883456.000000 M 1626883459.340000 C 1626883459.340000}}
18:05:11.311585 rx 4: ACCESS n1 {u=501 g=20 w} 
18:05:11.311668 tx 4:     OK
18:05:11.311804 rx 2: CREATE n1 {0100644 [RDWR,CREAT,EXCL]} ["test.txt"] 9b
18:05:11.312007 using reserved ID 1 for inode number
panic: using reserved ID 1 for inode number

goroutine 197 [running]:
log.Panicf(0x12825f3, 0x25, 0xc000109c70, 0x1, 0x1)
    log/log.go:361 +0xc5
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).newInodeUnlocked(0xc0001be140, 0x12c3358, 0xc0000f8100, 0xc0000081ff, 0x1, 0x1, 0x123e200, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:95 +0xf4
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).newInode(0xc0001be140, 0x0, 0x0, 0x12c3358, 0xc0000f8100, 0x81ff, 0x1, 0x1, 0x1000000, 0xc0000ac7e0)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:133 +0x74
github.com/hanwen/go-fuse/v2/fs.(*Inode).newInode(...)
    github.com/hanwen/go-fuse/[email protected]/fs/inode.go:367
github.com/hanwen/go-fuse/v2/fs.(*Inode).NewInode(...)
    github.com/hanwen/go-fuse/[email protected]/fs/inode.go:363
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*Node).newChild(0xc0001c0000, 0x0, 0x0, 0xc0000ac7e0, 0xc004214588, 0xc0000ac7e0)
    github.com/rfjakob/gocryptfs/internal/fusefrontend/node_helpers.go:176 +0x11c
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*Node).Create(0xc0001c0000, 0x0, 0x0, 0xc000016b38, 0x8, 0x81a400000a02, 0xc004214588, 0x0, 0x1267dc0, 0xc0000d6000, ...)
    github.com/rfjakob/gocryptfs/internal/fusefrontend/node_open_create.go:106 +0x4e8
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).Create(0xc0001be140, 0xc000100060, 0xc004214640, 0xc000016b38, 0x8, 0xc004214588, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fs/bridge.go:444 +0x12c
github.com/hanwen/go-fuse/v2/fuse.doCreate(0xc0001c2000, 0xc004214480)
    github.com/hanwen/go-fuse/[email protected]/fuse/opcode.go:159 +0x8f
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc0001c2000, 0xc004214480, 0x0)
    github.com/hanwen/go-fuse/[email protected]/fuse/server.go:483 +0x2b6
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).loop
    github.com/hanwen/go-fuse/[email protected]/fuse/server.go:454 +0x185

On the file system itself, I now have a file with inode 1:

 $> ls -ahli ./test-crypt-dir/
 total 1024
       1 -rwxrwxrwx  1 user  group     0B 21 Jul 18:05 -ttIedajkhmIYrDdphR5iQ
 5163393 drwxrwxrwx  1 user  group   128K 21 Jul 18:04 .
      18 drwxrwxrwx  1 user  group   128K 31 Dez  1979 ..
 5163394 -rwxrwxrwx  1 user  group   450B 21 Jul 18:04 gocryptfs.conf
 5163395 -rwxrwxrwx  1 user  group    16B 21 Jul 18:04 gocryptfs.diriv

According to macOS's 'Disk Utility', the exFAT file system is healthy.

I have some Linux machine at work. I'll check again there and will report again.

Jul 21 '21 16:07 ghost

Wow, the macos exfat driver actually reports inode number 1. But why doesn't gocryptfs.conf or gocryptfs.diriv get 1? Or do they, for a second, and then it changes to something higher?

Jul 21 '21 16:07 rfjakob

Wow, the macos exfat driver actually reports inode number 1. But why doesn't gocryptfs.conf or gocryptfs.diriv get 1?

I have no idea. The gocryptfs files are created via gocryptfs -init ..., so not via the fuse layer.

Or do they, for a second, and then it changes to something higher?

Can I somehow test for that?

I haven't had access to a Linux machine yet, so I couldn't test for it. I'll set up some virtualbox later that day and check how an up to date Linux handles this.

Jul 27 '21 07:07 ghost

I would just try and see what

rm foo
touch foo
ls -li foo

says on exfat. What linux does on exfat is also interesting, but i can test myself easily.

In any case, macos does use inode number 1 and there is nothing inherently wrong with that, so i will add a workaround in gocryptfs (probably next week)

Jul 27 '21 07:07 rfjakob

PS: could you open a new ticket just for the inode 1 issue so I can reference it in the commit later?

Jul 27 '21 07:07 rfjakob

I bumped into this problem after updating to OSX 11.5.1 Updating gocryptfs / reinstalling macfuse did not help. This comment helped me (building from source): https://github.com/rfjakob/gocryptfs/issues/556#issuecomment-845810085

Aug 03 '21 12:08 gp42

Ran on MacBook Pro (14-inch, 2021) M1 Pro with 16GB

$ gocryptfs -speed

gocryptfs v2.2.1; go-fuse [vendored]; 2021-10-20 go1.18 darwin/arm64
cpu: unknown; with AES acceleration
AES-GCM-256-OpenSSL             2893.19 MB/s
AES-GCM-256-Go                  5112.39 MB/s    (selected in auto mode)
AES-SIV-512-Go                   520.46 MB/s
XChaCha20-Poly1305-OpenSSL      1424.34 MB/s    (selected in auto mode)
XChaCha20-Poly1305-Go            824.12 MB/s

Apr 04 '22 17:04 karbassi

gocryptfs gocryptfs copied to clipboard

gocryptfs on macOS BigSur, Apple M1

gocryptfs
gocryptfs copied to clipboard