httpdirfs
httpdirfs copied to clipboard
Published
20 hours ago •
fangfufu
fangfufu
Single File Mode does not seem to work Linux Mint 20.2 Cinnamon
httpdirfs version: commit 40c750f URL: https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z OS: Linux Mint 20.2 Cinnamon
user@pc:~/mnt/httpdirfs$ curl -I 'https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z'
HTTP/2 200
date: Sat, 06 Nov 2021 12:11:21 GMT
content-type: application/x-7z-compressed
content-length: 193730549
last-modified: Wed, 24 Jun 2020 16:28:52 GMT
etag: "1f62161-b8c17f5-5a8d6f8ac5e17"
cache-control: max-age=14400
cf-cache-status: HIT
age: 608
accept-ranges: bytes
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=hKCBfG9ZRIkWc9RPaxQuNbDbQ3UfklgFJ5rqZnOpEY%2Byoh5WArC2N96zTXqqz5efl2oapJm%2Fgj%2FeKsoJWZBk%2B8P6GviqjmJQtSZW4tF%2BmAIRYOzubcM2RzrO0IPt36NPbgHN"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 6a9e3950e86e71a8-LHR
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400
user@pc:~/mnt/httpdirfs$ curl --header "Range: bytes=100-107" -I 'https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z'
HTTP/2 206
date: Sat, 06 Nov 2021 12:11:24 GMT
content-type: application/x-7z-compressed
content-length: 8
last-modified: Wed, 24 Jun 2020 16:28:52 GMT
etag: "1f62161-b8c17f5-5a8d6f8ac5e17"
cache-control: max-age=14400
cf-cache-status: HIT
age: 611
content-range: bytes 100-107/193730549
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=FuX34oZmtfnpC2WEDMIMOrADiDe8qVYaLCscSv9Iektqsw1rTNnnM9ir3C58IEl13drr6UtzEb9%2F90WE0RZR9HGV2eEFs%2FlwIia8fwOmb7OHBq8FAGDK5tB%2FUlFxwAmHWaOF"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 6a9e39688ae072ae-LHR
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400
user@pc:~/mnt/httpdirfs$ httpdirfs --single-file-mode 'https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z' ~/mnt/httpdirfs
print_version: HTTPDirFS version 1.2.3
print_version: libcurl SSL engine: OpenSSL/1.1.1f
LinkTable_print: --------------------------------------------
LinkTable_print: LinkTable 0x558843718db0 for https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: --------------------------------------------
LinkTable_print: 0 H 0 https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: 1 F 193730549 Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: --------------------------------------------
LinkTable_print: Invalid link count: 0
LinkTable_print: --------------------------------------------
user@pc:~/mnt/httpdirfs$ ls
ls: cannot open directory '.': Transport endpoint is not connected
user@pc:~/mnt/httpdirfs$
Have you unmounted the directory you are trying to mount? Please make clean, and recompile. Please ensure you use your locally compiled version, not the one that you installed in your system, because I noticed you invoked httpdirfs rather than ./httpdirfs
$ ./httpdirfs --single-file-mode 'https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z' mnt
print_version: HTTPDirFS version 1.2.3
print_version: libcurl SSL engine: OpenSSL/1.1.1k
Debug(10):src/link.c:193:LinkTable_uninitialised_fill: ... ... Done!
LinkTable_print: --------------------------------------------
LinkTable_print: LinkTable 0x555ada801230 for https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: --------------------------------------------
LinkTable_print: 0 H 0 https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: 1 F 193730549 Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: --------------------------------------------
LinkTable_print: Invalid link count: 0
LinkTable_print: --------------------------------------------
$ ls mnt
Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
If you crashed httpdirfs, you need to manually unmount the directory using fusermount -u $DIR
@fangfufu
Okay I've made sure that the installed one is the latest commit now (uninstalled via apt, installed latest commit via sudo make install and compared the sha256s of each copy of the binary). I've been unmounting using sudo fusermount $DIR, but just to make sure, I restarted the PC and mounted to a new directory, but I got the same error.
You were in ~/mnt/httpdirfs you try mounting it. Have you tried mounting while you are in ~/mnt?
Either way I couldn't reproduce it.
Linux Mint 20.2 Cinnamon
user@pc:~/mnt$ httpdirfs --single-file-mode 'https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z' httpdirfs
print_version: HTTPDirFS version 1.2.3
print_version: libcurl SSL engine: OpenSSL/1.1.1f
LinkTable_print: --------------------------------------------
LinkTable_print: LinkTable 0x5610ccb6b4b0 for https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: --------------------------------------------
LinkTable_print: 0 H 0 https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: 1 F 193730549 Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: --------------------------------------------
LinkTable_print: Invalid link count: 0
LinkTable_print: --------------------------------------------
user@pc:~/mnt$ ls httpdirfs/
Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
user@pc:~/mnt$ cat httpdirfs/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
cat: httpdirfs/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z: Transport endpoint is not connected
cat: httpdirfs/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z: Transport endpoint is not connected
Yeah I'm pretty puzzled too.
Yeah, e.g. httpdirfs 'https://cdimage.debian.org/debian-cd/' httpdirfs works fine. Can browse in the file browser, view text files, explore/mount tar.gz/ISOs etc.
Re-cloned it again, no luck. Note that I've been using make CC=gcc-10 instead of make to avoid this error:
rm -f ./.depend
cc -g -O2 -Wall -Wextra -Wshadow -fsanitize=undefined -fanalyzer -Wno-analyzer-file-leak -rdynamic -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\"1.2.3\" `pkg-config --cflags-only-I gumbo libcurl fuse uuid expat` -MM src/log.c src/fuse_local.c src/config.c src/main.c src/memcache.c src/util.c src/cache.c src/network.c src/sonic.c src/link.c -MF ./.depend;
cc: error: unrecognized command line option ‘-fanalyzer’
Makefile:95: .depend: No such file or directory
make: *** [Makefile:94: .depend] Error 1
Why don't you get rid off the -fanalyzer flag and compile using your default compiler and see what happens?
To save you some time, I'll burn a DVD with a live image of Linux Mint 20.2 to see if it works with that, as if it does, then that would probably narrow the problem down to my configuration.
Could you try if it works on this image? https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-11.1.0-amd64-DVD-1.iso
Yeah. I'll actually try both in a virtual machine. No point wasting time/DVDs.
Sorry, I meant have you tried mounting that CD image using single file mode? That's the test image I typically use. I just want to rule out problem with your Internet connection, i.e. if somehow your IP got banned by the server.
Ah okay. Tried that, works fine.
What do you mean? You managed to mount the Debian CD image in Single File Mode?
Should have clarified - yeah. Managed to mount the debian image using single file mode.
Ah no. There is still the issue with the 7z archive from https://hiddenpalace.org. It seems like that was working for you, but not me for some reason. Trying to browse the archive (or simply use cat on it), and in some cases just trying to use ls in the mount point, results in the "Transport endpoint is not connected" error.
Edit: not sure I actually have the disk space at the moment to virtualise a full linux mint/debian installation at the moment.
Hmm I wonder if it is Cloudflare - it seems my software has poor compatibility with Cloudflare.
A headless Debian installation takes less than 10GB.
Okay, will try Debian. And now that I think about it, Linux Mint should probably be max 50GB on a clean install.
I'm getting this error when trying to build it in both linux mint (similar/same problem with expat in debian too):
user1@mint-virtualbox:~/httpdirfs-master$ make CC=gcc-10
rm -f ./.depend
gcc-10 -g -O2 -Wall -Wextra -Wshadow -fsanitize=undefined -fanalyzer -Wno-analyzer-file-leak -rdynamic -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\"1.2.3\" `pkg-config --cflags-only-I gumbo libcurl fuse uuid expat` -MM src/link.c src/log.c src/fuse_local.c src/config.c src/main.c src/memcache.c src/util.c src/cache.c src/network.c src/sonic.c -MF ./.depend;
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
/bin/sh: 1: gcc-10: not found
Makefile:95: .depend: No such file or directory
make: *** [Makefile:94: .depend] Error 127
I did install the dependencies listed in the readme. Don't remember how I resolved this when building it on my normal mint instllation - but maybe the required package(s) were already installed.
Ah. I forgot that's not installed by default. But I get a similar error after installing (although it gets further):
user1@mint-virtualbox:~$ sudo apt-get install gcc-10
...
user1@mint-virtualbox:~$ cd httpdirfs-master/
user1@mint-virtualbox:~/httpdirfs-master$ make CC=gcc-10
rm -f ./.depend
gcc-10 -g -O2 -Wall -Wextra -Wshadow -fsanitize=undefined -fanalyzer -Wno-analyzer-file-leak -rdynamic -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\"1.2.3\" `pkg-config --cflags-only-I gumbo libcurl fuse uuid expat` -MM src/link.c src/log.c src/fuse_local.c src/config.c src/main.c src/memcache.c src/util.c src/cache.c src/network.c src/sonic.c -MF ./.depend;
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
gcc-10 -g -O2 -Wall -Wextra -Wshadow -fsanitize=undefined -fanalyzer -Wno-analyzer-file-leak -rdynamic -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\"1.2.3\" `pkg-config --cflags-only-I gumbo libcurl fuse uuid expat` `pkg-config --libs-only-L gumbo libcurl fuse uuid expat` -c -o main.o src/main.c
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
gcc-10 -g -O2 -Wall -Wextra -Wshadow -fsanitize=undefined -fanalyzer -Wno-analyzer-file-leak -rdynamic -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\"1.2.3\" `pkg-config --cflags-only-I gumbo libcurl fuse uuid expat` `pkg-config --libs-only-L gumbo libcurl fuse uuid expat` -c -o network.o src/network.c
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
gcc-10 -g -O2 -Wall -Wextra -Wshadow -fsanitize=undefined -fanalyzer -Wno-analyzer-file-leak -rdynamic -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\"1.2.3\" `pkg-config --cflags-only-I gumbo libcurl fuse uuid expat` `pkg-config --libs-only-L gumbo libcurl fuse uuid expat` -c -o fuse_local.o src/fuse_local.c
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
gcc-10 -g -O2 -Wall -Wextra -Wshadow -fsanitize=undefined -fanalyzer -Wno-analyzer-file-leak -rdynamic -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\"1.2.3\" `pkg-config --cflags-only-I gumbo libcurl fuse uuid expat` `pkg-config --libs-only-L gumbo libcurl fuse uuid expat` -c -o link.o src/link.c
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
src/link.c: In function ‘LinkTable_disk_open’:
src/link.c:668:5: warning: ignoring return value of ‘fread’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
668 | fread(&linktbl->num, sizeof(int), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/link.c:672:9: warning: ignoring return value of ‘fread’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
672 | fread(linktbl->links[i]->linkname, sizeof(char),
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
673 | MAX_FILENAME_LEN, fp);
| ~~~~~~~~~~~~~~~~~~~~~
src/link.c:674:9: warning: ignoring return value of ‘fread’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
674 | fread(linktbl->links[i]->f_url, sizeof(char), MAX_PATH_LEN, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/link.c:675:9: warning: ignoring return value of ‘fread’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
675 | fread(&linktbl->links[i]->type, sizeof(LinkType), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/link.c:676:9: warning: ignoring return value of ‘fread’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
676 | fread(&linktbl->links[i]->content_length, sizeof(size_t), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/link.c:677:9: warning: ignoring return value of ‘fread’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
677 | fread(&linktbl->links[i]->time, sizeof(long), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
gcc-10 -g -O2 -Wall -Wextra -Wshadow -fsanitize=undefined -fanalyzer -Wno-analyzer-file-leak -rdynamic -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\"1.2.3\" `pkg-config --cflags-only-I gumbo libcurl fuse uuid expat` `pkg-config --libs-only-L gumbo libcurl fuse uuid expat` -c -o cache.o src/cache.c
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
src/cache.c: In function ‘Meta_read’:
src/cache.c:166:5: warning: ignoring return value of ‘fread’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
166 | fread(&cf->time, sizeof(long), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/cache.c:167:5: warning: ignoring return value of ‘fread’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
167 | fread(&cf->content_length, sizeof(off_t), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/cache.c:168:5: warning: ignoring return value of ‘fread’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
168 | fread(&cf->blksz, sizeof(int), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/cache.c:169:5: warning: ignoring return value of ‘fread’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
169 | fread(&cf->segbc, sizeof(long), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
gcc-10 -g -O2 -Wall -Wextra -Wshadow -fsanitize=undefined -fanalyzer -Wno-analyzer-file-leak -rdynamic -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\"1.2.3\" `pkg-config --cflags-only-I gumbo libcurl fuse uuid expat` `pkg-config --libs-only-L gumbo libcurl fuse uuid expat` -c -o util.o src/util.c
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
gcc-10 -g -O2 -Wall -Wextra -Wshadow -fsanitize=undefined -fanalyzer -Wno-analyzer-file-leak -rdynamic -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\"1.2.3\" `pkg-config --cflags-only-I gumbo libcurl fuse uuid expat` `pkg-config --libs-only-L gumbo libcurl fuse uuid expat` -c -o sonic.o src/sonic.c
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
src/sonic.c:9:10: fatal error: expat.h: No such file or directory
9 | #include <expat.h>
| ^~~~~~~~~
compilation terminated.
make: *** [Makefile:31: sonic.o] Error 1
@mariomadproductions , could you try and see if this problem is still reproducible, now https://github.com/fangfufu/httpdirfs/pull/104 is merged in.
Hi. I've built the latest commit (e6b5688) on Linux Mint 21. Here's the result of the command:
$ httpdirfs --single-file-mode 'https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z' ~/mnt/httpdirfs
Warning:src/link.c:456:Link_set_file_stat: HTTP 404Warning:src/link.c:461:Link_set_file_stat: .
LinkTable_print: --------------------------------------------
LinkTable_print: LinkTable 0x55e1dec8b630 for https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: --------------------------------------------
LinkTable_print: 0 H 0 https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: 1 I 0 Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z https://hiddenpalace.org/w/images/4/40/Kung_Fu_Panda_%28December_7%2C_2007_prototype%29.7z
LinkTable_print: --------------------------------------------
LinkTable_print: Invalid link count: 1
LinkTable_print: --------------------------------------------
~/mnt/httpdirfs is empty