Can only list 1 directory deep on OpenStack Swift S3

[tcassaert]

geesefs version: v0.34.2

procedure

Mounting a bucket with:

./geesefs -f --debug_s3 --endpoint http://data.cloudferro.com DIAS /opt/eodata

expected result Being able to access subdirectories like

ls /opt/eodata/Sentinel-1
SAR

actual result

ls /opt/eodata/Sentinel-1

log output

./geesefs -f --debug_s3 --endpoint http://data.cloudferro.com DIAS /opt/eodata
2023/01/09 15:00:59.113838 s3.DEBUG HEAD http://data.cloudferro.com/DIAS = 200 []
2023/01/09 15:00:59.114094 s3.DEBUG Date = [Mon, 09 Jan 2023 00:00:01 GMT]
2023/01/09 15:00:59.114109 s3.DEBUG Server = [nginx/1.21.3]
2023/01/09 15:00:59.114118 s3.DEBUG Content-Type = [binary/octet-stream]
2023/01/09 15:00:59.114125 s3.DEBUG Content-Length = [0]
2023/01/09 15:00:59.114132 s3.DEBUG Connection = [keep-alive]
2023/01/09 15:00:59.114139 s3.DEBUG Accept-Ranges = [bytes]
2023/01/09 15:00:59.114146 s3.DEBUG Etag = ["d41d8cd98f00b204e9800998ecf8427e"]
2023/01/09 15:00:59.114154 s3.DEBUG Last-Modified = [Mon, 09 Jan 2023 00:00:01 GMT]
2023/01/09 15:00:59.114163 s3.INFO anonymous bucket detected
2023/01/09 15:00:59.114798 s3.DEBUG DEBUG: Request s3/HeadObject Details:
---[ REQUEST POST-SIGN ]-----------------------------
HEAD /DIAS/chqtv4ak2ibgb3q1mej6huqpbkhi5j2k HTTP/1.1
Host: data.cloudferro.com
User-Agent: GeeseFS/0.34.1 (go1.16.15; linux; amd64)


-----------------------------------------------------
2023/01/09 15:00:59.128790 s3.DEBUG DEBUG: Response s3/HeadObject Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 404 NOT FOUND
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Date: Mon, 09 Jan 2023 14:00:59 GMT
Server: nginx/1.21.3
Content-Length: 0


-----------------------------------------------------
2023/01/09 15:00:59.128854 s3.DEBUG DEBUG: Validate Response s3/HeadObject failed, attempt 0/3, error NotFound: Not Found
        status code: 404, request id: , host id:
2023/01/09 15:00:59.129212 s3.DEBUG DEBUG: Request s3/ListMultipartUploads Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /DIAS?uploads= HTTP/1.1
Host: data.cloudferro.com
User-Agent: GeeseFS/0.34.1 (go1.16.15; linux; amd64)
Accept-Encoding: identity


-----------------------------------------------------
2023/01/09 15:00:59.133956 s3.DEBUG DEBUG: Response s3/ListMultipartUploads Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 404 NOT FOUND
Content-Length: 39
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Date: Mon, 09 Jan 2023 14:00:59 GMT
Server: nginx/1.21.3


-----------------------------------------------------
2023/01/09 15:00:59.134014 s3.DEBUG DEBUG: Validate Response s3/ListMultipartUploads failed, attempt 0/3, error NotFound: Not Found
        status code: 404, request id: , host id:
2023/01/09 15:00:59.135832 main.INFO File system has been successfully mounted.

so that seems fine. Doing an ls /opt/eodata/Sentinel-1 gives

2023/01/09 15:01:35.493122 s3.DEBUG DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /DIAS?marker=Sentinel-1&prefix= HTTP/1.1
Host: data.cloudferro.com
User-Agent: GeeseFS/0.34.1 (go1.16.15; linux; amd64)
Accept-Encoding: identity


-----------------------------------------------------
2023/01/09 15:01:35.600684 s3.DEBUG DEBUG: Response s3/ListObjects Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 404 NOT FOUND
Content-Length: 27
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Date: Mon, 09 Jan 2023 14:01:35 GMT
Server: nginx/1.21.3


-----------------------------------------------------
2023/01/09 15:01:35.600768 s3.DEBUG DEBUG: Validate Response s3/ListObjects failed, attempt 0/3, error NotFound: Not Found
        status code: 404, request id: , host id:
2023/01/09 15:01:35.600833 s3.ERROR ListObjects &{0xc000069ca0 <nil> <nil> 0xc000069c90 <nil>} = NotFound: Not Found
        status code: 404, request id: , host id:
2023/01/09 15:01:35.602105 s3.DEBUG DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /DIAS?marker=Sentinel-1&prefix= HTTP/1.1
Host: data.cloudferro.com
User-Agent: GeeseFS/0.34.1 (go1.16.15; linux; amd64)
Accept-Encoding: identity


-----------------------------------------------------
2023/01/09 15:01:35.706549 s3.DEBUG DEBUG: Response s3/ListObjects Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 404 NOT FOUND
Content-Length: 27
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Date: Mon, 09 Jan 2023 14:01:35 GMT
Server: nginx/1.21.3


-----------------------------------------------------
2023/01/09 15:01:35.706606 s3.DEBUG DEBUG: Validate Response s3/ListObjects failed, attempt 0/3, error NotFound: Not Found
        status code: 404, request id: , host id:
2023/01/09 15:01:35.706637 s3.ERROR ListObjects &{0xc00052c5c0 <nil> <nil> 0xc00052c5b0 <nil>} = NotFound: Not Found
        status code: 404, request id: , host id:

The problem seems to be that GET /DIAS?marker=Sentinel-1&prefix= HTTP/1.1. Doing a direct curl gives:

curl 'data.cloudferro.com/DIAS?marker=Sentinel-1&prefix='
'Sentinel-1' is not in list

While this does work with for example goofys, which seems to be doing a GET /DIAS?delimiter=%2F&prefix=Sentinel-1%2F. This does give the correct output:

curl 'data.cloudferro.com/DIAS?delimiter=%2F&prefix=Sentinel-1%2F'
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
        <Name>DIAS</Name>
        <Prefix>Sentinel-1/</Prefix>
        <Marker></Marker>
        <NextMarker></NextMarker>
        <MaxKeys>1000</MaxKeys>
        <Delimiter>/</Delimiter>
        <IsTruncated>false</IsTruncated>

        <CommonPrefixes>
                <Prefix>Sentinel-1/SAR/</Prefix>
        </CommonPrefixes>

</ListBucketResult>