udata icon indicating copy to clipboard operation
udata copied to clipboard

Fix error when retrieving GeoZones

Open mdamien opened this issue 4 years ago • 6 comments

The issue is that the script try to write to /udata/fs/tmp/ but the tmp directory does not exist

Related issue: https://github.com/opendatateam/docker-udata/issues/197

I'm not sure it's the best place to ensure the /tmp/ directory is there, maybe directly at the initialization phase in udata/core/storages/__init__.py

mdamien avatar Nov 08 '20 09:11 mdamien

This solution might work but creates a specific behavior which differs from the rest of the storages. Flask-fs uses ensure_path method on its write() and save() methods. The flask-fs way would be to download the file using the chunk's storage and use its save() method. Another simpler way would be to use python's temporary file feature instead of using a Flask-fs storage.

quaxsze avatar Nov 09 '20 10:11 quaxsze

I could do:

        log.info('Downloading GeoZones bundle: %s', filename)
        with tmp.open(GEOZONE_FILENAME, 'w') as f:
            pass
        filename, _ = urlretrieve(filename, tmp.path(GEOZONE_FILENAME))

So it works with all backends

mdamien avatar Nov 09 '20 12:11 mdamien

It's actually even more hacky than the previous one :sweat_smile: When I mentioned other storages, I meant that they download files using the chunk way and the ensure_path is implicitly called within the save function. The solution is to either use a storage provided by Flask-FS and to proceed the way the librairy is designed for, or if this is over complicated for the wanted result, to access the system tmp by using python`s tempfile instead of using flask-fs.

quaxsze avatar Nov 09 '20 12:11 quaxsze

@quaxsze ahah, this time I pushed something you will like I think, downloading the chunk way with urlopen

mdamien avatar Nov 10 '20 07:11 mdamien

Sorry my previous comment was not very clear. When I mentioned the chunks way I was thinking of using chunk's storage like in the handle_upload function. It might seem a bit over complicated that is why mentioned accessing the system tmp by using python`s tempfile instead of using flask-fs.

quaxsze avatar Nov 13 '20 16:11 quaxsze

@quaxsze pushed a simple solution with tempfile.NamedTemporaryFile, let me know if it's what you're looking for

mdamien avatar Nov 19 '20 08:11 mdamien

I don't think this bug exists anymore. Feel free to open an issue or a new PR if it's still the case.

ThibaudDauce avatar Apr 04 '24 09:04 ThibaudDauce