operator
operator copied to clipboard
Allow (or document) Container.push behavior on overwrite.
Issue
Container.push
currently does not allow file overwriting, but the behavior is not documented in docs.
Sugestion
- Add a boolean
overwrite
flag that whenTrue
test if path exists and remove it before pushing. - Update docs about overwrite behavior.
@paulomach I can't reproduce this locally. I'm running the Pebble server/daemon in one terminal:
$ PEBBLE=~/pebble go run ./cmd/pebble run
...
And then using the test Pebble CLI to execute a push
, like so:
$ PEBBLE=~/pebble python3 -m test.pebble_cli push ./setup.py /home/ben/w/pebble/setup.py
wrote ./setup.py to remote file /home/ben/w/pebble/setup.py
$ PEBBLE=~/pebble python3 -m test.pebble_cli push ./setup.py /home/ben/w/pebble/setup.py
wrote ./setup.py to remote file /home/ben/w/pebble/setup.py
# modify ./setup.py
$ PEBBLE=~/pebble python3 -m test.pebble_cli push ./setup.py /home/ben/w/pebble/setup.py
wrote ./setup.py to remote file /home/ben/w/pebble/setup.py
# changes appear in /home/ben/w/pebble/setup.py
I can do the push successfully many times, and when I make changes in the source, the destination updates fine.
Are you able to send a repro for the case you're seeing, and the error messages you get?
Hi @benhoyt , absolutely.
Since I've observed that on charm context, I've wrote a simple charm that implements replicates the error, available here.
The traceback observed is:
Traceback (most recent call last):
File "./src/charm.py", line 44, in <module>
main(TestPebblePushCharm)
File "/var/lib/juju/agents/unit-test-pebble-push-0/charm/venv/ops/main.py", line 431, in main
_emit_charm_event(charm, dispatcher.event_name)
File "/var/lib/juju/agents/unit-test-pebble-push-0/charm/venv/ops/main.py", line 142, in _emit_charm_event
event_to_emit.emit(*args, **kwargs)
File "/var/lib/juju/agents/unit-test-pebble-push-0/charm/venv/ops/framework.py", line 283, in emit
framework._emit(event)
File "/var/lib/juju/agents/unit-test-pebble-push-0/charm/venv/ops/framework.py", line 743, in _emit
self._reemit(event_path)
File "/var/lib/juju/agents/unit-test-pebble-push-0/charm/venv/ops/framework.py", line 790, in _reemit
custom_handler(event)
File "./src/charm.py", line 38, in _on_pebble_ready
container.push("/etc/resolv.conf", "\n".join(content))
File "/var/lib/juju/agents/unit-test-pebble-push-0/charm/venv/ops/model.py", line 1282, in push
self._pebble.push(path, source, encoding=encoding, make_dirs=make_dirs,
File "/var/lib/juju/agents/unit-test-pebble-push-0/charm/venv/ops/pebble.py", line 1726, in push
self._raise_on_path_error(resp, path)
File "/var/lib/juju/agents/unit-test-pebble-push-0/charm/venv/ops/pebble.py", line 1683, in _raise_on_path_error
raise PathError(error['kind'], error['message'])
ops.pebble.PathError: generic-file-error - rename /etc/resolv.conf.8xThkKlBDntr~ /etc/resolv.conf: device or resource busy
This is because another process has resolv.conf
opened - the file is currently in use and can't be removed/overwritten. You can lsof | grep 'resolv.conf'
to check what processes are using it.
Hey @rwcarlsen , that's was my initial thought also, but there's no process owning the file.
Running lsof
does not show any process with the file open (link to pastebin)
And this can be reproduced to others containers files, not only resolv.conf
.
Sounds like this might be the issue: https://stackoverflow.com/questions/60549775/device-or-resource-busy-when-i-try-move-etc-resolv-conf-in-ubuntu18-04-how
That's the issue. There other files of interest that are bind mounts, like /etc/hosts
.
Having the push
implementation on pebble to copy over the original file can make it work (instead of mv/rename).
But I don't know if there's any drawbacks on doing that.
Should that limitation (overwrite bind mounts) be documented?
It's probably worth mentioning in docs somewhere that bind mounts for files will interfere with workload-side file operations.
Yeah, let's update the push docs to mention this gotcha and perhaps suggest a workaround for locked file / bind mount cases ("use Container.exec for full control" or whatever).