How about offsite backup?

[jospoortvliet]

ownCloud has external storage support - and has encryption. Combining the two with backup seems to make a lot of sense.

As in - offer to put the backup on one of the external storage locations, using the built in encryption to encrypt the data there.

I am not sure if this is currently even remotely doable in the API of ownCloud - just using one of the external storage plugins for data storage, while enabling encryption just for that - within an app. Pretty likely it isn't - @deepdiver can, for sure, tell me how crazy this request is, API wise.

As I said in the request for also backing up data of users - this is of course a 'layering violation', more experienced sysadmins who run several services on their server will have a whole-server backup solution. But simple users like myself, who only run ownCloud on their server anyway, are immensely helped with a built-in backup solution like what you are building.

[pbek]

I've yet found no documented way to write and read to files on an external storage but I guess there must be a way since OC is doing it in the files app. Does anyone know more?

[jospoortvliet]

@PVince81 @Xenopathic @icewind1991 have been working on storage and will know the answer.

[pbek]

Then I'm looking forward to getting some input! :+1:

[PVince81]

So far if you use ownCloud's APIs you can only operate on mounted external storages, the ones that are visible in the user's filesystem view. This would be using \OC::$server->getUserFolder().

But if you're writing an app and wanted to use the external storage backends directly without mounting them, this is also possible but not officially supported as there are only private APIs for that which are going to change in 8.2.

Basically for external storage what you need is:

• get a list of supported/registered external storage backends
• provide a UI for the user to configure them for the backup app (reuse the existing UI or provide a new one as the current one is not portable). Each backend can provide its own list of config settings.
• use the settings to instantiate the matching storage instance (which implements \OCP\Files\Storage)
• set the encryption option to true in its mount options
• make sure the encryption app is enabled, because when it's disabled the encryption option is not available for external storages
• implement the backup feature using cross-storage copy (see \OCP\Files\Storage::copyFromStorage)

[PVince81]

Most of the code for this is in "files_external", the external storage app. There are plans to move it to core, so this would prevent inter-app dependencies. See https://github.com/owncloud/core/issues/18160

[pbek]

Thanks a lot for your help, @PVince81 ! Is the external storage configured per OC installation or per user? Because the backups of ownbackup are done for the whole installation. I guess it would be enough to use mounted external storages...

[PVince81]

Normally external storages can be mounted either system-wide (admin page) or by users themselves (personal page). I suggest you try it out to get a better idea.

If the backup is done for the whole installation anyway, not just the data, then I'm not sure whether external storage as discussed here is the right approach. I thought you'd just be copying files there.

But then you could create a special "backup" user in ownCloud who has an external storage mounted locally, then upload the backup files there, that probably consist of the data of all users + database ?

[jospoortvliet]

@PVince81 that sounds like a pretty decent solution, yes. I think the ideal storage wouldn't be one that is available for all users, but a special 'backup' external storage. Like a remote FTP drive used just used for backups. Or even another ownCloud instance ;-)

[pbek]

@PVince81, so what's about an external storage, that is configured in the administration area and only made visible to administrators... Could that be used as backup storage for the backup cronjob?

Currently files of users aren't backed up, only the database is backed up.

[PVince81]

Yes, that could work. You might need to give it a special name to make sure the backup cronjob can distinguish it from the other possible external storages that admins might have configured.

[PVince81]

Please also note that if encryption is enabled, you cannot automatically decrypt the user's files. That should be fine, so the backup should not only save the encrypted files of the users but also the matching encryption keys from "files_encryption". You might also want to backup the versions which are in the "files_versions" folder.

[pbek]

Ah, ok. But I haven't take care of that as long only the database is backed up, don't I?

[PVince81]

No, encryption keys and versions are stored on disk. The database might contain references to these files but not the contents.

[pbek]

As I currently see it if I add external storage support one needs to enable the "External Storage" app to be able to use ownbackup again even if one doesn't use external storages. Am I right?

[pbek]

And has anyone an example of storing a file to an external storage and reading a file from an external storage? I didn't find one so far through reverse engineering.

[PVince81]

You either need to:

• manually instantiate the storage backend instance and then use copyFromStorage or any of its methods to write files

or

• have that external storage mounted in the admin user's FS and then use the regular FS functions (node API / \OC::$server->getUserFolder() and then write into the folder in which the external storage is mounted).

You are right that the files_external app needs to be enabled. But also that app is where the admin will be able to configure the backup folder.

[pbek]

Thanks for your hints!

[pbek]

@PVince81, I haven't figured out yet how to get from a \OCA\Files_external\Lib\StorageConfig instance, that I am already able to load, to an actual storage instance what I can write files to. Do you have a hint for me...

[PVince81]

I had a quick look in core and I think it is going to be too complicated to go the "instantiate storage" directly route. This is because you'd need a UI to be able to properly configure it, and that UI isn't simple to implement or reuse. The place where it happens in core is in some private code here https://github.com/owncloud/core/blob/v8.2.1/apps/files_external/lib/config/configadapter.php#L120 where $storage is the StorageConfig instance.

It would be much easier to have the admin configure a special user in the system with a specific external storage mount point (configured through the usual GUI), for example "/backups" and then make the backup app use the usual public API to write there. \OC::$server->getUserFolder($theUser)->get($configuredMountPointPath)->newFile()->putContent($data) were "$configuredMountPointPath" is "/backups" and "$theUser" is the configured special user who will "received" the backup.

[PVince81]

This is the interface that getUserFolder() will return: http://api.owncloud.org/classes/OCP.Files.Folder.html

[pbek]

Thank you for your great advice. Goes straight to my todo list!