Support FileProvider API (VFS) in macOS

[helmut72]

https://www.apple.com/macos/catalina-preview/features/

Third-party cloud service integration An all-new FileProvider API for cloud storage providers delivers a new way to seamlessly integrate their services into the Finder without requiring a kernel extension, helping to maintain the security of your Mac. Cloud storage providers can now deliver their apps through the Mac App Store.

[pgauret]

Now that Catalina is out may be a good opportunity to look into FileProvider. This would allow to modernise the macOS desktop client and bring equivalent functionality to the virtual drive for Windows client.

[theJian]

Catalina is out but I can't find a way to create FileProviderExtension on macOS. Does anyone notice that FileProvider is actually removed from Catalina features page?

[mikelynch]

It's strange that it's not listed as a feature in the release, but the Developer documentation does say it's available in macOS 10.15+. I'm afraid I don't know anything about macOS development, but there are some First Steps topics there. @theJian, do those provide any useful guidance for getting started?

It's also just occured to me that this is already implemented in the iOS app, and hopefully that component should also work on macOS. Would that give an easier route to using this API?

One consideration is that presumably the current logic will have to be kept for the moment, in order to support older versions of macOS – I imagine it wouldn't be desirable to require Catalina for a while yet. It'd be great to be support this API as an option though!

[theJian]

@mikelynch I didn't find those useful. Luckily once you created a FileProviderExtension, Xcode will generate template code for you. It's a working simple demo and full of comments. Getting started by reading those code is much easy in my opinion.

[pointlessone]

Documentation is a little inconsistent. On the File Provider Framework page it states availability to be macOS 10.15+ but on the individual protocol pages (e.g. NSFileProviderReplicatedExtension) it's macOS 11.0+. I guess this feature didn't make it into Catalina but since Big Sur is around the corner maybe it's time to revisit this issue?

[leiless]

FYI, File Provider - macOS Support seems available by now.

[aniqueta]

Is this something being worked on? With Big Sur changes, it seems safe to assume that in the next macOS version using the FileProvider API will become mandatory and the Finder extension approach will be deprecated. Thanks.

[leiless]

Is this something being worked on?

Don't know.

It seems that FileProvider not ready yet, and Apple hasn't given any sample code on it.

[hemancuso]

@leiless we actually just shipped Nextcloud support in our FileProvider product (Strongsync). Definitely is ready. Happy to answer questions.

[leiless]

@leiless we actually just shipped Nextcloud support in our FileProvider product (Strongsync). Definitely is ready. Happy to answer questions.

Does there any sample code on FileProvider API? I've checked Apple Developer docs, haven't seeing it yet.

How you guys support FileProvider for Strongsync?

[hemancuso]

I believe the very latest XCode betas generate a template project

[helmut72]

@leiless we actually just shipped Nextcloud support in our FileProvider product (Strongsync). Definitely is ready. Happy to answer questions.

Target: common Apache2 WebDAV

Tasks:

1. Created test folder on macOS: ok, also on the server
2. Copied my Notes (a lot of Markdown files in private and work with subfolders): failed. Work and all subfolders are on the server, no subfolder in Private. On both no single file was uploaded!
3. Deleted test folder: still on the server (empty trashcan on macOS doesn't helped)
4. Result: uninstalled Strongsync

[andreasley]

Xcode 12.5, which was released today, contains a File Provider Extension target template to help getting started with macOS File Provider extensions.

[leiless]

Xcode 12.5, which was released today, contains a File Provider Extension target template to help getting started with macOS File Provider extensions.

FYI: https://developer.apple.com/documentation/xcode-release-notes/xcode-12_5-release-notes

[aniqueta]

Glad to see the activity here. Looks like the API is maturing in macOS Monterey. FYI: Box has adopted it now: https://blog.box.com/get-more-secure-seamless-box-drive-experience-across-macs

[solracsf]

See https://github.com/nextcloud/desktop/blob/master/doc/architecture.rst#virtual-files for how to enable experimental support on v3.4.0.

[pszypowicz]

See https://github.com/nextcloud/desktop/blob/master/doc/architecture.rst#virtual-files for how to enable experimental support on v3.4.0.

Hi, is this coded with a FileProviderAPI? Could you point to a commit/PR implementing this?

[aniqueta]

Note that OneDrive [1] and Dropbox [2] are stating there are changes coming in macOS 12.3 (March?) that is making them fully migrate over to the FileProvider API, with macOS 12.2 being the last version they support with their current virtual files / online files implementation. I can't find anything from Apple indicating what these changes are, but I suspect we'll find out soon...

[1] https://techcommunity.microsoft.com/t5/microsoft-onedrive-blog/inside-the-new-files-on-demand-experience-on-macos/bc-p/3066588 [2] https://help.dropbox.com/installs-integrations/desktop/macos-12-monterey-support

[jeffmancuso]

They both have exceptions to the no kernel extension policy allowing them to use their previous solutions. I imagine this exception is finally going away in a late-cycle point release rather than in macOS 13. This increases their pressure to actually migrate over to FPE.

[marcotrevisan]

Just for the records: https://developer.apple.com/documentation/fileprovider/macos_support/syncing_files_on_macos contains an example XCode project that uses NSFileProviderReplicatedExtension.

Would be great to have the system take care of most of the synchronization work for you, and you only have to focus on the client calls and the extra features.

[claucambra]

Adding support for File Provider is not exactly straightforward with the desktop client as all of our networking and synchronisation code is written in Qt/C++ and therefore not easily usable by a File Provider extension. It seems to me that an easier way of going about this would be to make the File Provider extensions independently network with the server, and therefore making them mostly independent.

Since we have already have a great iOS app that supports the File Provider API on iOS, we should borrow some of the code from there. Unfortunately, there are also a lot of small differences in how some APIs work on iOS vs macOS (including File Provider itself) so this is not exactly straightforward either.

Here is a basic list of the things that need to be done before the extension is ready:

• [X] Get the Nextcloud iOS communication library to build on macOS
• [ ] Test the Nextcloud iOS communication library on macOS and ensure nothing is borked
• [X] Graft iOS networking and service code onto the File Provider Extension in the desktop client
• [X] Get the File Provider extension with iOS code to compile on macOS
• [X] Get adding per-account File Provider domains to work
• [ ] Correctly acquire desktop client user account data from the side of the File Provider extension
• [ ] Test each iOS component to ensure there is no borkage
• [ ] Securely connect to the server through the File Provider extension
• [ ] Successfully fetch user files in File Provider
• [ ] Successfully sync local changes in File Provider
• [ ] Successfully sync server-side changes in File Provider

[marcotrevisan]

@claucambra in my humble opinion this is a very important improvement, seems very reasonable as you described it and should be kept in high priority, as the current Mac OS implementation is struggling to keep up with the everyday user needs, especially when many files and folders are involved. This change would resolve "by design" many important issues and shortcomings (just today a colleague's local disk went out of space because he downloaded too much stuff and the NC folder was over 100GB even if he's using virtual files.. a FileProvider implementation would have freed up space).

As a user, I have tried other clients (both paid and free) that perform solutions similar to the prospected one, but they all fall short in using only a "provider agnostic" set of the Webdav API. For example most don't establish a websocket for push notifications (assuming the Client Push app is up and running), or don't use the NC chunked I/O api, etc etc... Or, they don't use File Provider API and kinda implement a local proxied network share... while that's good for space saving and generally works (except for Spotlight searches), when they run into exceptions the behaviour is often unpredictable and the cause is obscure to the user.

Thanks!

[alex0411]

Hi, are there any updates when this feature will be pulled into the development / released? At the moment this missing feature is holding us back to use Nextcloud in a productive environment, since a lot of our users are using macos based clients and using the experimental option in the client is not an option for a productive environment.

Thanks a lot!

[alex0411]

Hi, are there any updates when this feature will be pulled into the development / released? At the moment this missing feature is holding us back to use Nextcloud in a productive environment, since a lot of our users are using macos based clients and using the experimental option in the client is not an option for a productive environment.

Thanks a lot!

Push

[gaodahai]

Hi, are there any updates when this feature will be pulled into the development / released? At the moment this missing feature is holding us back to use Nextcloud in a productive environment, since a lot of our users are using macos based clients and using the experimental option in the client is not an option for a productive environment.

Thanks a lot!

Hi, we also just saw the mentioned troubles happen. Is there any possibility to get this fixed? Or how do you work to avoid these troubles? Thank you very much!

[alex0411]

Hi, are there any updates when this feature will be pulled into the development / released? At the moment this missing feature is holding us back to use Nextcloud in a productive environment, since a lot of our users are using macos based clients and using the experimental option in the client is not an option for a productive environment.

Thanks a lot!

Push again. Maybe somebody could give us some feedback here.

[marcotrevisan]

I'm not a NC developer but seems like the team is working on improving the current sync engine quality, even if I was unable to properly check for progress in the FileProvider support (anyway NextcloudKit was created and received merges). #4970, #5016, #5011 are just examples of that. I'm just guessing but the dev team may be currently preferring focusing on the "SyncEngine", making it better. In this case I'd urge implementing #4464 which in my humble opinion would make it work hugely better on all platforms. I can live with ".nextcloud" files, much less with a sync engine that scans the whole world. Thanks!

[hemancuso]

Only posting again on this thread, just in case people are curious or want some help on an implementation. I had implemented support in a product I make back in 2021 comment and will be soon adding support to ExpanDrive. Whoever on the team ends up taking on this project feel free to ping me if you need assistance.