Directory virtualization / Cloud filesystem

[Dark Dragon <darkdragon-001@web.de>]

Is anyone working on a filesystem driver or filesystem overlay with the
following characteristics:
- *Kernel space* for performance reasons (no FUSE)
- *All data* is stored on some server
- *All metadata* is locally available
- *Some file data* is locally available
- *Other file data* is automatically fetched on-demand
-> *All files* can be accessed from *anywhere* (explorer, console, APIs,
...) independent of their location

Many projects currently work on their own (partly closed source)
implementations - for performance reasons often in *kernel space*
(instead of FUSE) in order to avoid context switches. In my opinion this
is bad in terms of security and stability!
Despite your personal opinion on whether (personal) data should be
stored in the (private) cloud, it is unquestionable that there is a
raising demand for these features (files are updated more often then
used locally, limited fast storage in contrast to raising amounts of
data, ...).

My suggestion is the following:
- Create a virtual filesystem (where is data stored?) or overlay
filesystem (store data in underlying filesystem) in the kernel
- Agree on a data retrieval protocol
- File selection for intelligent caching should be performed in
userspace

Current projects working on this:
- Seafile Drive client [1]
- Git Virtual Filesystem (GVFS) [2, 3]
- Dropbox Smart Sync (Project Infinite) [4, 5]
- OneDrive On-Demand [6, 7]

I also suggest the following news coverage about some more details:
- "Microsoft and GitHub team up to take Git virtual file system to
macOS, Linux" [8]
  -> last two paragraphs about implementation thoughts
- "OneDrive done right is back, and now it works properly" [9]
  -> lessons learned from Microsoft's first unsuccessful implementation

Links:
[1]:
https://blogs.seafile.com/2016/09/02/announcing-seafile-drive-client-a-new-way-to-map-seafile-storage-as-virtual-drive/
[2]: https://github.com/Microsoft/GVFS
[3]: https://github.com/Microsoft/GVFS/issues/4
[4]: https://www.dropbox.com/smartsync
[5]:
https://blogs.dropbox.com/tech/2016/05/going-deeper-with-project-infinite/
[6]:
https://support.office.com/en-us/article/Learn-about-OneDrive-Files-On-Demand-0e6860d3-d9f3-4971-b321-7092438fb38e
[7]:
https://blogs.office.com/en-us/2017/05/11/introducing-onedrive-files-on-demand-and-additional-features-making-it-easier-to-access-and-share-files/
[8]:
https://arstechnica.com/gadgets/2017/11/microsoft-and-github-team-up-to-take-git-virtual-file-system-to-macos-linux/
[9]:
https://arstechnica.com/information-technology/2017/06/onedrive-done-right-is-back-and-now-it-works-properly/