++Currently these features are dropped temporary from aufs4.
++See design/08plan.txt in detail.
++- nested mount, i.e. aufs as readonly no-whiteout branch of another aufs
++ (robr)
++- statistics of aufs thread (/sys/fs/aufs/stat)
++
++Features or just an idea in the future (see also design/*.txt),
++- reorder the branch index without del/re-add.
++- permanent xino files for NFSD
++- an option for refreshing the opened files after add/del branches
++- light version, without branch manipulation. (unnecessary?)
++- copyup in userspace
++- inotify in userspace
++- readv/writev
++
++
++2. Download
++----------------------------------------
++There are three GIT trees for aufs4, aufs4-linux.git,
++aufs4-standalone.git, and aufs-util.git. Note that there is no "4" in
++"aufs-util.git."
++While the aufs-util is always necessary, you need either of aufs4-linux
++or aufs4-standalone.
++
++The aufs4-linux tree includes the whole linux mainline GIT tree,
++git://git.kernel.org/.../torvalds/linux.git.
++And you cannot select CONFIG_AUFS_FS=m for this version, eg. you cannot
++build aufs4 as an external kernel module.
++Several extra patches are not included in this tree. Only
++aufs4-standalone tree contains them. They are described in the later
++section "Configuration and Compilation."
++
++On the other hand, the aufs4-standalone tree has only aufs source files
++and necessary patches, and you can select CONFIG_AUFS_FS=m.
++But you need to apply all aufs patches manually.
++
++You will find GIT branches whose name is in form of "aufs4.x" where "x"
++represents the linux kernel version, "linux-4.x". For instance,
++"aufs4.0" is for linux-4.0. For latest "linux-4.x-rcN", use
++"aufs4.x-rcN" branch.
++
++o aufs4-linux tree
++$ git clone --reference /your/linux/git/tree \
++ git://github.com/sfjro/aufs4-linux.git aufs4-linux.git
++- if you don't have linux GIT tree, then remove "--reference ..."
++$ cd aufs4-linux.git
++$ git checkout origin/aufs4.0
++
++Or You may want to directly git-pull aufs into your linux GIT tree, and
++leave the patch-work to GIT.
++$ cd /your/linux/git/tree
++$ git remote add aufs4 git://github.com/sfjro/aufs4-linux.git
++$ git fetch aufs4
++$ git checkout -b my4.0 v4.0
++$ (add your local change...)
++$ git pull aufs4 aufs4.0
++- now you have v4.0 + your_changes + aufs4.0 in you my4.0 branch.
++- you may need to solve some conflicts between your_changes and
++ aufs4.0. in this case, git-rerere is recommended so that you can
++ solve the similar conflicts automatically when you upgrade to 4.1 or
++ later in the future.
++
++o aufs4-standalone tree
++$ git clone git://github.com/sfjro/aufs4-standalone.git aufs4-standalone.git
++$ cd aufs4-standalone.git
++$ git checkout origin/aufs4.0
++
++o aufs-util tree
++$ git clone git://git.code.sf.net/p/aufs/aufs-util aufs-util.git
++- note that the public aufs-util.git is on SourceForge instead of
++ GitHUB.
++$ cd aufs-util.git
++$ git checkout origin/aufs4.0
++
++Note: The 4.x-rcN branch is to be used with `rc' kernel versions ONLY.
++The minor version number, 'x' in '4.x', of aufs may not always
++follow the minor version number of the kernel.
++Because changes in the kernel that cause the use of a new
++minor version number do not always require changes to aufs-util.
++
++Since aufs-util has its own minor version number, you may not be
++able to find a GIT branch in aufs-util for your kernel's
++exact minor version number.
++In this case, you should git-checkout the branch for the
++nearest lower number.
++
++For (an unreleased) example:
++If you are using "linux-4.10" and the "aufs4.10" branch
++does not exist in aufs-util repository, then "aufs4.9", "aufs4.8"
++or something numerically smaller is the branch for your kernel.
++
++Also you can view all branches by
++ $ git branch -a
++
++
++3. Configuration and Compilation
++----------------------------------------
++Make sure you have git-checkout'ed the correct branch.
++
++For aufs4-linux tree,
++- enable CONFIG_AUFS_FS.
++- set other aufs configurations if necessary.
++
++For aufs4-standalone tree,
++There are several ways to build.
++
++1.
++- apply ./aufs4-kbuild.patch to your kernel source files.
++- apply ./aufs4-base.patch too.
++- apply ./aufs4-mmap.patch too.
++- apply ./aufs4-standalone.patch too, if you have a plan to set
++ CONFIG_AUFS_FS=m. otherwise you don't need ./aufs4-standalone.patch.
++- copy ./{Documentation,fs,include/uapi/linux/aufs_type.h} files to your
++ kernel source tree. Never copy $PWD/include/uapi/linux/Kbuild.
++- enable CONFIG_AUFS_FS, you can select either
++ =m or =y.
++- and build your kernel as usual.
++- install the built kernel.
++ Note: Since linux-3.9, every filesystem module requires an alias
++ "fs-<fsname>". You should make sure that "fs-aufs" is listed in your
++ modules.aliases file if you set CONFIG_AUFS_FS=m.
++- install the header files too by "make headers_install" to the
++ directory where you specify. By default, it is $PWD/usr.
++ "make help" shows a brief note for headers_install.
++- and reboot your system.
++
++2.
++- module only (CONFIG_AUFS_FS=m).
++- apply ./aufs4-base.patch to your kernel source files.
++- apply ./aufs4-mmap.patch too.
++- apply ./aufs4-standalone.patch too.
++- build your kernel, don't forget "make headers_install", and reboot.
++- edit ./config.mk and set other aufs configurations if necessary.
++ Note: You should read $PWD/fs/aufs/Kconfig carefully which describes
++ every aufs configurations.
++- build the module by simple "make".
++ Note: Since linux-3.9, every filesystem module requires an alias
++ "fs-<fsname>". You should make sure that "fs-aufs" is listed in your
++ modules.aliases file.
++- you can specify ${KDIR} make variable which points to your kernel
++ source tree.
++- install the files
++ + run "make install" to install the aufs module, or copy the built
++ $PWD/aufs.ko to /lib/modules/... and run depmod -a (or reboot simply).
++ + run "make install_headers" (instead of headers_install) to install
++ the modified aufs header file (you can specify DESTDIR which is
++ available in aufs standalone version's Makefile only), or copy
++ $PWD/usr/include/linux/aufs_type.h to /usr/include/linux or wherever
++ you like manually. By default, the target directory is $PWD/usr.
++- no need to apply aufs4-kbuild.patch, nor copying source files to your
++ kernel source tree.
++
++Note: The header file aufs_type.h is necessary to build aufs-util
++ as well as "make headers_install" in the kernel source tree.
++ headers_install is subject to be forgotten, but it is essentially
++ necessary, not only for building aufs-util.
++ You may not meet problems without headers_install in some older
++ version though.
++
++And then,
++- read README in aufs-util, build and install it
++- note that your distribution may contain an obsoleted version of
++ aufs_type.h in /usr/include/linux or something. When you build aufs
++ utilities, make sure that your compiler refers the correct aufs header
++ file which is built by "make headers_install."
++- if you want to use readdir(3) in userspace or pathconf(3) wrapper,
++ then run "make install_ulib" too. And refer to the aufs manual in
++ detail.
++
++There several other patches in aufs4-standalone.git. They are all
++optional. When you meet some problems, they will help you.
++- aufs4-loopback.patch
++ Supports a nested loopback mount in a branch-fs. This patch is
++ unnecessary until aufs produces a message like "you may want to try
++ another patch for loopback file".
++- vfs-ino.patch
++ Modifies a system global kernel internal function get_next_ino() in
++ order to stop assigning 0 for an inode-number. Not directly related to
++ aufs, but recommended generally.
++- tmpfs-idr.patch
++ Keeps the tmpfs inode number as the lowest value. Effective to reduce
++ the size of aufs XINO files for tmpfs branch. Also it prevents the
++ duplication of inode number, which is important for backup tools and
++ other utilities. When you find aufs XINO files for tmpfs branch
++ growing too much, try this patch.
++- lockdep-debug.patch
++ Because aufs is not only an ordinary filesystem (callee of VFS), but
++ also a caller of VFS functions for branch filesystems, subclassing of
++ the internal locks for LOCKDEP is necessary. LOCKDEP is a debugging
++ feature of linux kernel. If you enable CONFIG_LOCKDEP, then you will
++ need to apply this debug patch to expand several constant values.
++ If don't know what LOCKDEP, then you don't have apply this patch.
++
++
++4. Usage
++----------------------------------------
++At first, make sure aufs-util are installed, and please read the aufs
++manual, aufs.5 in aufs-util.git tree.
++$ man -l aufs.5
++
++And then,
++$ mkdir /tmp/rw /tmp/aufs
++# mount -t aufs -o br=/tmp/rw:${HOME} none /tmp/aufs
++
++Here is another example. The result is equivalent.
++# mount -t aufs -o br=/tmp/rw=rw:${HOME}=ro none /tmp/aufs
++ Or
++# mount -t aufs -o br:/tmp/rw none /tmp/aufs
++# mount -o remount,append:${HOME} /tmp/aufs
++
++Then, you can see whole tree of your home dir through /tmp/aufs. If
++you modify a file under /tmp/aufs, the one on your home directory is
++not affected, instead the same named file will be newly created under
++/tmp/rw. And all of your modification to a file will be applied to
++the one under /tmp/rw. This is called the file based Copy on Write
++(COW) method.
++Aufs mount options are described in aufs.5.
++If you run chroot or something and make your aufs as a root directory,
++then you need to customize the shutdown script. See the aufs manual in
++detail.
++
++Additionally, there are some sample usages of aufs which are a
++diskless system with network booting, and LiveCD over NFS.
++See sample dir in CVS tree on SourceForge.
++
++
++5. Contact
++----------------------------------------
++When you have any problems or strange behaviour in aufs, please let me
++know with:
++- /proc/mounts (instead of the output of mount(8))
++- /sys/module/aufs/*
++- /sys/fs/aufs/* (if you have them)
++- /debug/aufs/* (if you have them)
++- linux kernel version
++ if your kernel is not plain, for example modified by distributor,
++ the url where i can download its source is necessary too.
++- aufs version which was printed at loading the module or booting the
++ system, instead of the date you downloaded.
++- configuration (define/undefine CONFIG_AUFS_xxx)
++- kernel configuration or /proc/config.gz (if you have it)
++- behaviour which you think to be incorrect
++- actual operation, reproducible one is better
++- mailto: aufs-users at lists.sourceforge.net
++
++Usually, I don't watch the Public Areas(Bugs, Support Requests, Patches,
++and Feature Requests) on SourceForge. Please join and write to
++aufs-users ML.
++
++
++6. Acknowledgements
++----------------------------------------
++Thanks to everyone who have tried and are using aufs, whoever
++have reported a bug or any feedback.
++
++Especially donators:
++Tomas Matejicek(slax.org) made a donation (much more than once).
++ Since Apr 2010, Tomas M (the author of Slax and Linux Live
++ scripts) is making "doubling" donations.
++ Unfortunately I cannot list all of the donators, but I really
++ appreciate.
++ It ends Aug 2010, but the ordinary donation URL is still available.
++ <http://sourceforge.net/donate/index.php?group_id=167503>
++Dai Itasaka made a donation (2007/8).
++Chuck Smith made a donation (2008/4, 10 and 12).
++Henk Schoneveld made a donation (2008/9).
++Chih-Wei Huang, ASUS, CTC donated Eee PC 4G (2008/10).
++Francois Dupoux made a donation (2008/11).
++Bruno Cesar Ribas and Luis Carlos Erpen de Bona, C3SL serves public
++ aufs2 GIT tree (2009/2).
++William Grant made a donation (2009/3).
++Patrick Lane made a donation (2009/4).
++The Mail Archive (mail-archive.com) made donations (2009/5).
++Nippy Networks (Ed Wildgoose) made a donation (2009/7).
++New Dream Network, LLC (www.dreamhost.com) made a donation (2009/11).
++Pavel Pronskiy made a donation (2011/2).
++Iridium and Inmarsat satellite phone retailer (www.mailasail.com), Nippy
++ Networks (Ed Wildgoose) made a donation for hardware (2011/3).
++Max Lekomcev (DOM-TV project) made a donation (2011/7, 12, 2012/3, 6 and
++11).
++Sam Liddicott made a donation (2011/9).
++Era Scarecrow made a donation (2013/4).
++Bor Ratajc made a donation (2013/4).
++Alessandro Gorreta made a donation (2013/4).
++POIRETTE Marc made a donation (2013/4).
++Alessandro Gorreta made a donation (2013/4).
++lauri kasvandik made a donation (2013/5).
++"pemasu from Finland" made a donation (2013/7).
++The Parted Magic Project made a donation (2013/9 and 11).
++Pavel Barta made a donation (2013/10).
++Nikolay Pertsev made a donation (2014/5).
++James B made a donation (2014/7 and 2015/7).
++Stefano Di Biase made a donation (2014/8).
++Daniel Epellei made a donation (2015/1).
++OmegaPhil made a donation (2016/1).
++Tomasz Szewczyk made a donation (2016/4).
++James Burry made a donation (2016/12).
++
++Thank you very much.
++Donations are always, including future donations, very important and
++helpful for me to keep on developing aufs.
++
++
++7.
++----------------------------------------
++If you are an experienced user, no explanation is needed. Aufs is
++just a linux filesystem.
++
++
++Enjoy!
++
++# Local variables: ;
++# mode: text;
++# End: ;
+diff -urNp -x '*.orig' linux-4.9/Documentation/filesystems/aufs/design/01intro.txt linux-4.9/Documentation/filesystems/aufs/design/01intro.txt
+--- linux-4.9/Documentation/filesystems/aufs/design/01intro.txt 1970-01-01 01:00:00.000000000 +0100
++++ linux-4.9/Documentation/filesystems/aufs/design/01intro.txt 2021-02-24 16:15:09.518240088 +0100
+@@ -0,0 +1,171 @@
++
++# Copyright (C) 2005-2018 Junjiro R. Okajima
++#
++# This program is free software; you can redistribute it and/or modify
++# it under the terms of the GNU General Public License as published by
++# the Free Software Foundation; either version 2 of the License, or
++# (at your option) any later version.
++#
++# This program is distributed in the hope that it will be useful,
++# but WITHOUT ANY WARRANTY; without even the implied warranty of
++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
++# GNU General Public License for more details.
++#
++# You should have received a copy of the GNU General Public License
++# along with this program. If not, see <http://www.gnu.org/licenses/>.
++
++Introduction
++----------------------------------------
++
++aufs [ei ju: ef es] | /ey-yoo-ef-es/ | [a u f s]
++1. abbrev. for "advanced multi-layered unification filesystem".
++2. abbrev. for "another unionfs".
++3. abbrev. for "auf das" in German which means "on the" in English.
++ Ex. "Butter aufs Brot"(G) means "butter onto bread"(E).
++ But "Filesystem aufs Filesystem" is hard to understand.
++4. abbrev. for "African Urban Fashion Show".
++
++AUFS is a filesystem with features:
++- multi layered stackable unification filesystem, the member directory
++ is called as a branch.
++- branch permission and attribute, 'readonly', 'real-readonly',
++ 'readwrite', 'whiteout-able', 'link-able whiteout', etc. and their
++ combination.
++- internal "file copy-on-write".
++- logical deletion, whiteout.
++- dynamic branch manipulation, adding, deleting and changing permission.
++- allow bypassing aufs, user's direct branch access.
++- external inode number translation table and bitmap which maintains the
++ persistent aufs inode number.
++- seekable directory, including NFS readdir.
++- file mapping, mmap and sharing pages.
++- pseudo-link, hardlink over branches.
++- loopback mounted filesystem as a branch.
++- several policies to select one among multiple writable branches.
++- revert a single systemcall when an error occurs in aufs.
++- and more...
++
++
++Multi Layered Stackable Unification Filesystem
++----------------------------------------------------------------------
++Most people already knows what it is.
++It is a filesystem which unifies several directories and provides a
++merged single directory. When users access a file, the access will be
++passed/re-directed/converted (sorry, I am not sure which English word is
++correct) to the real file on the member filesystem. The member
++filesystem is called 'lower filesystem' or 'branch' and has a mode
++'readonly' and 'readwrite.' And the deletion for a file on the lower
++readonly branch is handled by creating 'whiteout' on the upper writable
++branch.
++
++On LKML, there have been discussions about UnionMount (Jan Blunck,
++Bharata B Rao and Valerie Aurora) and Unionfs (Erez Zadok). They took
++different approaches to implement the merged-view.
++The former tries putting it into VFS, and the latter implements as a
++separate filesystem.
++(If I misunderstand about these implementations, please let me know and
++I shall correct it. Because it is a long time ago when I read their
++source files last time).
++
++UnionMount's approach will be able to small, but may be hard to share
++branches between several UnionMount since the whiteout in it is
++implemented in the inode on branch filesystem and always
++shared. According to Bharata's post, readdir does not seems to be
++finished yet.
++There are several missing features known in this implementations such as
++- for users, the inode number may change silently. eg. copy-up.
++- link(2) may break by copy-up.
++- read(2) may get an obsoleted filedata (fstat(2) too).
++- fcntl(F_SETLK) may be broken by copy-up.
++- unnecessary copy-up may happen, for example mmap(MAP_PRIVATE) after
++ open(O_RDWR).
++
++In linux-3.18, "overlay" filesystem (formerly known as "overlayfs") was
++merged into mainline. This is another implementation of UnionMount as a
++separated filesystem. All the limitations and known problems which
++UnionMount are equally inherited to "overlay" filesystem.
++
++Unionfs has a longer history. When I started implementing a stackable
++filesystem (Aug 2005), it already existed. It has virtual super_block,
++inode, dentry and file objects and they have an array pointing lower
++same kind objects. After contributing many patches for Unionfs, I
++re-started my project AUFS (Jun 2006).
++
++In AUFS, the structure of filesystem resembles to Unionfs, but I
++implemented my own ideas, approaches and enhancements and it became
++totally different one.
++
++Comparing DM snapshot and fs based implementation
++- the number of bytes to be copied between devices is much smaller.
++- the type of filesystem must be one and only.
++- the fs must be writable, no readonly fs, even for the lower original
++ device. so the compression fs will not be usable. but if we use
++ loopback mount, we may address this issue.
++ for instance,
++ mount /cdrom/squashfs.img /sq
++ losetup /sq/ext2.img
++ losetup /somewhere/cow
++ dmsetup "snapshot /dev/loop0 /dev/loop1 ..."
++- it will be difficult (or needs more operations) to extract the
++ difference between the original device and COW.
++- DM snapshot-merge may help a lot when users try merging. in the
++ fs-layer union, users will use rsync(1).
++
++You may want to read my old paper "Filesystems in LiveCD"
++(http://aufs.sourceforge.net/aufs2/report/sq/sq.pdf).
++
++
++Several characters/aspects/persona of aufs
++----------------------------------------------------------------------
++
++Aufs has several characters, aspects or persona.
++1. a filesystem, callee of VFS helper
++2. sub-VFS, caller of VFS helper for branches
++3. a virtual filesystem which maintains persistent inode number
++4. reader/writer of files on branches such like an application
++
++1. Callee of VFS Helper
++As an ordinary linux filesystem, aufs is a callee of VFS. For instance,
++unlink(2) from an application reaches sys_unlink() kernel function and
++then vfs_unlink() is called. vfs_unlink() is one of VFS helper and it
++calls filesystem specific unlink operation. Actually aufs implements the
++unlink operation but it behaves like a redirector.
++
++2. Caller of VFS Helper for Branches
++aufs_unlink() passes the unlink request to the branch filesystem as if
++it were called from VFS. So the called unlink operation of the branch
++filesystem acts as usual. As a caller of VFS helper, aufs should handle
++every necessary pre/post operation for the branch filesystem.
++- acquire the lock for the parent dir on a branch
++- lookup in a branch
++- revalidate dentry on a branch
++- mnt_want_write() for a branch
++- vfs_unlink() for a branch
++- mnt_drop_write() for a branch
++- release the lock on a branch
++
++3. Persistent Inode Number
++One of the most important issue for a filesystem is to maintain inode
++numbers. This is particularly important to support exporting a
++filesystem via NFS. Aufs is a virtual filesystem which doesn't have a
++backend block device for its own. But some storage is necessary to
++keep and maintain the inode numbers. It may be a large space and may not
++suit to keep in memory. Aufs rents some space from its first writable
++branch filesystem (by default) and creates file(s) on it. These files
++are created by aufs internally and removed soon (currently) keeping
++opened.
++Note: Because these files are removed, they are totally gone after
++ unmounting aufs. It means the inode numbers are not persistent
++ across unmount or reboot. I have a plan to make them really
++ persistent which will be important for aufs on NFS server.
++
++4. Read/Write Files Internally (copy-on-write)
++Because a branch can be readonly, when you write a file on it, aufs will
++"copy-up" it to the upper writable branch internally. And then write the
++originally requested thing to the file. Generally kernel doesn't
++open/read/write file actively. In aufs, even a single write may cause a
++internal "file copy". This behaviour is very similar to cp(1) command.
++
++Some people may think it is better to pass such work to user space
++helper, instead of doing in kernel space. Actually I am still thinking
++about it. But currently I have implemented it in kernel space.
+diff -urNp -x '*.orig' linux-4.9/Documentation/filesystems/aufs/design/02struct.txt linux-4.9/Documentation/filesystems/aufs/design/02struct.txt
+--- linux-4.9/Documentation/filesystems/aufs/design/02struct.txt 1970-01-01 01:00:00.000000000 +0100
++++ linux-4.9/Documentation/filesystems/aufs/design/02struct.txt 2021-02-24 16:15:09.518240088 +0100
+@@ -0,0 +1,258 @@
++
++# Copyright (C) 2005-2018 Junjiro R. Okajima
++#
++# This program is free software; you can redistribute it and/or modify
++# it under the terms of the GNU General Public License as published by
++# the Free Software Foundation; either version 2 of the License, or
++# (at your option) any later version.
++#
++# This program is distributed in the hope that it will be useful,
++# but WITHOUT ANY WARRANTY; without even the implied warranty of
++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
++# GNU General Public License for more details.
++#
++# You should have received a copy of the GNU General Public License
++# along with this program. If not, see <http://www.gnu.org/licenses/>.
++
++Basic Aufs Internal Structure
++
++Superblock/Inode/Dentry/File Objects
++----------------------------------------------------------------------
++As like an ordinary filesystem, aufs has its own
++superblock/inode/dentry/file objects. All these objects have a
++dynamically allocated array and store the same kind of pointers to the
++lower filesystem, branch.
++For example, when you build a union with one readwrite branch and one
++readonly, mounted /au, /rw and /ro respectively.
++- /au = /rw + /ro
++- /ro/fileA exists but /rw/fileA
++
++Aufs lookup operation finds /ro/fileA and gets dentry for that. These
++pointers are stored in a aufs dentry. The array in aufs dentry will be,
++- [0] = NULL (because /rw/fileA doesn't exist)
++- [1] = /ro/fileA
++
++This style of an array is essentially same to the aufs
++superblock/inode/dentry/file objects.
++
++Because aufs supports manipulating branches, ie. add/delete/change
++branches dynamically, these objects has its own generation. When
++branches are changed, the generation in aufs superblock is
++incremented. And a generation in other object are compared when it is
++accessed. When a generation in other objects are obsoleted, aufs
++refreshes the internal array.
++
++
++Superblock
++----------------------------------------------------------------------
++Additionally aufs superblock has some data for policies to select one
++among multiple writable branches, XIB files, pseudo-links and kobject.
++See below in detail.
++About the policies which supports copy-down a directory, see
++wbr_policy.txt too.
++
++
++Branch and XINO(External Inode Number Translation Table)
++----------------------------------------------------------------------
++Every branch has its own xino (external inode number translation table)
++file. The xino file is created and unlinked by aufs internally. When two
++members of a union exist on the same filesystem, they share the single
++xino file.
++The struct of a xino file is simple, just a sequence of aufs inode
++numbers which is indexed by the lower inode number.
++In the above sample, assume the inode number of /ro/fileA is i111 and
++aufs assigns the inode number i999 for fileA. Then aufs writes 999 as
++4(8) bytes at 111 * 4(8) bytes offset in the xino file.
++
++When the inode numbers are not contiguous, the xino file will be sparse
++which has a hole in it and doesn't consume as much disk space as it
++might appear. If your branch filesystem consumes disk space for such
++holes, then you should specify 'xino=' option at mounting aufs.
++
++Aufs has a mount option to free the disk blocks for such holes in XINO
++files on tmpfs or ramdisk. But it is not so effective actually. If you
++meet a problem of disk shortage due to XINO files, then you should try
++"tmpfs-ino.patch" (and "vfs-ino.patch" too) in aufs4-standalone.git.
++The patch localizes the assignment inumbers per tmpfs-mount and avoid
++the holes in XINO files.
++
++Also a writable branch has three kinds of "whiteout bases". All these
++are existed when the branch is joined to aufs, and their names are
++whiteout-ed doubly, so that users will never see their names in aufs
++hierarchy.
++1. a regular file which will be hardlinked to all whiteouts.
++2. a directory to store a pseudo-link.
++3. a directory to store an "orphan"-ed file temporary.
++
++1. Whiteout Base
++ When you remove a file on a readonly branch, aufs handles it as a
++ logical deletion and creates a whiteout on the upper writable branch
++ as a hardlink of this file in order not to consume inode on the
++ writable branch.
++2. Pseudo-link Dir
++ See below, Pseudo-link.
++3. Step-Parent Dir
++ When "fileC" exists on the lower readonly branch only and it is
++ opened and removed with its parent dir, and then user writes
++ something into it, then aufs copies-up fileC to this
++ directory. Because there is no other dir to store fileC. After
++ creating a file under this dir, the file is unlinked.
++
++Because aufs supports manipulating branches, ie. add/delete/change
++dynamically, a branch has its own id. When the branch order changes,
++aufs finds the new index by searching the branch id.
++
++
++Pseudo-link
++----------------------------------------------------------------------
++Assume "fileA" exists on the lower readonly branch only and it is
++hardlinked to "fileB" on the branch. When you write something to fileA,
++aufs copies-up it to the upper writable branch. Additionally aufs
++creates a hardlink under the Pseudo-link Directory of the writable
++branch. The inode of a pseudo-link is kept in aufs super_block as a
++simple list. If fileB is read after unlinking fileA, aufs returns
++filedata from the pseudo-link instead of the lower readonly
++branch. Because the pseudo-link is based upon the inode, to keep the
++inode number by xino (see above) is essentially necessary.
++
++All the hardlinks under the Pseudo-link Directory of the writable branch
++should be restored in a proper location later. Aufs provides a utility
++to do this. The userspace helpers executed at remounting and unmounting
++aufs by default.
++During this utility is running, it puts aufs into the pseudo-link
++maintenance mode. In this mode, only the process which began the
++maintenance mode (and its child processes) is allowed to operate in
++aufs. Some other processes which are not related to the pseudo-link will
++be allowed to run too, but the rest have to return an error or wait
++until the maintenance mode ends. If a process already acquires an inode
++mutex (in VFS), it has to return an error.
++
++
++XIB(external inode number bitmap)
++----------------------------------------------------------------------
++Addition to the xino file per a branch, aufs has an external inode number
++bitmap in a superblock object. It is also an internal file such like a
++xino file.
++It is a simple bitmap to mark whether the aufs inode number is in-use or
++not.
++To reduce the file I/O, aufs prepares a single memory page to cache xib.
++
++As well as XINO files, aufs has a feature to truncate/refresh XIB to
++reduce the number of consumed disk blocks for these files.
++
++
++Virtual or Vertical Dir, and Readdir in Userspace
++----------------------------------------------------------------------
++In order to support multiple layers (branches), aufs readdir operation
++constructs a virtual dir block on memory. For readdir, aufs calls
++vfs_readdir() internally for each dir on branches, merges their entries
++with eliminating the whiteout-ed ones, and sets it to file (dir)
++object. So the file object has its entry list until it is closed. The
++entry list will be updated when the file position is zero and becomes
++obsoleted. This decision is made in aufs automatically.
++
++The dynamically allocated memory block for the name of entries has a
++unit of 512 bytes (by default) and stores the names contiguously (no
++padding). Another block for each entry is handled by kmem_cache too.
++During building dir blocks, aufs creates hash list and judging whether
++the entry is whiteouted by its upper branch or already listed.
++The merged result is cached in the corresponding inode object and
++maintained by a customizable life-time option.
++
++Some people may call it can be a security hole or invite DoS attack
++since the opened and once readdir-ed dir (file object) holds its entry
++list and becomes a pressure for system memory. But I'd say it is similar
++to files under /proc or /sys. The virtual files in them also holds a
++memory page (generally) while they are opened. When an idea to reduce
++memory for them is introduced, it will be applied to aufs too.
++For those who really hate this situation, I've developed readdir(3)
++library which operates this merging in userspace. You just need to set
++LD_PRELOAD environment variable, and aufs will not consume no memory in
++kernel space for readdir(3).
++
++
++Workqueue
++----------------------------------------------------------------------
++Aufs sometimes requires privilege access to a branch. For instance,
++in copy-up/down operation. When a user process is going to make changes
++to a file which exists in the lower readonly branch only, and the mode
++of one of ancestor directories may not be writable by a user
++process. Here aufs copy-up the file with its ancestors and they may
++require privilege to set its owner/group/mode/etc.
++This is a typical case of a application character of aufs (see
++Introduction).
++
++Aufs uses workqueue synchronously for this case. It creates its own
++workqueue. The workqueue is a kernel thread and has privilege. Aufs
++passes the request to call mkdir or write (for example), and wait for
++its completion. This approach solves a problem of a signal handler
++simply.
++If aufs didn't adopt the workqueue and changed the privilege of the
++process, then the process may receive the unexpected SIGXFSZ or other
++signals.
++
++Also aufs uses the system global workqueue ("events" kernel thread) too
++for asynchronous tasks, such like handling inotify/fsnotify, re-creating a
++whiteout base and etc. This is unrelated to a privilege.
++Most of aufs operation tries acquiring a rw_semaphore for aufs
++superblock at the beginning, at the same time waits for the completion
++of all queued asynchronous tasks.
++
++
++Whiteout
++----------------------------------------------------------------------
++The whiteout in aufs is very similar to Unionfs's. That is represented
++by its filename. UnionMount takes an approach of a file mode, but I am
++afraid several utilities (find(1) or something) will have to support it.
++
++Basically the whiteout represents "logical deletion" which stops aufs to
++lookup further, but also it represents "dir is opaque" which also stop
++further lookup.
++
++In aufs, rmdir(2) and rename(2) for dir uses whiteout alternatively.
++In order to make several functions in a single systemcall to be
++revertible, aufs adopts an approach to rename a directory to a temporary
++unique whiteouted name.
++For example, in rename(2) dir where the target dir already existed, aufs
++renames the target dir to a temporary unique whiteouted name before the
++actual rename on a branch, and then handles other actions (make it opaque,
++update the attributes, etc). If an error happens in these actions, aufs
++simply renames the whiteouted name back and returns an error. If all are
++succeeded, aufs registers a function to remove the whiteouted unique
++temporary name completely and asynchronously to the system global
++workqueue.
++
++
++Copy-up
++----------------------------------------------------------------------
++It is a well-known feature or concept.
++When user modifies a file on a readonly branch, aufs operate "copy-up"
++internally and makes change to the new file on the upper writable branch.
++When the trigger systemcall does not update the timestamps of the parent
++dir, aufs reverts it after copy-up.
++
++
++Move-down (aufs3.9 and later)
++----------------------------------------------------------------------
++"Copy-up" is one of the essential feature in aufs. It copies a file from
++the lower readonly branch to the upper writable branch when a user
++changes something about the file.
++"Move-down" is an opposite action of copy-up. Basically this action is
++ran manually instead of automatically and internally.
++For desgin and implementation, aufs has to consider these issues.
++- whiteout for the file may exist on the lower branch.
++- ancestor directories may not exist on the lower branch.
++- diropq for the ancestor directories may exist on the upper branch.
++- free space on the lower branch will reduce.
++- another access to the file may happen during moving-down, including
++ UDBA (see "Revalidate Dentry and UDBA").
++- the file should not be hard-linked nor pseudo-linked. they should be
++ handled by auplink utility later.
++
++Sometimes users want to move-down a file from the upper writable branch
++to the lower readonly or writable branch. For instance,
++- the free space of the upper writable branch is going to run out.
++- create a new intermediate branch between the upper and lower branch.
++- etc.
++
++For this purpose, use "aumvdown" command in aufs-util.git.
+diff -urNp -x '*.orig' linux-4.9/Documentation/filesystems/aufs/design/03atomic_open.txt linux-4.9/Documentation/filesystems/aufs/design/03atomic_open.txt
+--- linux-4.9/Documentation/filesystems/aufs/design/03atomic_open.txt 1970-01-01 01:00:00.000000000 +0100
++++ linux-4.9/Documentation/filesystems/aufs/design/03atomic_open.txt 2021-02-24 16:15:09.518240088 +0100
+@@ -0,0 +1,85 @@
++
++# Copyright (C) 2015-2018 Junjiro R. Okajima
++#
++# This program is free software; you can redistribute it and/or modify
++# it under the terms of the GNU General Public License as published by
++# the Free Software Foundation; either version 2 of the License, or
++# (at your option) any later version.
++#
++# This program is distributed in the hope that it will be useful,
++# but WITHOUT ANY WARRANTY; without even the implied warranty of
++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
++# GNU General Public License for more details.
++#
++# You should have received a copy of the GNU General Public License
++# along with this program. If not, see <http://www.gnu.org/licenses/>.
++
++Support for a branch who has its ->atomic_open()
++----------------------------------------------------------------------
++The filesystems who implement its ->atomic_open() are not majority. For
++example NFSv4 does, and aufs should call NFSv4 ->atomic_open,
++particularly for open(O_CREAT|O_EXCL, 0400) case. Other than
++->atomic_open(), NFSv4 returns an error for this open(2). While I am not
++sure whether all filesystems who have ->atomic_open() behave like this,
++but NFSv4 surely returns the error.
++
++In order to support ->atomic_open() for aufs, there are a few
++approaches.
++
++A. Introduce aufs_atomic_open()
++ - calls one of VFS:do_last(), lookup_open() or atomic_open() for
++ branch fs.
++B. Introduce aufs_atomic_open() calling create, open and chmod. this is
++ an aufs user Pip Cet's approach
++ - calls aufs_create(), VFS finish_open() and notify_change().
++ - pass fake-mode to finish_open(), and then correct the mode by
++ notify_change().
++C. Extend aufs_open() to call branch fs's ->atomic_open()
++ - no aufs_atomic_open().
++ - aufs_lookup() registers the TID to an aufs internal object.
++ - aufs_create() does nothing when the matching TID is registered, but
++ registers the mode.
++ - aufs_open() calls branch fs's ->atomic_open() when the matching
++ TID is registered.
++D. Extend aufs_open() to re-try branch fs's ->open() with superuser's
++ credential
++ - no aufs_atomic_open().
++ - aufs_create() registers the TID to an internal object. this info
++ represents "this process created this file just now."
++ - when aufs gets EACCES from branch fs's ->open(), then confirm the
++ registered TID and re-try open() with superuser's credential.
++
++Pros and cons for each approach.
++
++A.
++ - straightforward but highly depends upon VFS internal.
++ - the atomic behavaiour is kept.
++ - some of parameters such as nameidata are hard to reproduce for
++ branch fs.
++ - large overhead.
++B.
++ - easy to implement.
++ - the atomic behavaiour is lost.
++C.
++ - the atomic behavaiour is kept.
++ - dirty and tricky.
++ - VFS checks whether the file is created correctly after calling
++ ->create(), which means this approach doesn't work.
++D.
++ - easy to implement.
++ - the atomic behavaiour is lost.
++ - to open a file with superuser's credential and give it to a user
++ process is a bad idea, since the file object keeps the credential
++ in it. It may affect LSM or something. This approach doesn't work
++ either.
++
++The approach A is ideal, but it hard to implement. So here is a
++variation of A, which is to be implemented.
++
++A-1. Introduce aufs_atomic_open()
++ - calls branch fs ->atomic_open() if exists. otherwise calls
++ vfs_create() and finish_open().
++ - the demerit is that the several checks after branch fs
++ ->atomic_open() are lost. in the ordinary case, the checks are
++ done by VFS:do_last(), lookup_open() and atomic_open(). some can
++ be implemented in aufs, but not all I am afraid.
+diff -urNp -x '*.orig' linux-4.9/Documentation/filesystems/aufs/design/03lookup.txt linux-4.9/Documentation/filesystems/aufs/design/03lookup.txt
+--- linux-4.9/Documentation/filesystems/aufs/design/03lookup.txt 1970-01-01 01:00:00.000000000 +0100
++++ linux-4.9/Documentation/filesystems/aufs/design/03lookup.txt 2021-02-24 16:15:09.518240088 +0100
+@@ -0,0 +1,113 @@
++
++# Copyright (C) 2005-2018 Junjiro R. Okajima
++#
++# This program is free software; you can redistribute it and/or modify
++# it under the terms of the GNU General Public License as published by
++# the Free Software Foundation; either version 2 of the License, or
++# (at your option) any later version.
++#
++# This program is distributed in the hope that it will be useful,
++# but WITHOUT ANY WARRANTY; without even the implied warranty of
++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
++# GNU General Public License for more details.
++#
++# You should have received a copy of the GNU General Public License
++# along with this program. If not, see <http://www.gnu.org/licenses/>.
++
++Lookup in a Branch
++----------------------------------------------------------------------
++Since aufs has a character of sub-VFS (see Introduction), it operates
++lookup for branches as VFS does. It may be a heavy work. But almost all
++lookup operation in aufs is the simplest case, ie. lookup only an entry
++directly connected to its parent. Digging down the directory hierarchy
++is unnecessary. VFS has a function lookup_one_len() for that use, and
++aufs calls it.
++
++When a branch is a remote filesystem, aufs basically relies upon its
++->d_revalidate(), also aufs forces the hardest revalidate tests for
++them.
++For d_revalidate, aufs implements three levels of revalidate tests. See
++"Revalidate Dentry and UDBA" in detail.
++
++
++Test Only the Highest One for the Directory Permission (dirperm1 option)
++----------------------------------------------------------------------
++Let's try case study.
++- aufs has two branches, upper readwrite and lower readonly.
++ /au = /rw + /ro
++- "dirA" exists under /ro, but /rw. and its mode is 0700.
++- user invoked "chmod a+rx /au/dirA"
++- the internal copy-up is activated and "/rw/dirA" is created and its
++ permission bits are set to world readable.
++- then "/au/dirA" becomes world readable?
++
++In this case, /ro/dirA is still 0700 since it exists in readonly branch,
++or it may be a natively readonly filesystem. If aufs respects the lower
++branch, it should not respond readdir request from other users. But user
++allowed it by chmod. Should really aufs rejects showing the entries
++under /ro/dirA?
++
++To be honest, I don't have a good solution for this case. So aufs
++implements 'dirperm1' and 'nodirperm1' mount options, and leave it to
++users.
++When dirperm1 is specified, aufs checks only the highest one for the
++directory permission, and shows the entries. Otherwise, as usual, checks
++every dir existing on all branches and rejects the request.
++
++As a side effect, dirperm1 option improves the performance of aufs
++because the number of permission check is reduced when the number of
++branch is many.
++
++
++Revalidate Dentry and UDBA (User's Direct Branch Access)
++----------------------------------------------------------------------
++Generally VFS helpers re-validate a dentry as a part of lookup.
++0. digging down the directory hierarchy.
++1. lock the parent dir by its i_mutex.
++2. lookup the final (child) entry.
++3. revalidate it.
++4. call the actual operation (create, unlink, etc.)
++5. unlock the parent dir
++
++If the filesystem implements its ->d_revalidate() (step 3), then it is
++called. Actually aufs implements it and checks the dentry on a branch is
++still valid.
++But it is not enough. Because aufs has to release the lock for the
++parent dir on a branch at the end of ->lookup() (step 2) and
++->d_revalidate() (step 3) while the i_mutex of the aufs dir is still
++held by VFS.
++If the file on a branch is changed directly, eg. bypassing aufs, after
++aufs released the lock, then the subsequent operation may cause
++something unpleasant result.
++
++This situation is a result of VFS architecture, ->lookup() and
++->d_revalidate() is separated. But I never say it is wrong. It is a good
++design from VFS's point of view. It is just not suitable for sub-VFS
++character in aufs.