ostree.sync + composefs failed
#1744
Comments
|
I tried to start 2 VMs with libvirtd, and add kernel argument nfs server: nfs client: |
|
is the error happenning when composefs is on for the server or the client ? or both ? |
That test is currently hitting issues when composefs is re-enabled: openshift#1744 While we look into it, let's denylist it so we get more composefs testing downstream. Note there's already another entry for that same test on s390x, but let's keep them separate since they seem to be separate issues.
|
For now I just denylisted it in #1735. |
|
Based on the above result, I thought |
Hit the error only when composefs is on for the server. Any idea about this? Thanks! |
It looks like this test case is trying to reuse the host root filesystem as a filesystem to export. This is a thing real world use cases aren't going to do in my opinion. But it should work (with and without composefs) to change the exported filesystem to be Alternatively, I think it'd be better to export a temporary tree, something like: mkdir /var/lib/test-exported-fs and then have the client validate that, etc. |
|
Although reading the code I don't see it trying to export The relevant nfs code looks to be this...I wonder if we need some debugging prints in here or so. |
Maybe use container like coreos/coreos-assembler#3922? Just think this could get more logs if failed, but no lucky.
Tried again, and check that can only mount (with composefs enabled) But can not mount different dirs except |
|
If add kernel argument And this is the Run kola command (using the latest scos as it includes the enablefs patch): |
Hum wait, so it's the opposite of what I was thinking, NFS is broken except mounting the rootfs? That's really strange. Ah I think I see...looking at Does it work if you set |
|
Offhand, how I would expect this to work nowadays is that the nfs server uses the equivalent of |
|
For NFSv4 you need to have something exported as fsid=0/fsid=root. This represents the root of the exported tree, so everything else needs to hang off of that. On a non-image mode (package mode?) setup, you don't have to explicitly specify it (and you haven't for years) - in which case rpc.mountd will construct a "pseudo root" and a "pseudo fs", where it will use the actual / as the root and it will fill in any path components necessary to be able to walk to the actual export (the root and intermediate path components are all read-only). But that only works if the root filesystem is exportable - hence the "Cannot export /, possibly unsupported filesystem or fsid= required" error message from rpc.mountd. So here's what I have on my NFS server that I'm using for testing on image mode. I have two filesystems I want to export: And here's my /etc/exports: /srv/export is my "root". Note that means that NFSv4 clients see that as "/" and need to use that when mounting, not "/srv/export"... i.e. this fails: [root@nfs ~]# mount -o v4.2 nfsd.bootc.test:/srv/export /mnt/t and this works: One wrinkle when you explicitly specify an fsid=0 is that NFSv3 clients have have a concept of a single root of the exported tree, so they see the exports differently: [root@nfs ~]# mount -o v3 nfsd.bootc.test:/ /mnt/t I see that overlayfs has an "nfs_export" mount option, which controls whether the fh_to_dentry export operation is enabled. Would it be possible for the image mode root fs to be mounted with "nfs_export=on"? That should make it unnecessary to do these fsid=0 shenanigans. |
I think so. But...here's the thing, we aren't actually trying to export that filesystem right? It's just being traversed (AIUI) in order to access the real exported directories which are ultimately held somewhere else (typically the default is the real "physical root" of e.g. xfs or whatever is backing I guess my question in a nutshell is: why do any intermediate filesystems in the exported paths defined in Again though I think we could enable nfs_export unconditionally given that per the docs it only conflicts with |
Correct - it's strictly for traversing to the "real" export. IOW it's only to make it unnecessary for an admin to have to explicitly specify an fsid=0 export (in fact I wouldn't expect anyone to explicitly export part of the overlayfs at all). At the risk of over-explaining I'll post a somewhat lengthy example below showing how this works on a non-image mode setup.
To expand on that code snippet from rpc.mountd that you pointed at in an earlier comment... when rpc.mountd calls nfsd_export() -> dump_to_cache(), it writes to /proc/net/rpc/nfsd.export/channel, which triggers svc_export_parse() to be called in the kernel. svc_export_parse() calls check_export(), which has this exportfs_can_decode_fh() just checks to see if filesystem in question has an fh_to_dentry() operation (which overlayfs doesn't have if it's mounted with "nfs_export=off"): You can see that dprintk above if you enable some debug logging at the rpc layer (e.g. "rpcdebug -m nfsd -s export"). For example, if I were to remove the fsid=0 entry from my /etc/exports, do an "exportfs -r", and then try to mount again from the client, here's what I would see in the log on the server: Regarding openat2(how=RESOLVE_IN_ROOT),
Finally, a rather lengthy example to hopefully illustrate how the pseudofs works in NFSv4 (if you stop reading here, I don't blame you). I'm using a standard (non-image mode) NFS server, with two filesystems mounted somewhat deep in the directory tree. Here are the fstab entries: And here are my /etc/exports entries. Note I'm not specifying an fsid anywhere: Furthermore, at each level in that directory hiearchy I've created a file, just to illustrate that the client see's only the path components that are necessary in order to traverse to the "real" export: Let's mount it from a client (again on image mode this would fail, assuming that everything leading up to the real export was on overlayfs): Now let's mount from / and see what we can see: At each level, my client can only see the path components necessary to get to the actual root. Furthermore, it cannot write to any of those intermediate directories until I get to the actual export: Finally, the client cannot see any of the files that I had previously created (on the server side) within that directory structure leading up to the actual export: I can look at /proc/net/rpc/nfsd.export/content to see exactly what's in the export cache in the kernel. For example, if I want to see the pseudofs exports that were constructed by rpc.mountd: Or I can dump the entire cache. Note all the entries with empty export options i.e. "*()". That's basically an indication that that path is not exported to anyone. |
See #1735 (comment)
Seems client can not mount nfs server shared directories when
composefsis enabled.The text was updated successfully, but these errors were encountered: