Fuse testsuite harness

[Abrar Quazi (AbrarQuazi)]

Testing out the xfstests and pjd-fstest testing-suite harness against FUSE, and have CI running docker containers for those tests.

[Abrar Quazi (AbrarQuazi)]

Currently have both xfstests and pjdfstests, but we may want to include only one.

Adapting xfstests to the CAS world might be more tricky as there is more mount/unmount tests there, and we would need to somehow save state in our mount.fuse helper. For pjdfstests, we are only mounting once and then running the tests

[Abrar Quazi (AbrarQuazi)]

An AI summary of the failed pjdstest:

• Files=21, Tests=469
• Result: FAIL
• Failing files: chmod/05.t, chown/05.t, link/06.t, link/07.t, open/00.t, open/06.t, open/07.t

High-level categories

1. Ownership and credential propagation is wrong

Most failures point at the same core problem: files and directories created through Wake FUSE are behaving as if they are owned by the daemon/backing filesystem user, not by the requesting pjdfstest user.

Why this is the most likely root cause:

• The pjdfstest mount is created with allow_other,default_permissions, so the kernel is doing normal Unix permission checks before or alongside FUSE operations.
• wakefuse_getattr returns raw backing inode metadata from fstatat, including st_uid and st_gid, without translating them to the requester's identity model.
• wakefuse_create and wakefuse_mkdir create backing objects with plain openat/mkdirat; they do not apply the requesting uid/gid to the new inode.
• The payload process runs in a user namespace that maps requested test ids like 65534 to the daemon's real uid/gid, so the test process identity and the backing inode identity can diverge.

Code paths that line up with this:

• test-suite/run-pjdfstest.sh: mount uses default_permissions
• test-suite/mount.fuse: forwards allow_other/default_permissions to the real FUSE mount
• tools/fuse-waked/main.cpp: wakefuse_getattr, wakefuse_create, wakefuse_mkdir, wakefuse_chmod, wakefuse_chown, wakefuse_open
• src/wakefs/namespace.cpp: user-namespace id mapping

2. Hard-link failures are probably a secondary effect of the ownership bug

The link failures are consistent with Linux returning EPERM for an unprivileged hard-link attempt when the source file appears to be owned by someone else. With fs.protected_hardlinks=1, this happens before the filesystem reaches the expected EACCES cases in some scenarios.

That matches the observed pattern:

• a link that should succeed fails with EPERM
• the cleanup unlink then fails with ENOENT because the link was never created
• tests expecting EACCES instead get EPERM

Failed tests by file

File	Failed tests	Category	Likely reason
tests/chmod/05.t	5-6, 10-11	Ownership/permission propagation	The test user 65534:65534 should be able to chmod its own file. The file appears to remain owned by 0:0, so chmod returns EPERM and the mode never changes from 0644.
tests/chown/05.t	5-6, 11-15	Ownership/permission propagation	The test expects a non-root owner to change group within its allowed groups and for stat to reflect the result. Instead the inode still reports 0:0, so chown/lchown return EPERM and follow-up stat checks fail.
tests/link/06.t	7-8	Hard-link / errno skew	A link that should succeed as user 65534 fails with EPERM, then the cleanup unlink sees ENOENT. Most likely the source file looks root-owned, triggering Linux hard-link protection rather than the expected normal success path.
tests/link/07.t	7-8, 10, 12	Hard-link / errno skew	Same underlying issue as link/06.t. Tests that expect either success or EACCES instead hit EPERM, which is consistent with protected-hardlink checks firing first because the inode ownership is wrong.
tests/open/00.t	19, 22, 26	Ownership propagation on O_CREAT	The test checks that open(..., O_CREAT, ...) assigns the new file's uid/gid from the effective user or parent directory. lstat reports 0,0 instead of the requested test identities, so creation ownership is wrong from the start.
tests/open/06.t	4, 6-8, 10-12, 14-16, 20, 24, 28-30, 32-34, 36-38, 40-41, 44-45, 48-49, 52-53, 56-57, 60-61, 66, 68-69, 71-72, 74-75, 79, 83, 87-88, 91-92, 95-96, 99-100, 103-104, 107-108, 113, 115, 117, 119, 121, 123, 125-142	Permission matrix collapse	This file is a big owner/group/other permission matrix for regular files, FIFOs, and directories. The repeated chmod -> EPERM and open -> wrong EACCES/0 pattern says the mode changes are not being applied by the intended owner, and subsequent access checks are being made against the wrong uid/gid and stale modes.
tests/open/07.t	4-5, 7, 9, 11, 13, 15, 17, 19, 21, 23	O_TRUNC permission handling	The first write-as-owner case already fails with EACCES, which means the file is not being treated as owned by the calling test uid. All later chmod-dependent `O_RDONLY

Per-file notes

tests/chmod/05.t

Failed cases:

• 5: chmod ... 0642 expected 0, got EPERM
• 6: stat ... mode expected 0642, got 0644
• 10: chmod ... 0420 expected 0, got EPERM
• 11: stat ... mode expected 0420, got 0644

Interpretation:

• The caller should own the file.
• The file likely does not appear owned by that caller once it comes back through getattr.
• Because the mode never changes, all follow-up stat checks remain at the original mode.

tests/chown/05.t

Failed cases:

• 5: chown ... -1 65533 expected 0, got EPERM
• 6: stat ... uid,gid expected 65534,65533, got 0,0
• 11: stat ... uid,gid expected 65534,65533, got 0,0
• 12: chown ... -1 65534 expected 0, got EPERM
• 13: stat ... uid,gid expected 65534,65534, got 0,0
• 14: lchown ... -1 65533 expected 0, got EPERM
• 15: stat ... uid,gid expected 65534,65533, got 0,0

Interpretation:

• The raw ownership reported back to userspace is wrong.
• Once ownership is wrong, non-root chown/lchown semantics are wrong too.

tests/link/06.t and tests/link/07.t

Interpretation:

• These tests depend on the source file being linkable by the unprivileged caller.
• If the file appears 0:0 instead of 65534:65534, Linux can reject link(2) with EPERM before the expected directory-permission outcome is observed.
• That explains both the unexpected EPERM and the follow-on ENOENT cleanup failure.

tests/open/00.t

Failed cases:

• 19: expected created file uid/gid 65534,65534, got 0,0
• 22: expected 65534,6553[34], got 0,0
• 26: expected 65533,6553[24], got 0,0

Interpretation:

• O_CREAT is creating the file with daemon/backing credentials.
• getattr then reports those backing credentials directly.

tests/open/06.t

Interpretation by subgroup:

• Regular-file failures:
  repeated owner/group/other chmod calls fail with EPERM, then open results match the unchanged root-owned mode instead of the requested one.
• FIFO failures:
  same issue as regular files, just surfaced through the FIFO permission matrix.
• Directory failures:
  directory chmod calls fail, so open(O_RDONLY) on directories keeps succeeding under 0755 when the test expects EACCES after mode changes.

This file is mostly symptom amplification of the same ownership bug.

tests/open/07.t

Interpretation:

• Test 4 already shows the synthetic owner cannot open the file for writing.
• Tests 5, 7, 9, 11, 13, 15, 17, 19, 21 are all failed chmod transitions.
• Test 23 then reports size 0 instead of 1 because the first owner-write case never succeeded.

Most likely fix areas

• Preserve request credentials on create/mkdir/mknod/symlink/link outcomes, or synthesize returned ownership consistently if backing ownership intentionally differs.
• Revisit whether default_permissions should be enabled for this test harness before uid/gid reporting is correct.
• Verify that getattr reports uid/gid/mode values that match the semantics pjdfstest expects for the calling user namespace.
• Re-test hard-link behavior after the ownership issue is fixed; the link failures may disappear without any dedicated hard-link code change.