Skip to content

nfs: support all NFSv4 client and server operations - #898

Open
neoLsH wants to merge 1 commit into
prometheus:masterfrom
neoLsH:feat/nfs-v4-ops-699
Open

neoLsH wants to merge 1 commit into
prometheus:masterfrom
neoLsH:feat/nfs-v4-ops-699

Conversation

@neoLsH

@neoLsH neoLsH commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Fixes #699.

How the two lines are laid out

Both lines carry one counter per operation, and the position of a counter is
fixed by the kernel, not by which operations happen to be implemented.

proc4 in /proc/net/rpc/nfs is emitted by net/sunrpc/stats.c: it prints
the operation count, then one counter per index of vers->counts[]. For NFSv4
that array is nfs4_procedures[] in fs/nfs/nfs4xdr.c, which is built with
designated initializers [NFSPROC4_CLNT_##proc] = {...}. So the counter at
position n of the line belongs to the operation whose NFSPROC4_CLNT_* value
is n-1, gaps included. include/linux/nfs4.h states the contract:

To ensure that /proc/net/rpc/nfs remains correctly ordered, please append
only to this enum when adding new client operations.

proc4ops in /proc/net/rpc/nfsd works the same way from fs/nfsd/stats.c:
it prints LAST_NFS4_OP + 1 and then a counter for every operation number from
0 to LAST_NFS4_OP. Those are the stable enum nfs_opnum4 numbers from the
RFCs, so the server line is indexed by operation number rather than by an
append-only enum.

Two bugs in the client mapping

LAYOUTGET and GETDEVICEINFO were swapped. The parser read
GetDeviceInfo: v[45] and LayoutGet: v[46], but the enum has ordered
NFSPROC4_CLNT_LAYOUTGET before NFSPROC4_CLNT_GETDEVICEINFO since at least
v4.0 — I checked v4.0, v4.9, v5.4, v6.12 and master. ClientV4Stats in
nfs.go already declared the two fields in the correct order, so this looks
unintended rather than a deliberate reading of some older kernel. Both counters
have been reported under the wrong operation on every kernel since v4.0.

The ten operations after CLONE were not mapped at all. COPY,
OFFLOAD_CANCEL, LOOKUPP, LAYOUTERROR, COPY_NOTIFY and the five RFC 8276 xattr
and READ_PLUS operations were parsed off the line and then dropped. The padding
threshold moves from 59 to 69 to match, which is what v6.12 emits.

Server side

Only the NFSv4.0 operations were mapped, plus RELLOCKOWNER behind a
values > 39 special case. I added the NFSv4.1 operations (40–58), the
NFSv4.2 operations (59–71) and the RFC 8276 xattr operations (72–75), and
replaced that special case with the same padding the client parser already
uses. Padding subsumes it: a 39-counter line from an nfs v2.5.x kernel now
leaves RelLockOwner and everything after it at zero instead of needing a
separate code path.

What I did not do

Linux master has inserted NFSPROC4_CLNT_ZERO_RANGE at index 57, which shifts
LAYOUTSTATS, CLONE, COPY and the rest by one, and appended
NFSPROC4_CLNT_OFFLOAD_STATUS. That insertion breaks the append-only rule the
enum documents, and no released kernel has it — ZERO_RANGE is absent from
v5.10, v5.15, v6.1, v6.6 and v6.12. I left it alone rather than guess: today's
parser is equally wrong on those slots for such a kernel, so nothing here
regresses, and telling the two layouts apart needs more information than the
counter count on the line. Worth a separate issue if a kernel ships it.

Verification

Two new tests pin the layout down positionally. TestParseClientV4StatsOpOrder
builds a 69-counter line where the counter for the operation with enum value
n is n+1, then asserts each field reads its own operation's value;
TestParseV4OpsOpOrder does the same for 76 server operations. Before the fix
the first one fails with LayoutGet: got 46, want 45 and
GetDeviceInfo: got 45, want 46.

I also ran three reverse controls to confirm the tests actually bite: reverting
the swap reproduces exactly those two failures; changing the server padding
threshold from < 76 to < 40 panics with index out of range [73] with length 73 on the proc4ops 72 fixture; and shifting Seek from v[70] to
v[69] fails with Seek: got 69, want 70.

One existing expectation changed: the proc4 48 case in
TestNewNFSClientRPCStats asserted the swapped values, so it encoded the bug.
Whole-repo go test ./... passes and golangci-lint run ./nfs/... reports
0 issues.

Both the "proc4" and the "proc4ops" line carry one counter per operation,
in the order of the NFSPROC4_CLNT_* enum and of the nfs_opnum4 operation
numbers respectively. Two mistakes had crept into the client mapping:

  - LAYOUTGET and GETDEVICEINFO were swapped, so both counters were
    reported under the wrong operation. The enum has ordered LAYOUTGET
    before GETDEVICEINFO since at least v4.0.
  - The ten operations after CLONE were not mapped at all, so the counts
    for COPY, OFFLOAD_CANCEL, LOOKUPP, LAYOUTERROR, COPY_NOTIFY and the
    RFC 8276 xattr operations were dropped.

On the server side only the NFSv4.0 operations were mapped. Add the
NFSv4.1, NFSv4.2 and RFC 8276 xattr operations, and replace the special
case for the 39 counter line of v2.5.x kernels with the padding the
client parser already uses, so that kernels which emit fewer operations
than are mapped leave the remaining counters at zero.

Fixes prometheus#699

Signed-off-by: neoLsH <43921685+neoLsH@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RFE: Support all Linux kernel NFSv4 client and server RPC operations

1 participant