Repository navigation
Description
node_filesystem_readonly reports writable filesystem as read-only after upgrade to 1.12.0/1.12.1
Bug report
After upgrading node_exporter from 1.11.x to 1.12.0 or 1.12.1, I am seeing multiple false positives for node_filesystem_readonly.
The most concerning example is /var/data, which is clearly mounted read-write on the host and accepts write operations, yet node_exporter reports it as read-only.
Environment
Docker Compose
node-exporter:
image: prom/node-exporter:${NODE_EXPORTER_VERSION}
user: "${NODE_EXPORTER_UID}:${NODE_EXPORTER_GID}"
container_name: node-exporter
restart: unless-stopped
network_mode: host
command:
- --path.rootfs=/host
- --web.listen-address=:9100
- --collector.textfile.directory=/textfile_collector
volumes:
- /:/host:ro,rslave
- ${NODE_EXPORTER_TEXTFILE_DIR:?NODE_EXPORTER_TEXTFILE_DIR must be set in .env}:/textfile_collector:roObserved metrics
Among others, I am seeing the following metric:
node_filesystem_readonly{
device="/dev/mapper/data--vg0-data--lv--0",
fstype="ext4",
mountpoint="/var/data"
} 1
There are also several entries with device_error="permission denied" and device_error="no such file or directory".
Additional verification
The filesystem is mounted as read-write on the host.
findmnt
findmnt -no SOURCE,TARGET,FSTYPE,OPTIONS /var/dataOutput:
/dev/mapper/data--vg0-data--lv--0 /var/data ext4 rw,relatime
mount
mount | grep ' /var/data 'Output:
/dev/mapper/data--vg0-data--lv--0 on /var/data type ext4 (rw,relatime)
The mount options clearly show rw.
Write test
I can successfully write to the filesystem:
touch /var/data/node_exporter_rw_test
echo test > /var/data/node_exporter_rw_test
cat /var/data/node_exporter_rw_test
rm -f /var/data/node_exporter_rw_testOutput:
test
All operations complete successfully.
Expected behavior
node_filesystem_readonly{mountpoint="/var/data"} 0
Actual behavior
node_filesystem_readonly{mountpoint="/var/data"} 1
Version comparison
| Version | Result |
|---|---|
| 1.11.x | Works as expected |
| 1.12.0 | Reports /var/data as read-only |
| 1.12.1 | Reports /var/data as read-only |
Related issue
I found issue #2506:
However, that issue appears to be related to the host root filesystem being bind-mounted read-only into the container, causing node_exporter to observe the filesystem as read-only from inside the container. 1
My situation appears different:
/var/datais mounted asrwon the host.- Write operations succeed.
- The issue only appears after upgrading from 1.11.x to 1.12.x.
- Multiple filesystems are now being reported as read-only or returning device errors.
Has anything changed in the filesystem collector between 1.11.x and 1.12.x that could affect how read-only status is determined for bind-mounted root filesystems in containers?
I am happy to provide additional debugging information if needed.
Activity
Merlijnv commentedon Aug 28, 2026
@locker95 what is the status on this issue?
Merlijnv commentedon Sep 15, 2026
@discordianfish @SuperQ I am waiting for the pullrequest to be merged is there an estimate when this wil be fixed?
Merlijnv commentedon Oct 1, 2026
@discordianfish @SuperQ Still waiting for this pr to be merged. Can't upgrade to newest version until it is fixed
muhammad-tariq-zaman commentedon Nov 16, 2026
Thanks for the detailed report — this matches the false positive described in #3758. With the setup in your compose file (
- /:/host:ro,rslavetogether with--path.rootfs=/host), every host mount carries a per-mountroflag inside the exporter's own mount namespace, even though the filesystem is writable on the host.One way to confirm this is the same false positive, rather than a genuinely read-only filesystem, is to look at the mount entry from inside the container in
/proc/self/mountinfo: for the affected mount, the per-mount options (field 6) will showro, while the super options (the last field) should showrwif the host filesystem is actually writable. A truly read-only filesystem would showroin the super options instead.Until #3758 is reviewed and merged, the workaround documented there is to run the container with
--pid=host, so the exporter reads the host's view from/proc/1/mountinfoinstead of its own namespace.Merlijnv commentedon Nov 16, 2026
Oh thanks for the workaround will try it out tomorrow.