Skip to content

pytest paths filter omits data/traits/**, so corpus tests never run on data-only PRs — 13 records drifted off the round-trip invariant #554

Closed

Description

@realmarcin

What is broken

tests/test_write_validated_round_trip.py::test_the_whole_corpus_round_trips fails on main (verified at 3316e80, clean checkout):

AssertionError: 13 record(s) no longer round-trip through write_validated_trait; first few:
['data/traits/environment/facultatively_aerobic.yaml', 'data/traits/environment/microaerotolerant.yaml',
 'data/traits/environment/nacl_delta_high.yaml', 'data/traits/environment/stenohaline.yaml',
 'data/traits/environment/thermotolerant.yaml']

The 13 are records touched by today's canonical-example and graph-connectivity tranches (#544–#552). They were written in a form write_validated_trait would not emit — the difference is purely line wrapping, e.g. thermotolerant.yaml:

-  note: Type strain CM3 (DSM 14292; LMG 21284) was described as thermotolerant,
-    with an approximately 47 degrees C optimum and a 55 degrees C maximum. The source
+  note: Type strain CM3 (DSM 14292; LMG 21284) was described as thermotolerant, with
+    an approximately 47 degrees C optimum and a 55 degrees C maximum. The source published

No data differs. #322/#344 normalised the whole corpus precisely so this invariant could be enforced; it has now drifted back for these records.

Why CI did not catch it

.github/workflows/pytest.yaml's paths filter is:

- "scripts/**"
- "src/traitmech/**"
- "tests/**"
- "pyproject.toml"
- ".github/workflows/pytest.yaml"
- "research/**"

There is no data/traits/**. Every canonical-example tranche is a data-only PR, so pytest never ran on any of them — confirmed on #551, whose only checks were canonical-example-taxonomy, history, pr-sanity, qc, validate-strict, vendored-sync. The corpus round-trip test reads data/traits/ directly, exactly like the research/** case already called out in that file's own comment (#250).

This is the same defect class as #184/#200/#250/#252 and as the DO_NOT_WORK.md gap fixed in #511: a filter narrower than what the job reads. audit-qc-paths guards the qc workflow this way but nothing guards pytest.

Fix

  1. Add data/traits/** to both the pull_request and push filters in pytest.yaml, with a comment naming the test that reads it.
  2. Re-normalise the 13 records through the emitter (formatting only, per the Normalise the corpus so the round-trip claim becomes enforceable (#322) #344 precedent: two-pass, semantic verification, no curation events).
  3. Consider extending the audit-qc-paths idea to pytest so the next added corpus-reading test cannot silently stop running.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions