Skip to content

Remove vendored nf-core utils subworkflows, use plugin imports - #241

Open
maxulysse wants to merge 21 commits into
devfrom
remove-nfcore-utils-subworkflows
Open

maxulysse wants to merge 21 commits into
devfrom
remove-nfcore-utils-subworkflows

Conversation

@maxulysse

@maxulysse maxulysse commented Jun 17, 2026 •

Copy link
Copy Markdown
Member

Replace the vendored subworkflows/nf-core/utils_* subworkflows with direct imports from plugin/nf-core-utils (0.7.0) and plugin/nf-schema, matching the pattern used by rnavar and createpanelrefs. Build the MultiQC methods tool citations on the fly from module meta.yml files instead of a hardcoded map.

Removed

  • subworkflows/nf-core/utils_nextflow_pipeline/: version display, params dump, conda check
  • subworkflows/nf-core/utils_nfcore_pipeline/: config validation, email, versions YAML
  • subworkflows/nf-core/utils_nfschema_plugin/: parameter validation, help display
  • toolReferencesMap(), toolReferencesText() and the local methodsDescriptionText() in subworkflows/local/utils_nfcore_seqinspector_pipeline/main.nf

Changes

  • subworkflows/local/utils_nfcore_seqinspector_pipeline/main.nf: imports come from the plugins, function calls inlined. subsampledMethodsText() keeps the Seqtk subsampling sentence in the methods description.
  • workflows/seqinspector.nf: paramsSummaryMultiqc, citationsOnTheFly and methodsDescriptionText come from plugin/nf-core-utils. Citations cover the tools found in the versions topic, plus MultiQC (always) and MultiQC_SAV (when selected), because both run after the topic is collected.
  • nextflow.config: pin nf-core-utils@0.7.0.
  • modules.json: remove references to the deleted subworkflows, add patch entries.
  • docs/CONTRIBUTING.md: document the citation setup and the module patch workflow.

Module meta.yml patches

citationsOnTheFly() reads the nested publication block of each tool in meta.yml. This block is not yet upstream for every module, so the following modules carry a patch (new, or regenerated to include meta.yml):

  • checkqc
  • fastqe
  • fastqscreen/fastqscreen (cites Wingett & Andrews 2018 instead of the Zenodo DOI)
  • multiqc
  • multiqcsav (restores the MultiQC entry and adds a multiqcsav tool entry)
  • samtools/faidx
  • samtools/index
  • seqkit/stats
  • sequali

These meta.yml changes need to be upstreamed to nf-core/modules.

Modules without a publication (bbmap/clumpify, chelae/trim, fastqc, fq/lint, picard/*, riker/multi, seqtk/sample, toulligqc, untar) fall back to their DOI or homepage.

Testing

  • nf-core pipelines lint: 0 failures
  • nf-test test tests/NovaSeq6000.nf.test tests/sample_size.nf.test tests/tools_multiqcsav.nf.test
  • Checked the generated MultiQC methods description on the test profile with --sample_size and --tools multiqcsav

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Jun 17, 2026 •

Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅

Posted for pipeline commit 5eb9074

+| ✅ 201 tests passed       |+
#| ❔   7 tests were ignored |#
Details

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 4.0.2
  • Run at 2026-06-18 09:00:42

Replace three subworkflows/nf-core/utils_* subworkflows with direct imports
from plugin/nf-core-utils and plugin/nf-schema, matching the pattern used
by rnavar and createpanelrefs pipelines.

- utils_nextflow_pipeline: version display, params dump, conda check
- utils_nfcore_pipeline: config validation, email, versions YAML
- utils_nfschema_plugin: parameter validation, help display

All functionality preserved in the local utils subworkflow using plugin
functions directly.
@maxulysse
maxulysse force-pushed the remove-nfcore-utils-subworkflows branch from 475e16c to 74fb6af Compare June 17, 2026 07:31
@maxulysse
maxulysse changed the base branch from master to dev June 17, 2026 07:31
@nf-core nf-core deleted a comment from github-actions Bot Jun 17, 2026
@maxulysse
maxulysse force-pushed the remove-nfcore-utils-subworkflows branch from 72dff97 to 39e1c25 Compare June 17, 2026 07:49
Move author/year/title/journal fields into a nested publication object
in meta.yml tool entries to align with the new meta-schema.json structure.

New meta.yml structure:
  tools:
    - samtools:
        doi: "10.1093/bioinformatics/btp352"
        publication:
          author: "Li Heng, et al."
          year: 2009
          title: "The Sequence Alignment/Map format and SAMtools"
          source: "Bioinformatics"

Changes:
- Moved author, year, title, journal into publication object
- Renamed journal to source (covers journals, conferences, workshops)
- Renamed authors (plural) to author (singular) for consistency
- Updated 16 meta.yml files across nf-core modules
@maxulysse maxulysse added this to the 1.2.0 milestone Jul 10, 2026
maxulysse and others added 2 commits October 31, 2026 12:55
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…bworkflows

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅

Posted for pipeline commit 4c14128

+| ✅ 204 tests passed       |+
#| ❔  10 tests were ignored |#
#| ❔   1 tests had warnings |#
Details

❔ Tests ignored:

  • files_exist - File is ignored: .github/workflows/linting_comment.yml
  • files_exist - File is ignored: tests/default.nf.test
  • nextflow_config - Config default ignored: params.fastq_screen_references
  • nf_test_content - nf_test_content
  • files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
  • files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
  • files_unchanged - File ignored due to lint config: assets/nf-core-seqinspector_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-seqinspector_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-seqinspector_logo_dark.png
  • multiqc_config - multiqc_config

❔ Tests fixed:

✅ Tests passed:

Run details

  • nf-core/tools version 4.2.1
  • Run at 2026-10-31 14:58:22

maxulysse and others added 2 commits October 31, 2026 15:38
The dev merge restored the hardcoded toolReferencesMap(). Build the
methods description citations from module meta.yml with
citationsOnTheFly() and methodsDescriptionText() from nf-core-utils,
keeping the Seqtk subsampling sentence.

Cite MultiQC always and MultiQC_SAV when selected, since both run after
the versions topic is collected. Add a multiqcsav tool entry to its
meta.yml.

Patch the modules whose meta.yml carries publication blocks not yet
upstream, add a publication to fastqscreen, fix the seqkit author
string, and document the setup in CONTRIBUTING.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Format pipeline Nextflow code and configs with
nextflow lint -format -harshil-alignment. Add trailing commas to
resourceLimits, the bam_qc test ext.args list and the citation function
calls so the formatter keeps them on multiple lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

❌ nf-test failed with latest Nextflow version

Note

Tests with Nextflow's latest version failed but it will not cause a CI workflow failure.
Please check if the failure is expected with newer (edge-)releases of Nextflow or if it needs fixing.

  • ❌ docker | latest-everything | Shard 4/7

See the full run for details.

These dependencies have no meta.yml tool entry. Keep the warning for any
other tool so new modules without citation metadata show up in
snapshots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant