Skip to content

Restructure CTD simulation to use a fieldset per Instrument deployment #387

Open

Description

@j-atkins

At present, like all other instruments, CTDs use a shared fieldset bounding all deployments across waypoints. This means there is a lot of redundant data fetched for this instrument which only requires the depth column in one place. Considering the depth coord is required, this can significantly balloon the ds size and risk memory related simulation crashes.

Therefore it probably preferable to allow CTDs to have their own (1D column x time) fieldsets. Complications which will arise from this though are that:

  • The simulation execution for all CTD deployments cannot occur in the same pset.execute(), and a loop through is required?
  • This would write an output file per deployment, whereas currently all CTDs are in one file. This is not necessarily a problem, and indeed could be interpreted as more realistic, but then need to consider other instruments and whether they should be consistent.

Activity

  1. erikvansebille commented on Sep 18, 2026

    @erikvansebille
    Member

    An alternative solution (much simpler to implement) could be to carefully consider the chunk-size of the CTD FieldSet. The default chunk size of a copernicusmarine dataset is quite large (time=31, depth=50, lat=512, lon=2048) and especially lon and lat could be much smaller for non-moving CTDs. This could potentially reduce the memory footprint massively?

  2. marked One fieldset per CTD cast #367 as a duplicate of this issue on Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions