You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Restructure CTD simulation to use a fieldset per Instrument deployment #387
At present, like all other instruments, CTDs use a shared fieldset bounding all deployments across waypoints. This means there is a lot of redundant data fetched for this instrument which only requires the depth column in one place. Considering the depth coord is required, this can significantly balloon the ds size and risk memory related simulation crashes.
Therefore it probably preferable to allow CTDs to have their own (1D column x time) fieldsets. Complications which will arise from this though are that:
The simulation execution for all CTD deployments cannot occur in the same pset.execute(), and a loop through is required?
This would write an output file per deployment, whereas currently all CTDs are in one file. This is not necessarily a problem, and indeed could be interpreted as more realistic, but then need to consider other instruments and whether they should be consistent.
An alternative solution (much simpler to implement) could be to carefully consider the chunk-size of the CTD FieldSet. The default chunk size of a copernicusmarine dataset is quite large (time=31, depth=50, lat=512, lon=2048) and especially lon and lat could be much smaller for non-moving CTDs. This could potentially reduce the memory footprint massively?
Activity
erikvansebille commentedon Sep 18, 2026
An alternative solution (much simpler to implement) could be to carefully consider the chunk-size of the CTD FieldSet. The default chunk size of a copernicusmarine dataset is quite large (
time=31, depth=50, lat=512, lon=2048) and especiallylonandlatcould be much smaller for non-moving CTDs. This could potentially reduce the memory footprint massively?