Skip to content

feat!: files transfer in 1 MiB chunks - #369

Open
llogen wants to merge 2 commits into
mainfrom
feat/chunked-file-transfer
Open

llogen wants to merge 2 commits into
mainfrom
feat/chunked-file-transfer

Conversation

@llogen

@llogen llogen commented Jul 24, 2026 •

Copy link
Copy Markdown
Contributor

Files move between dutctl and dutagent in 1 MiB chunks, and module output and console input keep flowing during a transfer.
The old File message held the whole file in memory on both ends and blocked the Run stream until it was sent.
This breaks the RPC protocol and the module API, so dutagent and dutctl must be upgraded together.

@llogen
llogen marked this pull request as draft July 27, 2026 06:33
@llogen
llogen force-pushed the feat/chunked-file-transfer branch from ea122c3 to 54cdde7 Compare July 27, 2026 08:00
@llogen
llogen marked this pull request as ready for review July 27, 2026 08:37
@llogen
llogen force-pushed the feat/chunked-file-transfer branch from 54cdde7 to 8babdbe Compare July 30, 2026 08:33
@jenstopp jenstopp changed the title feat: chunked file transfer protocol Chunked file transfer Aug 19, 2026
@llogen llogen changed the title Chunked file transfer feat!: files transfer in 1 MiB chunks Nov 3, 2026
@llogen
llogen force-pushed the feat/chunked-file-transfer branch from 8babdbe to f4e0c1d Compare November 3, 2026 10:50
@llogen
llogen force-pushed the feat/chunked-file-transfer branch 2 times, most recently from bdceb4f to 21122f3 Compare November 5, 2026 10:46
Files move between client and agent in acknowledged chunks instead of
one message, so memory stays bounded and module output keeps flowing
during a transfer.

BREAKING CHANGE: FileTransferRequest, FileChunk and FileTransferResponse
replace the File and FileRequest messages on the Run stream, so dutagent
and dutctl must be upgraded together. In module.Session, RequestFile and
SendFile take a context, and SendFile takes the file size and closes the
reader once the transfer ends.

Signed-off-by: llogen <christoph.lange@blindspot.software>
ADR-0035 records the chunked file transfer over the Run stream, which
replaces the file-transfer points of ADR-0003.

Signed-off-by: llogen <christoph.lange@blindspot.software>
@llogen
llogen force-pushed the feat/chunked-file-transfer branch from 21122f3 to 9669658 Compare November 5, 2026 11:15

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant