Repository navigation
Closed
Labels
flaky-testIssues and PRs involving tests that fail intermittently in CI.Issues and PRs involving tests that fail intermittently in CI.httpIssues and PRs related to the http subsystem.Issues and PRs related to the http subsystem.
Description
- Version: 13.0.0-pre (master branch)
- Platform: SmartOS
- Subsystem: http
https://ci.nodejs.org/job/node-test-commit-smartos/nodes=smartos18-64/27655/console
04:56:09 not ok 965 parallel/test-http-server-keepalive-req-gc
04:56:09 ---
04:56:09 duration_ms: 0.345
04:56:09 severity: fail
04:56:09 exitcode: 1
04:56:09 stack: |-
04:56:09 Mismatched <anonymous> function calls. Expected exactly 1, actual 0.
04:56:09 at Object.mustCall (/home/iojs/build/workspace/node-test-commit-smartos/nodes/smartos18-64/test/common/index.js:337:10)
04:56:09 at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-smartos/nodes/smartos18-64/test/parallel/test-http-server-keepalive-req-gc.js:18:36)
04:56:09 at Module._compile (internal/modules/cjs/loader.js:936:30)
04:56:09 at Object.Module._extensions..js (internal/modules/cjs/loader.js:947:10)
04:56:09 at Module.load (internal/modules/cjs/loader.js:789:32)
04:56:09 at Function.Module._load (internal/modules/cjs/loader.js:702:12)
04:56:09 at Function.Module.runMain (internal/modules/cjs/loader.js:999:10)
04:56:09 at internal/main/run_main_module.js:17:11
04:56:09 ...Others:
https://ci.nodejs.org/job/node-test-commit-smartos/nodes=smartos18-64/27653/console
https://ci.nodejs.org/job/node-test-commit-smartos/nodes=smartos18-64/27651/console
https://ci.nodejs.org/job/node-test-commit-smartos/nodes=smartos18-64/27649/console
https://ci.nodejs.org/job/node-test-commit-smartos/nodes=smartos18-64/27648/console
https://ci.nodejs.org/job/node-test-commit-smartos/nodes=smartos18-64/27647/console
Metadata
Metadata
Assignees
Labels
flaky-testIssues and PRs involving tests that fail intermittently in CI.Issues and PRs involving tests that fail intermittently in CI.httpIssues and PRs related to the http subsystem.Issues and PRs related to the http subsystem.
Activity
Trott commentedon Aug 27, 2019
@nodejs/platform-smartos @nodejs/build
cjihrig commentedon Aug 27, 2019
FWIW, this failure is definitely not specific to SmartOS. I just observed the same failures on macOS while developing a test for #29288.
[-]flaky test-http-server-keepalive-req-gc.js on SmartOS[/-][+]flaky test-http-server-keepalive-req-gc.js[/+]Trott commentedon Aug 27, 2019
I think @mcollina mentioned seeing it on macOS too. It's definitely occurring more often on SmartOS in CI, but that could very easily be more about (for example) resources that the hosts are provisioned with than OS....
addaleax commentedon Aug 27, 2019
For those who can reproduce locally (I am unable on Linux): Could you maybe add this and share the resulting snapshot?
process.on('exit', (code) => code !== 0 && require('v8').writeHeapSnapshot());Trott commentedon Aug 27, 2019
I'm able to reproduce reliably on macOS with this:
For other people trying to get a heap snapshot: Adding the above
exithandler didn't work, I think becausecommon.mustCall()is failing in anexithandler itself. So I added thisexithandler tomustCall()in thecommonmodule:addaleax commentedon Aug 27, 2019
@Trott Could you share the heap snapshot? And thanks for fixing that :)
Trott commentedon Aug 27, 2019
Yeah, trying to figure out where to upload them.
Trott commentedon Aug 27, 2019
@addaleax I've emailed you a link to one of the snapshot files. Let me know if you need more.
addaleax commentedon Aug 27, 2019
@Trott Okay, it looks like the req object is not part of the heap snapshot, meaning it has likely been gc’ed, but we didn’t get the notification – Does adding a dummy
setTimeout(() => {}, 100);after theglobal.gc()call help?addaleax commentedon Aug 27, 2019
Alternatively, what would be a good platform to debug this on in CI?
Trott commentedon Aug 27, 2019
No. Nor does changing
100to1000or even10000(although obviously that slows the test down a lot--but it still fails frequently).Trott commentedon Aug 27, 2019
I think I have a solution. PR coming momentarily.
Trott commentedon Aug 27, 2019
Proposed fix in #29347. Works for me locally.
test: fix flaky test-http-server-keepalive-req-gc
test: fix flaky test-http-server-keepalive-req-gc
test: fix flaky test-http-server-keepalive-req-gc
test: fix flaky test-http-server-keepalive-req-gc