Bug 204409

Summary: [EWS] iOS layout-tests fails with 30+ failures on re-run and on clean-tree
Product: WebKit Reporter: Aakash Jain <aakash_jain>
Component: New BugsAssignee: Jonathan Bedard <jbedard>
Status: RESOLVED FIXED    
Severity: Normal CC: ap, commit-queue, jbedard, simon.fraser, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: Other   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=203792
https://bugs.webkit.org/show_bug.cgi?id=204415
https://bugs.webkit.org/show_bug.cgi?id=204416
https://bugs.webkit.org/show_bug.cgi?id=204417
Attachments:
Description Flags
Patch none

Aakash Jain
Reported 2019-11-20 05:14:37 PST
If the layout-test finds any test failure, we re-run the tests (to rule-out flakiness) and do clean-tree run (to check pre-existing failures). on iOS, recently the re-runs and clean-tree runs have frequently started failing with 30+ failures. e.g.: https://ews-build.webkit.org/#/builders/24/builds/5262 https://ews-build.webkit.org/#/builders/24/builds/5260 https://ews-build.webkit.org/#/builders/24/builds/5259 https://ews-build.webkit.org/#/builders/24/builds/5256 https://ews-build.webkit.org/#/builders/24/builds/5251 https://ews-build.webkit.org/#/builders/24/builds/5077 https://ews-build.webkit.org/#/builders/24/builds/4981 https://ews-build.webkit.org/#/builders/24/builds/4779 This behavior is somewhat flaky. For e.g. in https://ews-build.webkit.org/#/builders/24/builds/5261 run-layout-tests-without-patch worked, and in https://ews-build.webkit.org/#/builders/24/builds/5254 re-run-layout-tests worked.
Attachments
Patch (4.87 KB, patch)
2019-11-20 11:13 PST, Jonathan Bedard 		no flags
DetailsFormatted DiffDiff
View All Add attachment proposed patch, testcase, etc.
Aakash Jain
Comment 1 2019-11-20 05:15:04 PST
This issue stalls iOS WK2 queue whenever there is any failure on trunk.
Jonathan Bedard
Comment 2 2019-11-20 08:59:53 PST
I don't understand how clean-tree results are somehow more flakey than the test runs with the patch.
Aakash Jain
Comment 3 2019-11-20 09:06:50 PST
This seems like another fallout from Catalina update on iOS bots. We updated the bots around Oct 30, and we started seeing 30+ layout-test failure in https://bugs.webkit.org/show_bug.cgi?id=203792 That issue was fixed on Nov 10. I believe this issue has been happening since then.
Jonathan Bedard
Comment 4 2019-11-20 10:27:14 PST
I bet that running kill-old-processes between runs would help. As Alexey pointed out, this is probably a bug of some sort, so I just landed <https://trac.webkit.org/r252691> to help us figure out what the problem is.
Jonathan Bedard
Comment 5 2019-11-20 11:13:58 PST
Created attachment 383974 [details] Patch
Jonathan Bedard
Comment 6 2019-11-20 11:56:00 PST
Comment on attachment 383974 [details] Patch Now that we have some logging from <https://trac.webkit.org/r252691>, it seems to be simulator and https. I'll work on fixes for those problems independently, but as I mention offline to Aakash: "I think the kill-old-processes stuff needs to land. If nothing else, it's possible that a patch which fails to clean up resources appropriately could cause failures in the clean run and cause us to mark the patch as 'good' even though it is demonstrably not"
Jonathan Bedard
Comment 7 2019-11-20 12:14:28 PST
Committed r252700: <https://trac.webkit.org/changeset/252700>
Jonathan Bedard
Comment 8 2019-11-20 12:15:09 PST
Comment on attachment 383974 [details] Patch Landed manually, removing flags.
Radar WebKit Bug Importer
Comment 9 2019-11-20 12:15:19 PST
<rdar://problem/57368402>
Radar WebKit Bug Importer
Comment 10 2019-11-20 12:15:25 PST
<rdar://problem/57368404>
Note You need to log in before you can comment on or make changes to this bug.