Opened 7 months ago
Closed 7 months ago
#18636 closed enhancement (fixed)
spidermonkey-115.3.0
| Reported by: | Owned by: | ||
|---|---|---|---|
| Priority: | normal | Milestone: | 12.1 |
| Component: | BOOK | Version: | git |
| Severity: | normal | Keywords: | |
| Cc: |
Description
Now available
Change History (8)
comment:2 by , 7 months ago
I guess the memory usage is related to the number of parallel jobs (i.e. the problem may be several tests using 1 or 2 GB memory are running simultaneously).
comment:3 by , 7 months ago
The test suite seems running with all available cores, passing -j x to make does not have effect.
comment:4 by , 7 months ago
The test named bug1782468-ptrdiff-veclen.js uses 3GB memory... And this test is ran 6 times (with different configurations) so the peak memory can be at least 18GB if the number of parallel jobs is 6 or more.
To make things worse, even if you run the test suite in a cgroup with only M cores, the test suite will still attempting to spawn N jobs where N is the number of all available cores on the system (it's obvious M < N). It's because the test suite uses Python 3 multiprocessing module for determine core numbers and this module does not know cgroup.
The only way to limit the number of parallel testing jobs is adding -jx into JITTEST_EXTRA_ARGS, like
make -C js/src check-jit-test JITTEST_EXTRA_ARGS="--timeout 300 -j3"
I'll note this in the book.
follow-up: 6 comment:5 by , 7 months ago
I've updated the note about memory usage at r12.0-287-g7655d92ffa and DO NOT REMOVE IT. I've provided enough reason for "why not" in the comment added in this commit.
comment:6 by , 7 months ago
Replying to Xi Ruoyao:
I've updated the note about memory usage at r12.0-287-g7655d92ffa and DO NOT REMOVE IT. I've provided enough reason for "why not" in the comment added in this commit.
Thanks for working on this. I've just come back to this machine with 16 cores enabled, and after doing my best to reduce the space in /tmp I've run the tests with 16 cores and it swapped heavily (about 13GB swap used) although there is a lot of free memory at the end.
The change looks clear. I had not realised the peak usages were so short-lived.
comment:7 by , 7 months ago
| Owner: | changed from to |
|---|---|
| Status: | new → assigned |
As always, no release notes for js in esr.
comment:8 by , 7 months ago
| Resolution: | → fixed |
|---|---|
| Status: | assigned → closed |
I was aware of the extreme memory requirements mentioned in the book, so not sure if I would be able to run the jit test (although that isn't measured). But I've got plenty of space on my zen2. I had the following results with only 4 cores online, using make -j4 for the build:
Build and DESTDIR 2.3 SBU, 3473 MB + 40 MB after removing 36 MB static lib (total 3.5 GB) check-jstests 1.7 SBU, add 29 MB including 5.5 MB for the test log check-jit-test 5.2 SBU, add 31 MB including 6.5 MB for the jitlog
But I cannot agree the comment about memory usage - with only 4 cores online, xfcewm, 4 xfce4terminals, two running firefoxes, the memory shown in 'top' was 17 GB free when I started, with 16 GB swap available. It quickly flushed something to show around 22GB free and >5GB used, plus the swap all free, and stayed within 0.2 GB of that for the rest of the jit test.
Test results: 8 failures in check-jstests, as expected, then jit-tests PASSED ALL.
For 4 cores, I suggest that there is no need to mention the amount of memory available, I had less than 6GB 'used' throughout the tests.
At this point I have not looked at what changed in spidermonkey between 115.2.1 and 113.0.
(edit - something had caused the page to scroll while typing, thought I'd lost the text, ended up leaving partial copy)