Repository navigation
Revert "pull request #1407 from bstatcomp/matrix_cl_refactor" - #1428
Conversation
|
Trying to think of the longer term fix testing wise to catch these things earlier |
|
@serban-nicusor-toptal this keeps getting errors like the below on windows Do you know why windows would run out of mem? I think it could be because of too many parallel jobs being run at once. I slammed it down to 4 atm. If that works maybe we should change the jenkins setup so it only runs like half as many possible parallel jobs on windows or something |
|
That seemed to do the trick, though things did not break all the time. So its hard to say if its a false positive. 4 does seem a bit low. Mind changing that to half and we can use that for some time, if things break again we can think come back to it. |
|
(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 1.01) |
|
Do we need beefier machines for windows because of this? It currently has m5d.12xlarge (48 vCPU, 192 GB RAM). |
I think that is plenty enough, I would say its overkill on the RAM.
Is it possible its that same issue we had before with how much RAM Jenkins can use? That came up for one of the tests some time ago. What is it set at currently? Maybe we should try setting a larger limit? By default its 1GB, the test that failed was usign 3GB or something. |
|
It's set to 32GB on Jenkins ( java process) |
|
I dont really have an idea on how RAM usage scales with multiple jobs, but I guess its (RAM usage for single core) * jobs. Compiling on single core easily takes up a few GB of RAM. If we multiply that with 24 we would get over that limit. Can we try setting it to 64GB for a test? |
|
Give it a try, it's on 64 GB now. |
|
Thanks!! restarted the tests, lets see. |
|
It looks fine now. Can we check if this ran on a EC2 instance? |
|
@serban-nicusor-toptal can you tell me which GPUs we use on the Mac machine and which one is used on the Linux machine? Would probably help find the reason for the failures that lead to this PR. |
|
It's not running on EC2 now. I need to turn off our windows instance so EC2 can boot up. I will send you an email with the GPU info later when I get home. |
|
(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 1.03) |
|
@rok-cesnovar can you approve? |
|
Or should we bump this back up to |
|
Nah lets try with this for now. |
This reverts commit 6723f81, reversing
changes made to 6455b69.
Checklist
Math issue deduplicate matrix_cl code #1407
Copyright holder: Steve Bronder
the basic tests are passing
./runTests.py test/unit)make test-headers)make doxygen)make cpplint)the code is written in idiomatic C++ and changes are documented in the doxygen
the new changes are tested