Skip to content

Revert reverted matrix cl refactor - #1451

Merged
rok-cesnovar merged 17 commits into
developfrom
revert_reverted_matrix_cl_refactor
Nov 29, 2019
Merged

rok-cesnovar merged 17 commits into
developfrom
revert_reverted_matrix_cl_refactor

Conversation

@rok-cesnovar

Copy link
Copy Markdown
Member

Summary

This PR will attempt to revert back the revert in #1428

For now, this modifies the Jenkinsfile severely in order to only run the tests on the machine they failed on, have a faster turnaround and not clog the Jenkins workers. The first goal is to consistently repeat the failing tests on Jenkins and then find a fix.

Tests

Side Effects

Are there any side effects that we should be aware of?

Checklist

  • Math issue #

  • Copyright holder: Rok Češnovar

    The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
    - Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
    - Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

  • the basic tests are passing

    • unit tests pass (to run, use: ./runTests.py test/unit)
    • header checks pass, (make test-headers)
    • docs build, (make doxygen)
    • code passes the built in C++ standards checks (make cpplint)
  • the code is written in idiomatic C++ and changes are documented in the doxygen

  • the new changes are tested

@rok-cesnovar

Copy link
Copy Markdown
Member Author

@serban-nicusor-toptal is Jenkins acting up? I might just have a Jenkinsfile issue here, though I think something weird is going on. If you are already aware then sorry for the spam.

@serban-nicusor-toptal

Copy link
Copy Markdown
Contributor

@rok-cesnovar Jenkins has some issues, I'm working on fixing it up and bringing it online.

@rok-cesnovar

Copy link
Copy Markdown
Member Author

Thank you for the extremely quick reply! Good luck making Jenkins to obey :)

@serban-nicusor-toptal

serban-nicusor-toptal commented Nov 11, 2019 •

Copy link
Copy Markdown
Contributor

@rok-cesnovar Jenkins looks sane now. I've restarted your job but it failed w/o any error ... If this is an issue from Jenkins please let me know so we can sort it out.

@rok-cesnovar

Copy link
Copy Markdown
Member Author

Probably a goof on my part. Thanks!

@rok-cesnovar

Copy link
Copy Markdown
Member Author

I think the gelman-group-linux is down, judging from the errors:

gelman-group-linux was marked offline: Connection was broken: java.io.EOFException
	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2681)
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3156)
	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:862)
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358)
	at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
	at hudson.remoting.Command.readFrom(Command.java:140)
	at hudson.remoting.Command.readFrom(Command.java:126)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:36)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)

Caused: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:77)
Unable to create live FilePath for gelman-group-linux

@serban-nicusor-toptal

Copy link
Copy Markdown
Contributor

Hey Rok, I've noticed that while running it looses connection. This is possibly an issue because of an upgrade of some plugins, yay Jenkins. This should be solved shortly.

@rok-cesnovar

Copy link
Copy Markdown
Member Author

I feel your pain @serban-nicusor-toptal :/ Thanks for your quick responses, as always.

@serban-nicusor-toptal

Copy link
Copy Markdown
Contributor

@rok-cesnovar I've figured it out but now it fails because of a test, can you please check it out and let me know if it's an issue from Jenkins ? Thanks!

@rok-cesnovar

rok-cesnovar commented Nov 13, 2019 •

Copy link
Copy Markdown
Member Author

Its correct that it fails on this test. Its the reason for this PR because this is the exact machine it was failing on.

@SteveBronder

Copy link
Copy Markdown
Collaborator

!! Excellent Rok thank you for the diligence here!

@stan-buildbot

Copy link
Copy Markdown
Contributor

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 1.03)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.01)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 0.99)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 0.99)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 1.0)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.0)
(performance.compilation, 1.03)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 1.01)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 1.0)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 0.99)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 1.0)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 1.0)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 1.0)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 1.01)
Result: 1.00544912121
Commit hash: a04ea0e

@rok-cesnovar

Copy link
Copy Markdown
Member Author

@t4c1 By commenting out only this part of the refactor, the tests pass.

Would you know how does this one differs from the others or are others just missing similar tests? I know you are busy so no rush for the answer.

@t4c1 t4c1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for finding out exact line of code with a bug. That makes it easy to see what is wrong - perfect forwarding is missing here. I think the following changes should fix it.

Also there is no major difference between the overloads. Others are probably missing some tests.

Comment thread stan/math/opencl/copy.hpp Resolved
Comment thread stan/math/opencl/copy.hpp Resolved
Comment thread stan/math/opencl/copy.hpp Resolved
rok-cesnovar and others added 4 commits November 23, 2019 19:16
Co-Authored-By: Tadej Ciglarič <tadej.c@gmail.com>
Co-Authored-By: Tadej Ciglarič <tadej.c@gmail.com>
Co-Authored-By: Tadej Ciglarič <tadej.c@gmail.com>
@rok-cesnovar rok-cesnovar changed the title [WIP - DO NOT MERGE] Revert reverted matrix cl refactor Revert reverted matrix cl refactor Nov 23, 2019
@rok-cesnovar

Copy link
Copy Markdown
Member Author

Thanks! Added the fixes, lets see.

@stan-buildbot

Copy link
Copy Markdown
Contributor

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 0.99)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 0.98)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.0)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 0.96)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 1.01)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 1.01)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.0)
(performance.compilation, 1.02)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 1.01)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 0.96)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 1.0)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 0.99)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 0.98)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 0.97)
Result: 0.99271677726
Commit hash: 8296c77

@rok-cesnovar

Copy link
Copy Markdown
Member Author

Ok, this one seems to be squashed. Ready for review.

@t4c1 t4c1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question and a problem with a merge.

Comment thread stan/math/opencl/matrix_cl.hpp Outdated
Comment on lines +225 to +230
* @param A the vector of Eigen matrices
*
* @throw <code>std::invalid_argument</code> if the
* matrices do not have matching dimensions
* @throw <code>std::system_error</code> if the memory on the device could not
* be allocated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you lost part of this comment in a merge. Same with the changes in lines 377 and 391 of the original refactor, but I can't comment there since there are no changes there in your PR.

Comment thread Jenkinsfile Resolved
@stan-buildbot

Copy link
Copy Markdown
Contributor

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 0.99)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 1.0)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.0)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 0.99)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 1.02)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 1.0)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.01)
(performance.compilation, 1.03)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 1.0)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 0.97)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 1.02)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 1.0)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 0.97)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 1.01)
Result: 1.00022234241
Commit hash: 0473a22

@stan-buildbot

Copy link
Copy Markdown
Contributor

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 0.99)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 0.99)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.0)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 1.08)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 1.04)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 1.0)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.0)
(performance.compilation, 1.02)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 1.01)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 0.97)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 0.99)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 0.99)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 0.99)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 1.05)
Result: 1.00706876092
Commit hash: 0473a22

@stan-buildbot

Copy link
Copy Markdown
Contributor

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 0.96)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.0)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 1.0)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 1.0)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 1.0)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.0)
(performance.compilation, 1.03)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 0.99)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 1.01)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 1.07)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 1.0)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 0.98)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 1.01)
Result: 1.00224702074
Commit hash: 0473a22

@stan-buildbot

Copy link
Copy Markdown
Contributor

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 1.0)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.0)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 0.99)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 1.0)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 1.01)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.0)
(performance.compilation, 1.03)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 1.0)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 1.02)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 0.92)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 0.99)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 0.99)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 1.02)
Result: 0.99828388915
Commit hash: 0f16696

@rok-cesnovar
rok-cesnovar merged commit 51b27d6 into develop Nov 29, 2019
@rok-cesnovar
rok-cesnovar deleted the revert_reverted_matrix_cl_refactor branch November 29, 2019 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants