Repository navigation
Add holder operation to kernel generator - #1865
Conversation
|
I named this operation |
…comp/math into cl_kernel_generator_holder
# Conflicts: # stan/math/opencl/kernel_generator.hpp
Jenkins Console Log Machine informationProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010CPU: G++: Clang: |
|
Is this equivalent to Eigen::Ref<>? The name is kind of weird, should it just be copy? |
Not even similar. This solves a different problem. Right now we can not have functions that use an argument multiple times and return an expression, as they would break for rvalues. Example: If x is a rvalue it will go out of scope when the function returns and multiplication will reference garbage memory as its arguments. Using const ref argument won't help. We could forward x as one of the arguments, but than we have the same problem with the other one. Forwarding an argument twice results in same problem. So this is the solution. There are two interfaces. The first one is more practical for local variables and the second one for arguments, such as in this example. 2:
I agree. I am open to suggestions once you understand what this is about.
No. |
|
This feels kind of onerous to put in the user space, should holder_cl be added inside of functions like By user space I mean any functions that are not in the kernel generator. Can the generator have a constructor for matrix_cl rvalues to handle this? |
|
Didn't we do something similar to this for |
That would be all kernel generator functions than. I don't think we can make it work that way. As I see it, holder must be explicitely constructed to make it work. When one kg operation is being constructed it has no knowledge of whether the same rvalue is also used in another function. An alternative would be to use dynamic allocations and
I don't think so. What exactley do you have in mind? |
|
I think Steve is thinking of this PR: #1421 |
|
In that PR we also moved some expressions to heap to extend their lifetime. But the reason for that was different. In matrix_cl we wanted to do asynchronous copy and we needed to ensure that the matrix we are copying exists untill copying finishes. Here we want to pass a rvalue to other function(s) multiple times. Instead we move it to heap and pass lvalue references to it. |
This is all very confusing. Correct where I'm wrong below. The problem is with this code template <typename T>
auto square(T&& x){
return elementwise_multiply(x, x);
}
If that's true, Does Eigen not run into this same issue? Could we have each matrix_cl / expression wait for all it's read/write events before calling the destructor? |
|
Exactley.
If a matrix_cl or expression is only used once, it can be moved into whatever expression it is used in. So the function return value will also contain the input to the function.
It does. We will need to do something very similar for many functions working with Eigen expressions, before they can return expressions.
At the time such a function returns no kernel is running yet - function is only constructing part of the expression that will be used in the kernel. |
|
@SteveBronder Is anything still unclear here? |
|
It seems @SteveBronder is too busy. @rok-cesnovar can you review this? |
|
Sure, though I am bad at naming stuff, so no new ideas on how to name this. Will look at this later today. |
I think I'm just confused on this. Does Eigen have the same holder type or what do they do for this? |
|
Eigen doesn't actually handle it, which has caused issues in the past (and kinda still is an issue), there's more information over in #1775 |
rok-cesnovar
left a comment
There was a problem hiding this comment.
Doxygen cleanup mostly. Otherwise looks good.
Element-wise functions they tend to implement elemet wise - similarly as we do in apply_scalar_unary, except they also add vectorized implementations. In some other cases they don't handle it at all - they evaluate the expression before returning. |
|
Also Eigen is written assuming C++03 standard, so they handle nothing about rvalues and moves. Instead they use simple heuristics to choose whether to store argument expressions by reference or value. Matrices and similar are stored by reference, expressions by value. |
|
@andrjohns Do you have any suggestion for a better name for this? |
rok-cesnovar
left a comment
There was a problem hiding this comment.
Changes look good, feel free to discuss the name a bit more. There are two outstanding issues on stan/cmdstan which will not allow any tests to pass right now.
…4.1 (tags/RELEASE_600/final)
Jenkins Console Log Machine informationProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010CPU: G++: Clang: |
Summary
Adds holder operation to kernel generator. This operation is a no-op, but the operation object also owns pointers to some dynamically allocated objects, used in the argument expression.
This is the solution to problems such as #1859 (comment) or #1775 with kernel generator expressions.
Also creates default
generatemember funtion inoperation_cl.Tests
Holder is tested to keep the owned objects alive.
Side Effects
None.
Release notes
Adds holder operation to kernel generator.
Checklist
Math issue Implement OpenCL kernel generator #1342, Propogating Eigen expression templates with local temporaries #1775
Copyright holder: Tadej Ciglarič
The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
- Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
- Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
the basic tests are passing
./runTests.py test/unit)make test-headers)make test-math-dependencies)make doxygen)make cpplint)the code is written in idiomatic C++ and changes are documented in the doxygen
the new changes are tested