-
-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to calculate gradients on GPU with Embedding + OneHot Tensors (dim >= 3) #31
Comments
The file Here's a smaller demonstration of the problem: julia> using CUDA, OneHotArrays, NNlibCUDA
julia> CUDA.allowscalar(false)
julia> x = cu(onehotbatch([3, 4],1:5))
5×2 OneHotMatrix(::CuArray{UInt32, 1, CUDA.Mem.DeviceBuffer}) with eltype Bool:
⋅ ⋅
⋅ ⋅
1 ⋅
⋅ 1
⋅ ⋅
julia> cu(ones(3,5)) * x
3×2 CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}:
1.0 1.0
1.0 1.0
1.0 1.0
julia> cu(ones(3,2)) * x'
3×5 CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}:
0.0 0.0 1.0 1.0 0.0
0.0 0.0 1.0 1.0 0.0
0.0 0.0 1.0 1.0 0.0
julia> @which cu(ones(3,2)) * x'
*(A::AbstractMatrix, B::LinearAlgebra.Adjoint{Bool, <:OneHotMatrix})
@ OneHotArrays ~/.julia/packages/OneHotArrays/T3yiq/src/linalg.jl:13
julia> y = reshape(cu(onehotbatch([3 4],1:5)), 5, 2)
5×2 reshape(OneHotArray(::CuArray{UInt32, 2, CUDA.Mem.DeviceBuffer}), 5, 2) with eltype Bool:
⋅ ⋅
⋅ ⋅
1 ⋅
⋅ 1
⋅ ⋅
julia> y isa OneHotLike
true
julia> cu(ones(3,5)) * y
3×2 CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}:
1.0 1.0
1.0 1.0
1.0 1.0
julia> cu(ones(3,2)) * y'
ERROR: Scalar indexing is disallowed.
...
[4] _generic_matmatmul!(C::Matrix{Float32}, tA::Char, tB::Char, A::CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, B::Base.ReshapedArray{Bool, 2, OneHotArrays.OneHotArray{UInt32, 2, 3, CuArray{UInt32, 2, CUDA.Mem.DeviceBuffer}}, Tuple{Base.MultiplicativeInverses.SignedMultiplicativeInverse{Int64}, Base.MultiplicativeInverses.SignedMultiplicativeInverse{Int64}}}, _add::LinearAlgebra.MulAddMul{true, true, Bool, Bool})
julia> @which cu(ones(3,2)) * y'
*(A::AbstractMatrix, B::AbstractMatrix)
@ LinearAlgebra ~/julia-9ded051e9f/share/julia/stdlib/v1.10/LinearAlgebra/src/matmul.jl:139
julia> typeof(y')
LinearAlgebra.Adjoint{Bool, Base.ReshapedArray{Bool, 2, OneHotArray{UInt32, 2, 3, CuArray{UInt32, 2, CUDA.Mem.DeviceBuffer}}, Tuple{Base.MultiplicativeInverses.SignedMultiplicativeInverse{Int64}, Base.MultiplicativeInverses.SignedMultiplicativeInverse{Int64}}}} Xref recent #30 about Flux.OneHotLike. |
This method: Lines 13 to 17 in ddbba63
ought to allow for |
@mcabbott I would like to take a stab at this if it's okay |
That would be great. There's no formal process of assigning things, although we try a little to avoid PR races. |
Simple Example to Replicate:
Error Log:
Work Around:
By replacing inputs_oh above with a Integer Tensor where characters have been mapped to 1:26 makes sure everything runs with no errors on GPU. To test try below:
Useful Background Information:
The text was updated successfully, but these errors were encountered: