Adding stats functions #694
Conversation
src/Finch.jl
Outdated
| @@ -54,6 +54,7 @@ export lazy, compute, fused, tensordot, @einsum | |||
| export choose, minby, maxby, overwrite, initwrite, filterop, d | |||
|
|
|||
| export fill_value, AsArray, expanddims, tensor_tree | |||
| export mean | |||
There was a problem hiding this comment.
instead of exporting our own mean, we should use the mean from the Statistics standard library. We also don't want to depend on Statistics explicitly, instead we should use an "optional dependency", where mean, std, and var definitions are loaded only when the user loads both Finch and Statistics. Finch currently takes this approach with SparseArrays. You can see the approach in ext/SparseArraysExt.jl. We would also need to handle Statistics the same way we handle SparseArrays in the Project.toml. More on this concept here: https://pkgdocs.julialang.org/v1/creating-packages/#Conditional-loading-of-code-in-packages-(Extensions)
| function LinearAlgebra.norm(arr::AbstractTensorOrBroadcast, p::Real = 2) | ||
| compute(norm(lazy(arr), p))[] | ||
| end | ||
|
|
||
| """ | ||
| expanddims(arr::AbstractTensor, dims) | ||
| expanddims(arr::AbstractTensor, dims) |
There was a problem hiding this comment.
This line should stay indented
| @@ -7,12 +7,13 @@ const AbstractArrayOrBroadcasted = Union{AbstractArray,Broadcasted} | |||
| mutable struct LazyTensor{T, N} | |||
| data | |||
| extrude::NTuple{N, Bool} | |||
| shape::NTuple{N, Int} | |||
There was a problem hiding this comment.
For better or worse, Finch array dimensions may use different types independently. So you could have a matrix with row dimension UInt8 and column dimension BigInt. The way we handle this is to parameterize the type of LazyTensor based on the type of the dimensions:
mutable struct LazyTensor{T, N, Shape <: Tuple}
data
extrude::NTuple{N, Bool}
shape::Shape
fill_value::T
end
Note that the dimension tuple is not constrained to be length N with the type annotations. That's okay, we can check that it's correct when we construct the LazyTensor.
src/interface/lazy.jl
Outdated
|
|
||
| function Base.show(io::IO, tns::LazyTensor) | ||
| join(io, [x ? "1" : "?" for x in tns.extrude], "×") | ||
| join(io, [string(x) for x in tns.shape], "×") |
There was a problem hiding this comment.
I think you may also be able to get away with join(io, tns.shape, "×"), but I'm not sure, try it?
src/interface/lazy.jl
Outdated
| end | ||
|
|
||
| function Base.map!(dst, f, src::LazyTensor, args...) | ||
| res = map(f, src, args...) | ||
| return LazyTensor(identify(reformat(dst, res.data)), res.extrude, res.fill_value) | ||
| return LazyTensor(identify(reformat(dst, res.data)), res.extrude, res.size, res.fill_value) |
There was a problem hiding this comment.
Almost certainly, this should be size(res) or res.shape, depending on whether or not we know that res is a LazyTensor.
fixes #574
Implementing size property to lazy tensors to use with stats functions.