Invalidations

[timholy]

Continuing from JuliaLang/julia#41913:

It's not just Polyester:

 deleting num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42 invalidated:
   mt_backedges:  1: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for Polyester.worker_size() (1 children)
                  2: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for Polyester._batch_no_reserve(::Polyester.var"#11#12", ::UInt16, ::UInt32, ::UInt16, ::UInt64, ::UInt64, ::UInt64, ::Static.StaticInt{1}, ::Static.StaticInt{1}) (1 children)
                  3: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for TriangularSolve.div_dispatch!(::Matrix{Float64}, ::Matrix{Float64}, ::Matrix{Float64}, ::Val{true}, ::Val{true}) (1 children)
                  4: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for TriangularSolve.nmuladd!(::VectorizationBase.StridedPointer{Float64, 2, 2, 0, (2, 1), Tuple{Int64, Static.StaticInt{8}}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::VectorizationBase.StridedPointer{Float64, 2, 2, 0, (2, 1), Tuple{Int64, Static.StaticInt{8}}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::VectorizationBase.StridedPointer{Float64, 2, 2, 0, (2, 1), Tuple{Int64, Static.StaticInt{8}}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::Int64, ::Int64, ::Int64) (1 children)
                  5: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for TriangularSolve.multithread_rdiv!(::VectorizationBase.StridedPointer{Float64, 2, 2, 0, (2, 1), Tuple{Int64, Static.StaticInt{8}}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::VectorizationBase.StridedPointer{Float64, 2, 2, 0, (2, 1), Tuple{Int64, Static.StaticInt{8}}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::VectorizationBase.StridedPointer{Float64, 2, 2, 0, (2, 1), Tuple{Int64, Static.StaticInt{8}}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::Int64, ::Int64, ::Int64, ::Val{false}, ::Static.StaticInt{2}) (1 children)
                  6: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for TriangularSolve.div_dispatch!(::LinearAlgebra.Transpose{Float64, Matrix{Float64}}, ::LinearAlgebra.Transpose{Float64, Matrix{Float64}}, ::LinearAlgebra.Transpose{Float64, Matrix{Float64}}, ::Val{true}, ::Val{true}) (1 children)
                  7: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for TriangularSolve.multithread_rdiv!(::VectorizationBase.StridedPointer{Float64, 2, 2, 0, (2, 1), Tuple{Int64, Static.StaticInt{8}}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::VectorizationBase.StridedPointer{Float64, 2, 2, 0, (2, 1), Tuple{Int64, Static.StaticInt{8}}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::VectorizationBase.StridedPointer{Float64, 2, 2, 0, (2, 1), Tuple{Int64, Static.StaticInt{8}}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::Int64, ::Int64, ::Int64, ::Val{true}, ::Static.StaticInt{2}) (1 children)
                  8: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for TriangularSolve._nthreads() (1 children)
                  9: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for TriangularSolve.div_dispatch!(::LinearAlgebra.Transpose{Float64, Matrix{Float64}}, ::LinearAlgebra.Transpose{Float64, Matrix{Float64}}, ::LinearAlgebra.Transpose{Float64, Matrix{Float64}}, ::Val{false}, ::Val{true}) (1 children)
                 10: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for TriangularSolve.multithread_rdiv!(::VectorizationBase.StridedPointer{Float64, 2, 1, 0, (1, 2), Tuple{Static.StaticInt{8}, Int64}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::VectorizationBase.StridedPointer{Float64, 2, 1, 0, (1, 2), Tuple{Static.StaticInt{8}, Int64}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::VectorizationBase.StridedPointer{Float64, 2, 1, 0, (1, 2), Tuple{Static.StaticInt{8}, Int64}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::Int64, ::Int64, ::Int64, ::Val{false}, ::Static.StaticInt{1}) (1 children)
                 11: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for TriangularSolve.multithread_rdiv!(::VectorizationBase.StridedPointer{Float64, 2, 1, 0, (1, 2), Tuple{Static.StaticInt{8}, Int64}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::VectorizationBase.StridedPointer{Float64, 2, 1, 0, (1, 2), Tuple{Static.StaticInt{8}, Int64}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::VectorizationBase.StridedPointer{Float64, 2, 1, 0, (1, 2), Tuple{Static.StaticInt{8}, Int64}, Tuple{Static.StaticInt{0}, Static.StaticInt{0}}}, ::Int64, ::Int64, ::Int64, ::Val{true}, ::Static.StaticInt{1}) (1 children)
                 12: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for TriangularSolve.div_dispatch!(::Matrix{Float64}, ::Matrix{Float64}, ::Matrix{Float64}, ::Val{false}, ::Val{true}) (1 children)
                 13: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for RecursiveFactorization.recurse!(::StrideArraysCore.PtrArray{Tuple{Int64, Int64}, (true, true), Float64, 2, 1, 0, (1, 2), Tuple{Static.StaticInt{8}, Int64}, Tuple{Static.StaticInt{1}, Static.StaticInt{1}}}, ::Val{true}, ::Int64, ::Int64, ::Int64, ::StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Int64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, ::Int64, ::Int64) (1 children)
                 14: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for RecursiveFactorization.apply_permutation_threaded!(::StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Int64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, ::StrideArraysCore.PtrArray{Tuple{Int64, Int64}, (true, false), Float64, 2, 1, 0, (1, 2), Tuple{Static.StaticInt{8}, Int64}, Tuple{Static.StaticInt{1}, Static.StaticInt{1}}}) (1 children)
                 15: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for RecursiveFactorization.apply_permutation_threaded!(::StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Int64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, ::Any) (1 children)
                 16: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for RecursiveFactorization.apply_permutation_threaded!(::StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Int64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, ::StrideArraysCore.PtrArray{Tuple{Int64, Int64}, (true, true), Float64, 2, 1, 0, (1, 2), Tuple{Static.StaticInt{8}, Int64}, Tuple{Static.StaticInt{1}, Static.StaticInt{1}}}) (1 children)
                 17: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for DiffEqBase.var"#_#32"(::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, ::DiffEqBase.LUFactorize, ::Vector{Float64}, ::Matrix{Float64}, ::Vector{Float64}, ::Bool) (1 children)
                 18: signature num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:94 (formerly num_threads() in CPUSummary at /home/tim/.julia/dev/CPUSummary/src/topology.jl:42) triggered MethodInstance for OrdinaryDiffEq.compute_step!(::OrdinaryDiffEq.NLSolver{NLNewton{Rational{Int64}, Rational{Int64}, Rational{Int64}}, true, Vector{Float64}, Float64, Nothing, Float64, Ordin...

The way I have it set:

julia> Sys.CPU_THREADS
4

julia> Threads.nthreads()
1

which may explain why I'm seeing those invalidations and others may not?

[chriselrod]

which may explain why I'm seeing those invalidations and others may not?

You shouldn't when starting Julia with -t4.

If we want to fix these, the solution is to have those libraries / functions stop using num_threads().
I don't think the invalidations of num_threads are avoidable if we want it to maintain the current behavior.

[timholy]

See the good news in SciML/DifferentialEquations.jl#786 (emergency is over 🙂 ). We should still poke at this but it may not beisn't urgent.

[timholy]

I did just start Julia with julia -t4 and still saw those invalidations. I'm wondering if there's some compilation-state dependence, and in particular whether it matters whether you build CPUSummary via ] precompile or via using SomePackageThatForcesItToBuild.

With master for Julia and ] dev SnoopCompile SnoopCompileCore (or just "regular" when 2.8 comes out) it's pretty easy to check:

using SnoopCompileCore
invalidations = @snoopr using OrdinaryDiffEq ModelingToolKit;
using SnoopCompile
trees = invalidation_trees(invalidations)

and then look for a num_threads tree.

Again, I don't think this is an issue of primary importance, but it does seems worth keeping track of for a rainy day. The perform_step! recompilation is ~0.5s, so not entirely cheap (but not catastrophic either).

[chriselrod]

using SnoopCompileCore
invalidations = @snoopr using OrdinaryDiffEq, ModelingToolkit;
using SnoopCompile, CPUSummary
trees = invalidation_trees(invalidations);
ctrees = filtermod(CPUSummary, trees)

I get

julia> ctrees = filtermod(CPUSummary, trees)
1-element Vector{SnoopCompile.MethodInvalidations}:
 inserting convert(S::Type{<:Union{Number, T}}, p::MultivariatePolynomials.AbstractPolynomialLike{T}) where T in MultivariatePolynomials at /home/chriselrod/.julia/packages/MultivariatePolynomials/vqcb5/src/conversion.jl:65 invalidated:
   mt_backedges: 1: signature Tuple{typeof(convert), Type{Hwloc.Attribute}, Any} triggered MethodInstance for CPUSummary.safe_topology_load!() (1 children)


julia> Threads.nthreads(), Sys.CPU_THREADS
(8, 8)

In another Julia session

julia> ctrees = filtermod(CPUSummary, trees)
2-element Vector{SnoopCompile.MethodInvalidations}:
 inserting convert(S::Type{<:Union{Number, T}}, p::MultivariatePolynomials.AbstractPolynomialLike{T}) where T in MultivariatePolynomials at /home/chriselrod/.julia/packages/MultivariatePolynomials/vqcb5/src/conversion.jl:65 invalidated:
   mt_backedges: 1: signature Tuple{typeof(convert), Type{Hwloc.Attribute}, Any} triggered MethodInstance for CPUSummary.safe_topology_load!() (1 children)

 deleting num_threads() in CPUSummary at /home/chriselrod/.julia/packages/CPUSummary/dEmFX/src/topology.jl:42 invalidated:
   backedges: 1: superseding num_threads() in CPUSummary at /home/chriselrod/.julia/packages/CPUSummary/dEmFX/src/topology.jl:42 with MethodInstance for CPUSummary.num_threads() (2 children)


julia> Threads.nthreads(), Sys.CPU_THREADS
(1, 8)

(ode) pkg> st CPUSummary
      Status `~/Documents/progwork/julia/env/ode/Project.toml`
  [2a0fbf3d] CPUSummary v0.1.2

So it appears to be working as intended for me.

[chriselrod]

Again, I don't think this is an issue of primary importance, but it does seems worth keeping track of for a rainy day. The perform_step! recompilation is ~0.5s, so not entirely cheap (but not catastrophic either).

I strongly suspect this is enough to favor Threads.nthreads() there.
If you or @ChrisRackauckas have a benchmark I can run to confirm negligible runtime difference, I'll do that. I'll also try a few microbenchmarks.

[timholy]

I have

$ env | grep -i thread
JULIA_CPU_THREADS=4

Is that possibly problematic? This is on a Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz (6 physical cores).

[chriselrod]

Ah, yes. It doesn't actually use Sys.CPU_THREADS, instead preferring to use information from Hwloc (width adjustments for ARM Macs -- I think Hwloc might have more details to handle things better).
Presumably you won't get invalidations when starting with 12 threads.

Perhaps, in the case of disagreement, I should favor Sys.CPU_THREADS over the actual number of threads, assuming the disagreement is because of a deliberate user choice.

[timholy]

At least on nightly, and when starting with -t4, the only thing that seems to be holding back good precompilation of LV-generated code is the redefinition of cache_size

CPUSummary.jl/src/x86.jl

Line 62 in f5315fc

@eval cache_size(::Union{Val{3},StaticInt{3}}) = $(static(cache_l3_per_core * nc))

which was formerly defined on

CPUSummary.jl/src/x86.jl

Line 34 in f5315fc

@eval cache_size(::Union{Val{$i},StaticInt{$i}}) = $(static(csi))

Not easy for me to fix because method redefinition like this is not "typical" (I recognize you do amazing, atypical things) and I don't know the motivations well enough to offer an alternative.

CC @Tokazama

[timholy]

I'd be happy to offer an integration test that checks for new inference when running the precompiled workload for a demo consumer of LoopVectorization. If you want it, just let me know which repo I should submit it to.

[chriselrod]

Not easy for me to fix because method redefinition like this is not "typical" (I recognize you do amazing, atypical things) and I don't know the motivations well enough to offer an alternative.

We should stop doing that.

[chriselrod]

I'd be happy to offer an integration test that checks for new inference when running the precompiled workload for a demo consumer of LoopVectorization. If you want it, just let me know which repo I should submit it to.

Which repo do you think would be best? LoopVectorization.jl itself, or something that depends on it like TriangularSolve.jl or RecursiveFactorization.jl?

[timholy]

Probably LV itself. The only issue to be aware of is that tracking down the origin of breakage might require a bit of hunting: if a PR to, say, this package breaks the integration test, then you won't know you've broken it until you next run the tests of LoopVectorization.jl. Unless you like the idea of running that specific test in several of LV's dependencies? You can see somethng similar to what I mean in CodeTracking, which exists to serve Revise:

• https://github.com/timholy/CodeTracking.jl/blob/426fc0e5af69ca410f3a1b77458db5fc5d68e864/test/runtests.jl#L160
• https://github.com/timholy/CodeTracking.jl/blob/426fc0e5af69ca410f3a1b77458db5fc5d68e864/.github/workflows/ci.yml#L45-L56

[Tokazama]

Apologies I'm not sure what the original motivation for redefining it like this was.