Permit n-argument operators

[MilesCranmer]

This opens up DynamicExpressions.jl to D-degree nodes via an additional type parameters. This is still a work-in-progress, but I wanted to track the TODO's here.

TODO:

• Write upgrade guide in CHANGELOG.md
• Tests
  • Test string paths
  • Random expression evals for arbitrary arity, compare against Julia evaluation
  • 100% diff coverage
• Performance
  • Evaluation performance (seems to be roughly the same now)
  • Allocation performance (at the moment, it seems to result in double the # of allocations, and 33% greater memory usage)
  • Either split types, or get generic type to match existing performance for
  • Consider using ::Union{Nothing,N} rather than Ref
  • Profile SymbolicRegression.jl against the diff
• Feature coverage
  • Generic operator enum compatibility
  • Derivative compatibility
  • Preallocation compatibility
  • LoopVectorization.jl compat
  • Bumper.jl compat
  • Verify graph node compat
  • Verify simplification compat
• Cleaning
  • Simplify evaluation code with generic Base.Cartesian version
  • Manually verify no internal uses of .l and .r (temporarily set these to be errors)
  • Introduce alias Node{T} = NNode{T,2} for full backwards compatibility

[MilesCranmer]

Should add a depwarn here

[coveralls]

Pull Request Test Coverage Report for Build 14946536677

Details

• 246 of 278 (88.49%) changed or added relevant lines in 14 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-0.9%) to 94.724%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/ExpressionAlgebra.jl	41	42	97.62%
src/OperatorEnum.jl	16	17	94.12%
src/Node.jl	70	72	97.22%
src/ReadOnlyNode.jl	10	16	62.5%
src/Evaluate.jl	16	38	42.11%

Totals
Change from base Build 14729298079:	-0.9%
Covered Lines:	2657
Relevant Lines:	2805

---

💛 - Coveralls

[github-actions]

Benchmark Results (Julia v1)

Time benchmarks

	master	fe1c9d7...	master / fe1c9d7...
eval/ComplexF32/evaluation	7.29 ± 0.56 ms	7.29 ± 0.58 ms	0.999 ± 0.11
eval/ComplexF64/evaluation	10.8 ± 1 ms	10.9 ± 1.2 ms	0.991 ± 0.14
eval/Float32/derivative	11.6 ± 0.55 ms	11.8 ± 0.78 ms	0.979 ± 0.08
eval/Float32/derivative_turbo	11.7 ± 0.55 ms	11.6 ± 0.64 ms	1 ± 0.073
eval/Float32/evaluation	2.75 ± 0.26 ms	2.77 ± 0.27 ms	0.991 ± 0.14
eval/Float32/evaluation_bumper	0.564 ± 0.016 ms	0.575 ± 0.015 ms	0.981 ± 0.038
eval/Float32/evaluation_turbo	0.516 ± 0.031 ms	0.535 ± 0.031 ms	0.966 ± 0.081
eval/Float32/evaluation_turbo_bumper	0.566 ± 0.016 ms	0.572 ± 0.016 ms	0.989 ± 0.039
eval/Float64/derivative	14.7 ± 1.4 ms	14.9 ± 1.5 ms	0.983 ± 0.13
eval/Float64/derivative_turbo	15 ± 1.3 ms	14.7 ± 1.2 ms	1.02 ± 0.12
eval/Float64/evaluation	3.14 ± 0.34 ms	3.15 ± 0.33 ms	0.995 ± 0.15
eval/Float64/evaluation_bumper	1.18 ± 0.044 ms	1.19 ± 0.044 ms	0.991 ± 0.052
eval/Float64/evaluation_turbo	1.01 ± 0.067 ms	1.01 ± 0.067 ms	0.999 ± 0.093
eval/Float64/evaluation_turbo_bumper	1.18 ± 0.044 ms	1.19 ± 0.042 ms	0.997 ± 0.051
utils/combine_operators/break_sharing	0.0387 ± 0.00064 ms	0.039 ± 0.00077 ms	0.991 ± 0.026
utils/convert/break_sharing	26.6 ± 1.2 μs	26.2 ± 1.9 μs	1.02 ± 0.087
utils/convert/preserve_sharing	0.0984 ± 0.0067 ms	0.0989 ± 0.0079 ms	0.995 ± 0.1
utils/copy/break_sharing	27.5 ± 1.6 μs	26.3 ± 1.9 μs	1.05 ± 0.098
utils/copy/preserve_sharing	0.0982 ± 0.0064 ms	0.0998 ± 0.0079 ms	0.984 ± 0.1
utils/count_constant_nodes/break_sharing	12 ± 0.41 μs	12 ± 0.7 μs	0.999 ± 0.067
utils/count_constant_nodes/preserve_sharing	0.0862 ± 0.0068 ms	0.0857 ± 0.0074 ms	1.01 ± 0.12
utils/count_depth/break_sharing	13 ± 0.56 μs	12.4 ± 0.75 μs	1.05 ± 0.078
utils/count_nodes/break_sharing	11.6 ± 0.64 μs	11.3 ± 0.54 μs	1.02 ± 0.075
utils/count_nodes/preserve_sharing	0.086 ± 0.0071 ms	0.083 ± 0.0045 ms	1.04 ± 0.1
utils/get_set_constants!/break_sharing	0.034 ± 0.0024 ms	0.0325 ± 0.0017 ms	1.05 ± 0.093
utils/get_set_constants!/preserve_sharing	0.176 ± 0.01 ms	0.177 ± 0.011 ms	0.994 ± 0.082
utils/get_set_constants_parametric	0.0454 ± 0.0026 ms	0.0448 ± 0.0031 ms	1.01 ± 0.09
utils/has_constants/break_sharing	6.7 ± 0.55 μs	6.86 ± 0.78 μs	0.977 ± 0.14
utils/has_operators/break_sharing	2.48 ± 0.2 μs	2.54 ± 0.25 μs	0.972 ± 0.12
utils/hash/break_sharing	23 ± 0.71 μs	21.8 ± 0.87 μs	1.06 ± 0.053
utils/hash/preserve_sharing	0.0962 ± 0.0052 ms	0.0985 ± 0.0075 ms	0.976 ± 0.091
utils/index_constant_nodes/break_sharing	24.6 ± 1 μs	26.6 ± 0.89 μs	0.925 ± 0.049
utils/index_constant_nodes/preserve_sharing	0.0986 ± 0.0064 ms	0.0994 ± 0.0071 ms	0.992 ± 0.095
utils/is_constant/break_sharing	7.3 ± 0.5 μs	7.35 ± 0.7 μs	0.993 ± 0.12
utils/simplify_tree/break_sharing	0.163 ± 0.0039 ms	24.5 ± 0.74 μs	6.65 ± 0.26
utils/simplify_tree/preserve_sharing	0.223 ± 0.01 ms	0.107 ± 0.0077 ms	2.08 ± 0.18
utils/string_tree/break_sharing	0.47 ± 0.02 ms	0.467 ± 0.022 ms	1.01 ± 0.063
utils/string_tree/preserve_sharing	0.583 ± 0.024 ms	0.585 ± 0.03 ms	0.998 ± 0.065
time_to_load	0.219 ± 0.0023 s	0.232 ± 0.0036 s	0.945 ± 0.018

Memory benchmarks

	master	fe1c9d7...	master / fe1c9d7...
eval/ComplexF32/evaluation	0.972 k allocs: 2.49 MB	0.987 k allocs: 2.53 MB	0.985
eval/ComplexF64/evaluation	0.981 k allocs: 5 MB	0.993 k allocs: 5.06 MB	0.988
eval/Float32/derivative	4.68 k allocs: 17.6 MB	4.69 k allocs: 17.7 MB	0.997
eval/Float32/derivative_turbo	4.7 k allocs: 17.7 MB	4.62 k allocs: 17.4 MB	1.02
eval/Float32/evaluation	0.978 k allocs: 1.27 MB	0.99 k allocs: 1.29 MB	0.988
eval/Float32/evaluation_bumper	0.303 k allocs: 0.393 MB	0.303 k allocs: 0.393 MB	1
eval/Float32/evaluation_turbo	0.969 k allocs: 1.26 MB	0.957 k allocs: 1.25 MB	1.01
eval/Float32/evaluation_turbo_bumper	0.303 k allocs: 0.393 MB	0.303 k allocs: 0.393 MB	1
eval/Float64/derivative	4.75 k allocs: 0.0348 GB	4.72 k allocs: 0.0345 GB	1.01
eval/Float64/derivative_turbo	4.72 k allocs: 0.0345 GB	4.78 k allocs: 0.035 GB	0.987
eval/Float64/evaluation	1.01 k allocs: 2.57 MB	0.996 k allocs: 2.55 MB	1.01
eval/Float64/evaluation_bumper	0.303 k allocs: 0.771 MB	0.303 k allocs: 0.771 MB	1
eval/Float64/evaluation_turbo	0.993 k allocs: 2.54 MB	0.978 k allocs: 2.5 MB	1.02
eval/Float64/evaluation_turbo_bumper	0.303 k allocs: 0.771 MB	0.303 k allocs: 0.771 MB	1
utils/combine_operators/break_sharing	4 allocs: 0.953 kB	4 allocs: 0.953 kB	1
utils/convert/break_sharing	2 k allocs: 0.0924 MB	2 k allocs: 0.0924 MB	1
utils/convert/preserve_sharing	2.4 k allocs: 0.161 MB	2.4 k allocs: 0.161 MB	1
utils/copy/break_sharing	2 k allocs: 0.0924 MB	2 k allocs: 0.0924 MB	1
utils/copy/preserve_sharing	2.4 k allocs: 0.161 MB	2.4 k allocs: 0.161 MB	1
utils/count_constant_nodes/break_sharing	4 allocs: 0.953 kB	4 allocs: 0.953 kB	1
utils/count_constant_nodes/preserve_sharing	0.404 k allocs: 0.0696 MB	0.404 k allocs: 0.0696 MB	1
utils/count_depth/break_sharing	4 allocs: 0.953 kB	4 allocs: 0.953 kB	1
utils/count_nodes/break_sharing	4 allocs: 0.953 kB	4 allocs: 0.953 kB	1
utils/count_nodes/preserve_sharing	0.404 k allocs: 0.0696 MB	0.404 k allocs: 0.0696 MB	1
utils/get_set_constants!/break_sharing	0.898 k allocs: 25.2 kB	0.898 k allocs: 25.2 kB	1
utils/get_set_constants!/preserve_sharing	1.7 k allocs: 0.138 MB	1.7 k allocs: 0.138 MB	1
utils/get_set_constants_parametric	1.42 k allocs: 0.0663 MB	1.42 k allocs: 0.0663 MB	1
utils/has_constants/break_sharing	4 allocs: 0.203 kB	4 allocs: 0.203 kB	1
utils/has_operators/break_sharing	4 allocs: 0.203 kB	4 allocs: 0.203 kB	1
utils/hash/break_sharing	0.104 k allocs: 2.52 kB	0.104 k allocs: 2.52 kB	1
utils/hash/preserve_sharing	0.504 k allocs: 0.0711 MB	0.504 k allocs: 0.0711 MB	1
utils/index_constant_nodes/break_sharing	1.67 k allocs: 0.0501 MB	2.1 k allocs: 0.0635 MB	0.789
utils/index_constant_nodes/preserve_sharing	2.07 k allocs: 0.119 MB	2.5 k allocs: 0.132 MB	0.899
utils/is_constant/break_sharing	4 allocs: 0.203 kB	4 allocs: 0.203 kB	1
utils/simplify_tree/break_sharing	1.33 k allocs: 0.0436 MB	4 allocs: 0.953 kB	46.8
utils/simplify_tree/preserve_sharing	1.58 k allocs: 0.101 MB	0.404 k allocs: 0.0696 MB	1.45
utils/string_tree/break_sharing	11.8 k allocs: 1.04 MB	11.8 k allocs: 1.04 MB	1
utils/string_tree/preserve_sharing	12.2 k allocs: 1.11 MB	12.2 k allocs: 1.11 MB	1
time_to_load	0.159 k allocs: 11.2 kB	0.159 k allocs: 11.2 kB	1

[MilesCranmer]

Ok I've pretty much verified that SymbolicRegression.jl works on this without a single change needed within SR. So I think it's safe to say that this is backwards compatible. No need for a 2.0 release!

[MilesCranmer]

Actually I take that back. I think a full breaking change Node{T} to Node{T,D} would be much cleaner longterm. I am not a fan of having a separate NNode{T,D} type to deal with this. DE.jl also isn't used too much apart from in SymbolicRegression.jl (yet).

So, let's do a 2.0 release.