Remove sort(::Dict) piracy (as bugfix)

[ericphanson]

Fixes #25

Alternative to #109 - if PkgEval says none of our direct dependents are relying on this deprecated pathway, perhaps we can remove the sort deprecation as a bugfix. (Suggested by @KristofferC).

@maleadt would it be possible to enable nanosoldier on this repo? (per https://github.com/JuliaCI/Nanosoldier.jl#reverse-ci-for-packages).

[ericphanson]

Let us try it:

@nanosoldier runtests()

[nanosoldier]

Update on PkgEvalJob f07867e vs. c3687a8: Accepted

[nanosoldier]

Update on PkgEvalJob f07867e vs. c3687a8: Running

[nanosoldier]

Your job failed. Consult the server logs for more details (cc @maleadt).

[maleadt]

I've deployed some fixes.

@nanosoldier runtests()

[nanosoldier]

Update on PkgEvalJob f07867e vs. c3687a8: Accepted

[nanosoldier]

Update on PkgEvalJob f07867e vs. c3687a8: Running

[nanosoldier]

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

[ericphanson]

I’m assuming the timeouts are unrelated, but the two failures are both real. Interestingly the OPFSampler one hits a Compat.jl method with this PR due to JuliaLang/Compat.jl#804

I’m not sure DataStructures should count bc we can just remove those tests, but probably we should test all of its dependents as it reexports the OrderedCollections api

[maleadt]

Lots of timeouts though... I've added some debugging code to see what's going on. Hopefully you don't mind me using this PR to debug PkgEval; feel free to remove those comments afterwards 🙂

@nanosoldier runtests()

[nanosoldier]

Update on PkgEvalJob f07867e vs. c3687a8: Accepted

[nanosoldier]

Update on PkgEvalJob f07867e vs. c3687a8: Running

[nanosoldier]

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

[KristofferC]

Looks ok to me, the failure in https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_hash/f07867e_vs_c3687a8/OPFSampler.primary.log is not a failure in the package itself but a failure that only occurs from the testing code (which has a trivial fix).

[ericphanson]

Shall we do DataStructures dependents too, as it reexports OrderedCollections api? I guess I’d need to pass them all in a long list to nanosoldier

[ericphanson]

Let us try DataStructures deps:

@nanosoldier runtests(["MatrixCompletion", "AvailablePotentialEnergyFramework", "CircoCore", "Jusdl", "CJKFrequencies", "Coluna", "QuantumCircuitOpt", "OMJulia", "Sched", "YasolSolver", "CurricularAnalytics", "Finch", "UnfoldMakie", "FiniteStateTransducers", "Polyomino", "MotifPvalue", "ODEInterfaceDiffEq", "StatsPlots", "StorageSystemsSimulations", "ValueHistories", "KongYiji", "MahjongTiles", "ModalDecisionTrees", "ARCHModels", "InvariantCausal", "Dolang", "ObjectOriented", "Discretizers", "TinyModia", "MultilayerGraphs", "BKTrees", "Santiago", "PSID", "PrincipalMomentAnalysisApp", "GIFImages", "PhyloNetworks", "Signals", "SegmentIntersections", "TraceEstimators", "IceFloeTracker", "Scruff", "Sisyphus", "TenPuzzle", "UnitRangesSortedSets", "Ignite", "Backtester", "StructC14N", "MriResearchTools", "LightBSON", "NeutralLandscapes", "DSGE", "IntegratedOscillatorModel", "GeneralizedGenerated", "ServerMetrics", "Omega", "Phylo", "ReactiveMP", "MIPLearn", "QXTns", "TiffImages", "Caching", "Pandoc", "Turing", "ADRIA", "AudioSchedules", "MusicManipulations", "ObjectPools", "ExprOptimization", "PPTX", "NearestNeighborDescent", "Gnuplot", "LabMaze", "DBnomics", "STAC", "PkgCite", "GenieBuiltLifeProto", "JsonGrinder", "DIVAnd", "TabletWeaving", "TimetableSolver", "StochasticIntegrals", "StringAnalysis", "DelaunayTriangulation", "GeoArrayOps", "PhaseSpaceIO", "ClimaCore", "Transformers", "Checkpoints", "DelaySSAToolkit", "SaguaroTrader", "Metrics", "OptimizedEinsum", "Molly", "Gridap", "DiscreteEventsLite", "RCall", "PolyJuMP", "EBIC", "OMETIFF", "SegmentAnalytics", "CSetAutomorphisms", "StaticWebPages", "CryoGrid", "Bibliography", "Decapodes", "Impostor", "K8sClusterManagers", "ImageComponentAnalysis", "Rocket", "Compose", "ZigZagBoomerang", "RRRMC", "QMDDocTools", "InfiniteOpt", "AsyPlots", "SpmGrids", "SpmSpectroscopy", "Luxor", "ReinforcementLearningTrajectories", "InfrastructureSystems", "FastRunningMedian", "WavePropBase", "PowerAnalytics", "Taylor", "Zarr", "StatProfilerHTML", "PowerGraphics", "UnitCommitment", "Catalyst", "Visor", "SpmImages", "Lerche", "SimString", "HetaSimulator", "Dagger", "ClusterRunner", "StructuralIdentifiability", "BokehServer", "GraphKernels", "Air", "FlashWeave", "SymmetricPolynomials", "Sundials", "SpinMonteCarlo", "TimeDag", "Dagitty", "Checkpointing", "Amaru", "GeometricFlux", "RegressionAndOtherStories", "BSONify", "DoubleEnded", "PyDSTool", "AStarSearch", "JuliaFormatter", "Pigeons", "ICD_GEMs", "PolynomialMatrices", "KeldyshED", "MLSuite", "Manopt", "Qwind", "TextParse", "DFTforge", "SimplePlots", "Mimi", "Graphs", "OptimalMatrixCompletion", "StatsBase", "ResourcePools", "Crispulator", "CDCLsat", "Eikonal", "SpinAdaptedSecondQuantization", "TeaFiles", "Scheduling", "HerbGrammar", "ALFA", "Sunny", "AlphaZero", "Classes", "Boscia", "MacroModelling", "StructuralCausalModels", "MHLib", "IterativeNelderMead", "InteractiveDynamics", "Catlab", "OptimalApplication", "PowerSimulationsDynamics", "ChartParsers", "Dolo", "JavaCall", "Publish", "SumOfSquares", "Luna", "MCPhylo", "ModelingToolkit", "SimplePartitions", "Vega", "TauP", "SourceWalk", "OrbitalTrajectories", "ComoniconGUI", "QuadGK", "RoundAndSwap", "GeneDrive", "Laplacians", "MultivariateFunctions", "Redis", "ElectricGrid", "StLifeIns", "JudiLing", "OceanRobots", "TabularTDLearning", "Gadfly", "JumpProcesses", "FastDifferentiation", "ImageFiltering", "SimpleValueGraphs", "Braket", "BifurcationKit", "GreedyAlign", "MatrixMerge", "NamedArrays", "BeaData", "PowerSystems", "GenomicFeatures", "ExtensibleScheduler", "PowerSimulations", "ParameterizedFunctions", "AlgebraicRewriting", "CompScienceMeshes", "SmallZarrGroups", "Tilde", "SoleModels", "ACTRSimulators", "VegaDatasets", "BitSAD", "MPIMeasurements", "DynamicAxisWarping", "SortingAlgorithms", "GATlab", "SymbolicCodegen", "CharFuncPricing", "MetidaStats", "CUTEst", "ChowLiuTrees", "ITensorNetworks", "KernelDensitySJ", "KiwiConstraintSolver", "Stan", "ImageEdgeDetection", "Symbolics", "PolynomialAmoebas", "Hydrographs", "MatrixNetworks", "SpmImageTycoon", "GlobalSearchRegressionGUI", "BibParser", "ONNXRunTime", "CVRPLIB", "Phonetics", "HybridAStar", "SoleLogics", "LaplacianOpt", "DynamicalSystems", "ForneyLab", "Rimu", "IteratedIntegration", "ImageMorphology", "HydroRefStations", "Polylabel", "QXTools", "QuantumLattices", "GraphNeuralNetworks", "PressureFieldContact", "SigmoidalProgramming", "FindComplexZeros", "CDLmotif", "LFUDACache", "QuantumClifford", "NumCME", "LifeInsuranceDataModel", "SymbolicNumericIntegration", "Causal", "GridWorlds", "CrypticCrosswords", "FoldRNA", "ProjectManagement", "DiffEqJump", "RxInfer", "TuringMachine", "SummarizedExperiments", "MCPhyloTree", "TextModels", "MultiAssayExperiments", "PlateMotionRequests", "Assignment", "Banyan", "Allocations", "OpenStreetMapX", "EchelleBase", "TLDR", "SFrontiers", "MultiAgentPathFinding", "GridGraphs", "SparseDiffTools", "EAGO", "FlagSOS", "TextAnalysis", "RealTimeScheduling", "OnlinePortfolioSelection", "Awabi", "RigidBodySim", "NeXLParticle", "UnfoldCDL", "SortedIteratorProducts", "MathOptInterface", "SimpleGraphs", "ReverseDiffSparse", "Jutul", "TuringCallbacks", "InducingPoints", "PlanarMaps", "FluxTraining", "ReliabilityDiagrams", "Mineos", "StatPlots", "TreesHeaps", "TypeDBClient", "CeMicrodata", "SimpleAiboWebAPI", "ClimaTimeSteppers", "Coulter", "FractionalGaussianFields", "PolynomialRings", "Bonobo", "SoleData", "REDCap", "HerbSearch", "Soss", "ZhuyinPinyin", "CurveFitParameters", "BloqadeQMC", "Empirikos", "ModiaMath", "Agents", "SemanticScholar", "AdaStress", "CMBLensing", "DiscreteMarkovChains", "DiscreteEvents", "MergedIterators", "Shrike", "StochasticDiffEq", "EcologicalNetworks", "NCDatasets", "ReactionNetworkImporters", "GlobalSensitivityAnalysis", "BayesNets", "SolidStateDetectors", "Geomorphometry", "DisplayStructure", "QXZoo", "ExprRules", "NetcdfIO", "EcRequests", "Circo", "Plasmo", "MomentClosure", "Kombinator", "MultivariatePolynomials", "InteractiveChaos", "LightOSM", "Alakazam", "Caesar", "Bangumis", "AccelInterfaces", "Numerics", "Bonsai", "HCubature", "AnytimeWeightedAStar", "Ripserer", "RealNeuralNetworks", "Mill", "YAXArrayBase", "GRIBDatasets", "TermFrequencyInverseDocumentFrequency", "WGPUgfx", "ContextTracking", "ConstraintSolver", "HierarchicalUtils", "SPDX", "VegaLite", "CombinatorialBandits", "Pickle", "ReusableFunctions", "Mads", "ImageSegmentation", "JunctionTrees", "JutulDarcy", "BioMakie", "PolyhedralRelaxations", "SymbolicPlanners", "SweepContractor", "JupyterParameters", "MathTeXEngine", "AssociativeWindowAggregation", "EconoSim", "NextGP", "QuadraticToBinary", "StochasticDelayDiffEq", "StatsModels", "BasicAkerRelationalScore", "Gen", "VectorSphericalWaves", "CurveProximityQueries", "MicrobiomeAnalysis", "SimpleHypergraphs", "ChemometricsData", "OptimizationAlgorithms", "DiffEqCallbacks", "ConcurrentSim", "POMDPStressTesting", "SymbolicUtils", "PDSampler", "RELOG", "ExpandNestedData", "QuantEcon", "VimBindings", "OdsIO", "Dolphyn", "ConstraintTrees", "DickeModel", "COSMO", "Chordal", "BytePairEncoding", "ConstrainedShortestPaths", "OrdinaryDiffEq", "PriorityChannels", "QuantumCitations", "YAXArrays", "MCMCDiagnosticTools", "OmicsProfiles", "SnowyOwl", "PowerApps", "BosonSampling", "DelayDiffEq", "HAML", "ACSets", "Cropbox", "CommonDataModel", "ScatterNNlib", "TexTables", "DrakeVisualizer", "Metatheory", "Erdos", "Dash", "GModelFit", "QueryOperators", "MOTIFs", "GraphMakie", "AlgorithmicCompetition", "ParGOGraphs", "DataFrames", "PyBraket", "SparseExtra", "RBNF", "vOptSpecific", "DiffEqBase", "IncrementalInference", "MemPool", "SmartParser", "RemoteHPC", "CalibrationErrors", "QXContexts", "Starlight", "MIToS", "SimJulia", "PerfectPacking", "Watershed", "Reactive", "PowerSystemCaseBuilder", "CombinatorialEnumeration", "QuickHeaps", "ImplicitGraphs", "ParallelUtilities", "TSPLIB", "LightGraphs", "DynamicBoundspODEsDiscrete", "DoloYAML", "SetProg", "ScHoLP", "Indicomb"])

[nanosoldier]

Update on PkgEvalJob f07867e vs. c3687a8: Accepted

[nanosoldier]

Update on PkgEvalJob f07867e vs. c3687a8: Running

[nanosoldier]

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

[ericphanson]

I am a pkg-eval-reader-noob, but I only see 1 failure, which looks like it would be fixed by AdamAker/BasicAkerRelationalScore.jl#3. So this looks pretty good to me.

If there is consensus, we could proceed by:

1. Possibly PR OPFSampler.jl to update it? Not sure if that repo is still active as invenia has folded, and I can't load the package on my m1 laptop
2. PR Datastructures to stop testing the deprecated sort methods for Dict
3. Merge and tag this as a patch release

Perhaps folks can 👍 or 👎 this post to chime in that they agree it is OK to remove the deprecated sort piracy as a bugfix release, given these results. And in a few days if there is agreement we can proceed.

[oxinabox]

Possibly PR OPFSampler.jl to update it? Not sure if that repo is still active as invenia has folded, and I can't load the package on my m1 laptop

I wouldn't bother.
I didn't know we open sourced that.
If anyone is using it I am sure they can debug it.

[maleadt]

Back to debugging PkgEval (ignore this comment):

@nanosoldier runtests(configuration = (env=["JULIA_DEBUG=Pkg,loading"],), vs_configuration = (env=["JULIA_DEBUG=Pkg,loading"],))

[nanosoldier]

Update on PkgEvalJob ff2564b vs. 87a2334: Accepted

[nanosoldier]

Update on PkgEvalJob ff2564b vs. 87a2334: Running

[nanosoldier]

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

[ericphanson]

Looks like we have agreement that this is the right way to proceed, so I will merge & tag tomorrow if no objections arise

[oxinabox]

So i have continued thinking.
(I have been thinking about this since the PR was openned.)
I think we shouldn't do this now.
I am sorry for my very delayed response, and I appreciate that a lot of work has already gone in to validating that there are negligable effects on the package ecosystem.
This package is upstream of like 1/3rd the ecosystem, which means it is upstream of a ton of user scripts and those scripts are not caught by package eval.

I think we should wait either for a major release of OrderedCollections.jl
or at least until we fix deprecations per JuliaLang/julia#49583 to show deprecations that are your fault during normal exectution. Then we can reassess.

I don't think this is e.g. causing a huge amount of invalidations etc. So I do not see that there is a rush.

It also came up for unrelated releated reasons in a Triage call with @LilithHafner. @JeffBezanson thinks this type piracy is good XD.

[ericphanson]

Ok, thanks very much for the feedback! I was actually motivated by Base adding (then reverting) sorting for iterables, which seemed to make this piracy worse to me, since the type you would get would depend on you dependency stack (and whether or not you had Revise loaded, which likely means a difference between the devs machine and prod).

On the other hand, having noisier deprecations first makes a lot of sense too.

[KristofferC]

Horrifying.