Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
ebe9e91
Depend on resolvo
jjerphan Jun 7, 2024
237ad2a
Add resolvo-cpp to the mamba environment
jjerphan Jun 13, 2024
b8795f7
Add a test for resolvo
jjerphan Jun 21, 2024
7bab460
Add specializations of `std::hash`
jjerphan Jun 21, 2024
d2a3160
Implement most methods
jjerphan Jun 24, 2024
4a0b12c
wip
jjerphan Jun 27, 2024
baca592
wip
jjerphan Jun 28, 2024
5d326dd
wip: Use custom pools
jjerphan Jun 28, 2024
183e365
Minimal Reproducer of the problem
jjerphan Jun 28, 2024
c28affb
Also sort on build number
jjerphan Jul 1, 2024
de9d09b
test: Addition of PackageInfo to PackageDatabase
jjerphan Jul 1, 2024
0fc9676
test: Filter solvables
jjerphan Jul 1, 2024
7f05c6f
Add more strings and names to pools
jjerphan Jul 1, 2024
e97546d
Initialize Candidates.{favored,locked} to nullptr
jjerphan Jul 1, 2024
6dbe87c
wip
jjerphan Jul 1, 2024
b5a7dcd
Parse repodata.json
jjerphan Jul 1, 2024
4ff86b7
wip: Test consistency with libsolv
jjerphan Jul 2, 2024
5f577a1
wip
jjerphan Jul 3, 2024
d05f0d2
test: mlflow=2.12.2 explicit
jjerphan Jul 8, 2024
c7d32dc
Debug
jjerphan Jul 8, 2024
6260b22
More tests
jjerphan Jul 9, 2024
c78107e
Complete ordering of solvables
jjerphan Jul 10, 2024
3864c84
Case for ordering on track features
jjerphan Jul 12, 2024
b1f764f
Remove old tests
jjerphan Jul 25, 2024
34f91a0
Combine test cases
jjerphan Jul 25, 2024
ade5030
Require, simplify, reformat, new sklearn example
jjerphan Feb 17, 2025
3f84945
Update scikit-learn explicit known resolution
jjerphan Feb 17, 2025
14a2010
Add comment regarding pip
jjerphan Feb 18, 2025
4814795
Reformat with recent prettier pre-commit setup
jjerphan Feb 18, 2025
83d85c5
Do not promote warnings as errors
jjerphan Feb 18, 2025
fd80f65
Add `resolvo-cpp` as a host dependency for micromamba builds
jjerphan Feb 18, 2025
623b3ff
Adapt Database and related signatures
jjerphan Apr 29, 2025
44b563e
Use `to_string` instead of `str`
jjerphan May 19, 2025
8afe9a4
Define a type for the variant
jjerphan May 19, 2025
92df455
Use DatabaseVariant for MTransaction and friends
jjerphan May 19, 2025
2590c52
Deduplicate configurable for using resolvo
jjerphan May 19, 2025
8350a72
Use DatabaseVariant for repoquery
jjerphan May 19, 2025
05eb464
Log solver used
jjerphan May 19, 2025
0926f05
WIP solver integration
jjerphan May 19, 2025
d727e79
Implement for resolvo
jjerphan May 20, 2025
f0487f6
Use resolvo 0.1.0 for now
jjerphan May 20, 2025
0976153
Pass ChannelParams to solver::resolvo::Database
jjerphan May 20, 2025
bb06b21
Revert changes made to `expected_caster.hpp`
jjerphan May 20, 2025
b50b59a
Adapt `create_repo_from_pkgs_dir`
jjerphan May 20, 2025
28fda54
Merge `database_has_package` implementations
jjerphan May 20, 2025
7a7fdc2
Deduplicate `database_has_package`
jjerphan May 20, 2025
413dd82
Resolvo on the install path
jjerphan May 20, 2025
4bce239
More install path
jjerphan May 20, 2025
cac9ad6
Define `lsplit_track_features` inline
jjerphan May 20, 2025
3e00e99
Parse respodata.json
jjerphan May 20, 2025
2748de1
Complete implementation of `solver::resolvo::Database`
jjerphan May 21, 2025
71e83ac
Unify naming of DatabaseVariant instances
jjerphan May 22, 2025
0a2a79d
Ignore packages signatures when using resolvo for now
jjerphan May 22, 2025
01f324f
Only create solver::libsolv::Database if needed
jjerphan May 22, 2025
a790564
Revert change made to test_env_lockfile.cpp
jjerphan May 22, 2025
61a4d1a
Post rebase unification
jjerphan May 28, 2025
0c3136c
Rename `Mapping` to `bijective_map`
jjerphan May 28, 2025
d9c2e01
Align APIs
jjerphan May 28, 2025
107188f
Post rebase adaptations
jjerphan Jun 11, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/static_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ jobs:
# Special values for running the feedstock with a local source
export FEEDSTOCK_ROOT="${PWD}"
export CI="local"
# Patch: add resolvo-cpp as a host dependency
sed -i 's/ - fmt/ - fmt\n - resolvo-cpp==0.1.0/' recipe/meta.yaml
# For OSX not using Docker
export CONDA_BLD_PATH="${PWD}/build_artifacts"
mkdir -p "${CONDA_BLD_PATH}"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/unix_impl.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:
--preset mamba-unix-shared-${{ inputs.build_type }} \
-D CMAKE_CXX_COMPILER_LAUNCHER=sccache \
-D CMAKE_C_COMPILER_LAUNCHER=sccache \
-D MAMBA_WARNING_AS_ERROR=ON \
-D MAMBA_WARNING_AS_ERROR=OFF \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should remove this before merging.

-D BUILD_LIBMAMBAPY=OFF \
-D ENABLE_MAMBA_ROOT_PREFIX_FALLBACK=OFF
cmake --build build/ --parallel
Expand Down
1 change: 1 addition & 0 deletions dev/environment-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ dependencies:
- libarchive>=3.8 lgpl_*
- libcurl >=7.86
- libsodium
- resolvo-cpp==0.1.0
- libsolv >=0.7.18
- nlohmann_json
- reproc-cpp >=14.2.4.post0
Expand Down
1 change: 1 addition & 0 deletions dev/environment-micromamba-static.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ dependencies:
- simdjson-static >=3.3.0
- spdlog
- fmt >=11.1.0
- resolvo-cpp==0.1.0
- libsolv-static >=0.7.24
- yaml-cpp-static >=0.8.0
- reproc-static >=14.2.4.post0
Expand Down
10 changes: 10 additions & 0 deletions libmamba/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,9 @@ set(
${LIBMAMBA_SOURCE_DIR}/solver/libsolv/repo_info.cpp
${LIBMAMBA_SOURCE_DIR}/solver/libsolv/solver.cpp
${LIBMAMBA_SOURCE_DIR}/solver/libsolv/unsolvable.cpp
# Solver resolvo implementation
${LIBMAMBA_SOURCE_DIR}/solver/resolvo/database.cpp
${LIBMAMBA_SOURCE_DIR}/solver/resolvo/solver.cpp
# Artifacts validation
${LIBMAMBA_SOURCE_DIR}/validation/errors.cpp
${LIBMAMBA_SOURCE_DIR}/validation/keys.cpp
Expand Down Expand Up @@ -350,6 +353,9 @@ set(
${LIBMAMBA_INCLUDE_DIR}/mamba/solver/libsolv/repo_info.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/solver/libsolv/solver.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/solver/libsolv/unsolvable.hpp
# Solver resolvo implementation
${LIBMAMBA_INCLUDE_DIR}/mamba/solver/resolvo/database.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/solver/resolvo/solver.hpp
# Artifacts validation
${LIBMAMBA_INCLUDE_DIR}/mamba/validation/errors.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/validation/keys.hpp
Expand Down Expand Up @@ -432,6 +438,8 @@ find_package(yaml-cpp CONFIG REQUIRED)
find_package(reproc CONFIG REQUIRED)
find_package(reproc++ CONFIG REQUIRED)
find_package(Libsolv MODULE REQUIRED)
find_package(Resolvo CONFIG REQUIRED)

add_subdirectory(ext/solv-cpp)

macro(libmamba_create_target target_name linkage output_name)
Expand Down Expand Up @@ -487,6 +495,7 @@ macro(libmamba_create_target target_name linkage output_name)
solv::libsolv_static
solv::libsolvext_static
solv::cpp
Resolvo::Resolvo
)

if(UNIX)
Expand Down Expand Up @@ -633,6 +642,7 @@ macro(libmamba_create_target target_name linkage output_name)
solv::libsolv
solv::libsolvext
solv::cpp
Resolvo::Resolvo
)
# CMake 3.17 provides a LibArchive::LibArchive target that could be used instead of
# LIBRARIES/INCLUDE_DIRS
Expand Down
9 changes: 8 additions & 1 deletion libmamba/include/mamba/api/channel_loader.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,20 @@
#define MAMBA_API_CHANNEL_LOADER_HPP

#include "mamba/core/error_handling.hpp"
#include "mamba/solver/resolvo/database.hpp"
#include "mamba/solver/solver_factory.hpp"

namespace mamba
{
namespace solver::libsolv
{
class Database;
}

namespace solver::resolvo
{
class Database;
}
class Context;
class ChannelContext;
class MultiPackageCache;
Expand All @@ -30,7 +37,7 @@ namespace mamba
auto load_channels(
Context& ctx,
ChannelContext& channel_context,
solver::libsolv::Database& database,
solver::DatabaseVariant& database,
MultiPackageCache& package_caches
) -> expected_t<void, mamba_aggregated_error>;

Expand Down
7 changes: 6 additions & 1 deletion libmamba/include/mamba/api/repoquery.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

#include "mamba/api/configuration.hpp"
#include "mamba/core/query.hpp"
#include "mamba/solver/solver_factory.hpp"

namespace mamba
{
Expand All @@ -23,7 +24,7 @@ namespace mamba
};

[[nodiscard]] auto make_repoquery(
solver::libsolv::Database& database,
solver::DatabaseVariant& database,
QueryType type,
QueryResultFormat format,
const std::vector<std::string>& queries,
Expand All @@ -32,6 +33,10 @@ namespace mamba
std::ostream& out
) -> bool;

[[nodiscard]] auto
repoquery_init(Context& ctx, Configuration& config, QueryResultFormat format, bool use_local)
-> solver::DatabaseVariant;

[[nodiscard]] auto repoquery(
Configuration& config,
QueryType type,
Expand Down
1 change: 1 addition & 0 deletions libmamba/include/mamba/core/context.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ namespace mamba
bool experimental = false;
bool experimental_repodata_parsing = true;
bool experimental_matchspec_parsing = false;
bool experimental_resolvo_solver = false;
bool debug = false;
bool use_uv = false;

Expand Down
18 changes: 14 additions & 4 deletions libmamba/include/mamba/core/package_database_loader.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

#include "mamba/core/error_handling.hpp"
#include "mamba/solver/libsolv/repo_info.hpp"
#include "mamba/solver/resolvo/database.hpp"
#include "mamba/specs/channel.hpp"

namespace mamba
Expand All @@ -22,18 +23,27 @@ namespace mamba
class Database;
}

namespace solver::resolvo
{
class Database;
}

void add_spdlog_logger_to_database(solver::libsolv::Database& database);

auto load_subdir_in_database( //
const Context& ctx,
solver::libsolv::Database& database,
std::variant<
std::reference_wrapper<solver::libsolv::Database>,
std::reference_wrapper<solver::resolvo::Database>> database,
const SubdirIndexLoader& subdir
) -> expected_t<solver::libsolv::RepoInfo>;
) -> expected_t<void>;

auto load_installed_packages_in_database(
const Context& ctx,
solver::libsolv::Database& database,
std::variant<
std::reference_wrapper<solver::libsolv::Database>,
std::reference_wrapper<solver::resolvo::Database>> database,
const PrefixData& prefix
) -> solver::libsolv::RepoInfo;
) -> expected_t<void>;
}
#endif
12 changes: 6 additions & 6 deletions libmamba/include/mamba/core/transaction.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
#include "mamba/core/package_cache.hpp"
#include "mamba/core/prefix_data.hpp"
#include "mamba/fs/filesystem.hpp"
#include "mamba/solver/libsolv/database.hpp"
#include "mamba/solver/solution.hpp"
#include "mamba/solver/solver_factory.hpp"
#include "mamba/specs/match_spec.hpp"
#include "mamba/specs/package_info.hpp"

Expand All @@ -36,15 +36,15 @@ namespace mamba

MTransaction(
const Context& ctx,
solver::libsolv::Database& database,
solver::DatabaseVariant& database,
std::vector<specs::PackageInfo> pkgs_to_remove,
std::vector<specs::PackageInfo> pkgs_to_install,
MultiPackageCache& caches
);

MTransaction(
const Context& ctx,
solver::libsolv::Database& database,
solver::DatabaseVariant& database,
const solver::Request& request,
solver::Solution solution,
MultiPackageCache& caches
Expand All @@ -53,7 +53,7 @@ namespace mamba
// Only use if the packages have been solved previously already.
MTransaction(
const Context& ctx,
solver::libsolv::Database& database,
solver::DatabaseVariant& database,
std::vector<specs::PackageInfo> packages,
MultiPackageCache& caches
);
Expand Down Expand Up @@ -90,15 +90,15 @@ namespace mamba

MTransaction create_explicit_transaction_from_urls(
const Context& ctx,
solver::libsolv::Database& database,
solver::DatabaseVariant& database,
const std::vector<std::string>& urls,
MultiPackageCache& package_caches,
std::vector<detail::other_pkg_mgr_spec>& other_specs
);

MTransaction create_explicit_transaction_from_lockfile(
const Context& ctx,
solver::libsolv::Database& database,
solver::DatabaseVariant& database,
const fs::u8path& env_lockfile_path,
const std::vector<std::string>& categories,
MultiPackageCache& package_caches,
Expand Down
62 changes: 62 additions & 0 deletions libmamba/include/mamba/solver/database.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
// Copyright (c) 2024, QuantStack and Mamba Contributors
//
// Distributed under the terms of the BSD 3-Clause License.
//
// The full license is in the file LICENSE, distributed with this software.

#ifndef MAMBA_SOLVER_DATABASE_HPP
#define MAMBA_SOLVER_DATABASE_HPP

#include <string>
#include <variant>
#include <vector>

#include "mamba/fs/filesystem.hpp"
#include "mamba/specs/channel.hpp"
#include "mamba/specs/match_spec.hpp"
#include "mamba/specs/package_info.hpp"

namespace mamba::solver
{
class Database
{
public:

virtual ~Database() = default;

virtual void add_repo_from_repodata_json(
const fs::u8path& filename,
const std::string& repo_url,
const std::string& channel_id,
bool verify_artifacts = false
) = 0;

virtual void add_repo_from_packages(
const std::vector<specs::PackageInfo>& packages,
const std::string& repo_name,
bool pip_as_python_dependency = false
) = 0;

virtual void set_installed_repo(const std::string& repo_name) = 0;

virtual bool has_package(const specs::MatchSpec& spec) = 0;
};

namespace libsolv
{
class Database;
}

namespace resolvo
{
class Database;
}

using DatabaseVariant = std::variant<libsolv::Database, resolvo::Database>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a classical inheritance tree (or type erasure) would be more flexible and extensible on the long run than a variant.
I understand this requires further refactoring (up to the APIs used by mamba / micromamba), but I think it is definitely worth doing it. Besides, using abstractions in the higher level should reduce the "common" APIs of the different database classes and simplify the abstraction itself.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is temporary for now, and this can later be changed so that the plan from #3387 is implemented.


// Remove or comment out the inline database_has_package function if DatabaseVariant is not
// visible or causes errors inline auto database_has_package(DatabaseVariant& database, const
// specs::MatchSpec& spec) -> bool;
}

#endif // MAMBA_SOLVER_DATABASE_HPP
24 changes: 24 additions & 0 deletions libmamba/include/mamba/solver/database_utils.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#ifndef MAMBA_SOLVER_DATABASE_UTILS_HPP
#define MAMBA_SOLVER_DATABASE_UTILS_HPP

#include <stdexcept>

#include "mamba/solver/database.hpp"

namespace mamba::solver
{
inline bool database_has_package(DatabaseVariant& database, const specs::MatchSpec& spec)
{
if (auto* libsolv_db = std::get_if<libsolv::Database>(&database))
{
return libsolv_db->has_package(spec);
}
else if (auto* resolvo_db = std::get_if<resolvo::Database>(&database))
{
return resolvo_db->has_package(spec);
}
throw std::runtime_error("Invalid database variant");
}
}

#endif // MAMBA_SOLVER_DATABASE_UTILS_HPP
14 changes: 14 additions & 0 deletions libmamba/include/mamba/solver/libsolv/database.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,20 @@ namespace mamba::solver::libsolv
template <typename Func>
void for_each_package_depending_on(const specs::MatchSpec& ms, Func&&);

bool has_package(const specs::MatchSpec& spec)
{
bool found = false;
for_each_package_matching(
spec,
[&](const auto&)
{
found = true;
return util::LoopControl::Break;
}
);
return found;
}

/**
* An access control wrapper.
*
Expand Down
Loading
Loading