ipfs 0.35.0 #224309
Merged
ipfs 0.35.0 #224309
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
go
krehel
approved these changes
May 21, 2025
Contributor
|
🤖 An automated task has requested bottles to be published to this PR. Please do not push to this PR branch before the bottle commits have been pushed, as this results in a state that is difficult to recover from. If you need to resolve a merge conflict, please use a merge commit. Do not force-push to this PR branch. |
github-actions
bot
added
the
CI-published-bottle-commits
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Created by
brew bumpCreated with
brew bump-formula-pr.Details
release notes
Reprovider.Strategyfor MFSipfs addOptionsImport.*ConfigurationImportProfilesDatastoreMetrics Now Opt-Inipfs addin online modeipfs stats providerBitswapconfiguration optionsRoutingconfiguration optionsOverview
This release brings significant UX and performance improvements to data onboarding, provisioning, and retrieval systems.
New configuration options let you customize the shape of UnixFS DAGs generated during the data import, control the scope of DAGs announced on the Amino DHT, select which delegated routing endpoints are queried, and choose whether to enable HTTP retrieval alongside Bitswap over Libp2p.
Continue reading for more details.
🗣 Discuss
If you have comments, questions, or feedback on this release, please post here.
If you experienced any bugs with the release, please post an issue.
🔦 Highlights
Opt-in HTTP Retrieval client
This release adds experimental support for retrieving blocks directly over HTTPS (HTTP/2), complementing the existing Bitswap over Libp2p.
The opt-in client enables Kubo to use delegated routing results with
/tls/httpmultiaddrs, connecting to HTTPS servers that support Trustless HTTP Gateway's Block Responses (?format=raw,application/vnd.ipld.raw). Fetching blocks via HTTPS (HTTP/2) simplifies infrastructure and reduces costs for storage providers by leveraging HTTP caching and CDNs.To enable this feature for testing and feedback, set:
$ ipfs config --json HTTPRetrieval.Enabled trueSee
HTTPRetrievalfor more details.Dedicated
Reprovider.Strategyfor MFSThe Mutable File System (MFS) in Kubo is a UnixFS filesystem managed with
ipfs filescommands. It supports familiar file operations like cp and mv within a folder-tree structure, automatically updating a MerkleDAG and a "root CID" that reflects the current MFS state. Files in MFS are protected from garbage collection, offering a simpler alternative toipfs pin. This makes it a popular choice for tools like IPFS Desktop and the WebUI.Previously, the
pinnedreprovider strategy required manual pin management: each dataset update meant pinning the new version and unpinning the old one. Now, new strategies—mfsandpinned+mfs—let users limit announcements to data explicitly placed in MFS. This simplifies updating datasets and announcing only the latest version to the Amino DHT.Users relying on the
pinnedstrategy can switch topinned+mfsand use MFS alone to manage updates and announcements, eliminating the need for manual pinning and unpinning. We hope this makes it easier to publish just the data that matters to you.See
Reprovider.Strategyfor more details.Experimental support for MFS as a FUSE mount point
The MFS root (filesystem behind the
ipfs filesAPI) is now available as a read/write FUSE mount point atMounts.MFS. This filesystem is mounted in the same way asMounts.IPFSandMounts.IPNSwhen runningipfs mountoripfs daemon --mount.Note that the operations supported by the MFS FUSE mountpoint are limited, since MFS doesn't store file attributes.
See
Mountsanddocs/fuse.mdfor more details.Grid view in WebUI
The WebUI, accessible at http://127.0.0.1:5001/webui/, now includes support for the grid view on the Files screen:
Enhanced DAG-Shaping Controls
This release advances CIDv1 support by introducing fine-grained control over UnixFS DAG shaping during data ingestion with the
ipfs addcommand.Wider DAG trees (more links per node, higher fanout, larger thresholds) are beneficial for large files and directories with many files, reducing tree depth and lookup latency in high-latency networks, but they increase node size, straining memory and CPU on resource-constrained devices. Narrower trees (lower link count, lower fanout, smaller thresholds) are preferable for smaller directories, frequent updates, or low-power clients, minimizing overhead and ensuring compatibility, though they may increase traversal steps for very large datasets.
Kubo now allows users to act on these tradeoffs and customize the width of the DAG created by
ipfs addcommand.New DAG-Shaping
ipfs addOptionsThree new options allow you to override default settings for specific import operations:
--max-file-links: Sets the maximum number of child links for a single file chunk.--max-directory-links: Defines the maximum number of child entries in a "basic" (single-chunk) directory.Import.UnixFSHAMTDirectorySizeThresholdare converted to HAMT-based (sharded across multiple blocks) structures.--max-hamt-fanout: Specifies the maximum number of child nodes for HAMT internal structures.Persistent DAG-Shaping
Import.*ConfigurationYou can set default values for these options using the following configuration settings:
Import.UnixFSFileMaxLinksImport.UnixFSDirectoryMaxLinksImport.UnixFSHAMTDirectoryMaxFanoutImport.UnixFSHAMTDirectorySizeThresholdUpdated DAG-Shaping
ImportProfilesThe release updated configuration profiles to incorporate these new
Import.*settings:test-cid-v1now includes current defaults as explicitImport.UnixFSFileMaxLinks=174,Import.UnixFSDirectoryMaxLinks=0,Import.UnixFSHAMTDirectoryMaxFanout=256andImport.UnixFSHAMTDirectorySizeThreshold=256KiBtest-cid-v1-wideadopts experimental directory DAG-shaping defaults, increasing the maximum file DAG width from 174 to 1024, HAMT fanout from 256 to 1024, and raising the HAMT directory sharding threshold from 256KiB to 1MiB, aligning with 1MiB file chunks.DatastoreMetrics Now Opt-InTo reduce overhead in the default configuration, datastore metrics are no longer enabled by default when initializing a Kubo repository with
ipfs init.Metrics prefixed with
<dsname>_datastore(e.g.,flatfs_datastore_...,leveldb_datastore_...) are not exposed unless explicitly enabled. For a complete list of affected default metrics, refer toprometheus_metrics_added_by_measure_profile.Convenience opt-in profiles can be enabled at initialization time with
ipfs init --profile:flatfs-measure,pebbleds-measure,badgerds-measureIt is also possible to manually add the
measurewrapper. See examples inDatastore.Specdocumentation.Improved performance of data onboarding
This Kubo release significantly improves both the speed of ingesting data via
ipfs addand announcing newly produced CIDs to Amino DHT.Fast
ipfs addin online modeAdding a large directory of data when
ipfs daemonwas running in online mode took a long time. A significant amount of this time was spent writing to and reading from the persisted provider queue. Due to this, many users had to shut down the daemon and perform data import in offline mode. This release fixes this known limitation, significantly improving the speed ofipfs add.Kubo v0.34:
Kubo v0.35:
Optimized, dedicated queue for providing fresh CIDs
From
kubov0.33.0,Bitswap stopped advertising newly added and received blocks to the DHT. Since
then
boxo/provideris responsible for the first time provide and the recurring reprovide logic. Priorto
v0.35.0, provides and reprovides were handled together in batches, leadingto delays in initial advertisements (provides).
Provides and Reprovides now have separate queues, allowing for immediate
provide of new CIDs and optimised batching of reprovides.
New
Providerconfiguration optionsThis change introduces a new configuration options:
Provider.Enabledis a global flag for disabling both Provider and Reprovider systems (announcing new/old CIDs to amino DHT).Provider.WorkerCountfor limiting the number of concurrent provide operations, allows for fine-tuning the trade-off between announcement speed and system load when announcing new CIDs.Experimental.StrategicProviding. Superseded byProvider.Enabled,Reprovider.IntervalandReprovider.Strategy.Deprecated
ipfs stats providerSince the
ipfs stats providercommand was displaying statistics for bothprovides and reprovides, this command isn't relevant anymore after separating
the two queues.
The successor command is
ipfs stats reprovide, showing the same statistics,but for reprovides only.
New
Bitswapconfiguration optionsBitswap.Libp2pEnableddetermines whether Kubo will use Bitswap over libp2p (both client and server).Bitswap.ServerEnabledcontrols whether Kubo functions as a Bitswap server to host and respond to block requests.Internal.Bitswap.ProviderSearchMaxResultsfor adjusting the maximum number of providers bitswap client should aim at before it stops searching for new ones.New
Routingconfiguration optionsRouting.IgnoreProvidersallows ignoring specific peer IDs when returned by the content routing system as providers of content.HTTPRetrieval.Enabledin setups where Bitswap over Libp2p and HTTP retrieval is served under different PeerIDs.Routing.DelegatedRoutersallows customizing HTTP routers used by Kubo whenRouting.Typeis set toautoorautoclient.New Pebble database format config
This Kubo release provides node operators with more control over Pebble's
FormatMajorVersion. This allows testing a new Kubo release without automatically migrating Pebble datastores, keeping the ability to switch back to older Kubo.When IPFS is initialized to use the pebbleds datastore (opt-in via
ipfs init --profile=pebbleds), the latest pebble database format is configured in the pebble datastore config as"formatMajorVersion". Setting this in the datastore config prevents automatically upgrading to the latest available version when Kubo is upgraded. If a later version becomes available, the Kubo daemon prints a startup message to indicate this. The user can them update the config to use the latest format when they are certain a downgrade will not be necessary.Without the
"formatMajorVersion"in the pebble datastore config, the database format is automatically upgraded to the latest version. If this happens, then it is possible a downgrade back to the previous version of Kubo will not work if new format is not compatible with the pebble datastore in the previous version of Kubo.When installing a new version of Kubo when
"formatMajorVersion"is configured, automatic repository migration (ipfs daemon with --migrate=true) does not upgrade this to the latest available version. This is done because a user may have reasons not to upgrade the pebble database format, and may want to be able to downgrade Kubo if something else is not working in the new version. If the configured pebble database format in the old Kubo is not supported in the new Kubo, then the configured version must be updated and the old Kubo run, before installing the new Kubo.See other caveats and configuration options at
kubo/docs/datastores.md#pebbledsNew environment variables
The
environment-variables.mdwas extended with two new features:Improved Log Output Setting
When stderr and/or stdout options are configured or specified by the
GOLOG_OUTPUTenviron variable, log only to the output(s) specified. For example:GOLOG_OUTPUT="stderr"logs only to stderrGOLOG_OUTPUT="stdout"logs only to stdoutGOLOG_OUTPUT="stderr+stdout"logs to both stderr and stdoutNew Repo Lock Optional Wait
The environment variable
IPFS_WAIT_REPO_LOCKspecifies the amount of time to wait for the repo lock. Set the value of this variable to a string that can be parsed as a golangtime.Duration. For example:If the lock cannot be acquired because someone else has the lock, and
IPFS_WAIT_REPO_LOCKis set to a valid value, then acquiring the lock is retried every second until the lock is acquired or the specified wait time has elapsed.📦️ Important dependency updates
boxoto v0.30.0ipfs-webuito v4.7.0go-ds-pebbleto v0.5.0pebbleto v2.0.3go-libp2p-pubsubto v0.13.1go-libp2p-kad-dhtto v0.33.1 (incl. v0.33.0, v0.32.0, v0.31.0)go-logto v2.6.0p2p-forge/clientto v0.5.1📝 Changelog
Full Changelog
Provider.Enabledflag (#10804) (ipfs/kubo#10804)FormatMajorVersion(#10789) (ipfs/kubo#10789)Provider.WorkerCountandstats reprovide(#10779) (ipfs/kubo#10779)ipfs addandImportoptions for controling UnixFS DAG Width (#10774) (ipfs/kubo#10774)👨👩👧👦 Contributors
View the full release notes at https://github.com/ipfs/kubo/releases/tag/v0.35.0.