Add new subcommands for forest-cli snasphot
#6128
Conversation
WalkthroughAdds RPCs and CLI commands to monitor and cancel long-running chain snapshot exports, introduces ApiExportStatus/ApiExportResult types and global export state, instruments chain streaming for progress tracking, and makes one block header field public. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant CLI
participant RPC as "RPC Server"
participant Export as "Export Task"
participant Status as "CHAIN_EXPORT_STATUS"
participant Cancel as "CANCELLATION_TOKEN"
CLI->>RPC: ForestChainExport(params)
RPC->>Status: start_export()
RPC->>Export: start export_to_forest_car(track_progress=true)
par Export completes or Cancel signalled
Export-->>RPC: Result (hash / error)
Cancel-->>RPC: cancellation signal
end
RPC->>Status: end_export() / mark cancelled
alt Completed
RPC-->>CLI: ApiExportResult::Done(Optional<Hash>)
else Cancelled
RPC-->>CLI: ApiExportResult::Cancelled
end
sequenceDiagram
autonumber
participant CLI
participant RPC as "RPC Server"
participant Status as "CHAIN_EXPORT_STATUS"
CLI->>RPC: ForestChainExportStatus (--wait?)
alt --wait
loop while exporting
RPC-->>CLI: ApiExportStatus { progress, exporting, cancelled, start_time }
end
else not waiting
RPC-->>CLI: ApiExportStatus
end
CLI->>RPC: ForestChainExportCancel
RPC->>Status: request cancellation (notify token)
RPC-->>CLI: bool (cancellation requested)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
elmattic
requested review from
akaladarshi and
hanabi1224
and removed request for
a team
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (7)
src/rpc/types/mod.rs (1)
33-33: Add safety notes for status fields; otherwise good
- Chrono DateTime with serde is fine; lotus_json wrapper will pass through.
- Consider adding #[serde(rename_all = "PascalCase")] for consistency with adjacent types (optional).
Also applies to: 566-582
src/chain/mod.rs (1)
31-36: Cancellation notifier risks missed or stale signalsUsing a global Notify without a per-job “generation” can race:
- If cancel is called before the export starts awaiting, notify_waiters() does nothing and the cancel is lost (race).
- If switched to notify_one() to avoid loss, a late cancel may leave a pending permit and cancel the next export spuriously.
Prefer a per-export token:
- Use tokio_util::sync::CancellationToken stored for the active export (e.g., in a Mutex<Option>). On start, create/replace the token; in cancel, call token.cancel(); in the export, select on token.cancelled().
src/cli/subcommands/snapshot_cmd.rs (3)
142-147: Fix progress template: use current bytes, not total.You're updating position but not setting a length, so
{binary_total_bytes}won’t show progress. Use{binary_bytes}.- ProgressStyle::with_template( - "{spinner} {msg} {binary_total_bytes} written in {elapsed} ({binary_bytes_per_sec})", - ) + ProgressStyle::with_template( + "{spinner} {msg} {binary_bytes} written in {elapsed} ({binary_bytes_per_sec})", + )
212-237: Throttle polling; cleanly clear the bar on early exit.Polling every 10ms will hammer the RPC. Use ~500–1000ms. Also abandon/finish the bar when returning early to avoid a stale line.
- if !result.exporting { - return Ok(()); - } + if !result.exporting { + pb.abandon(); + return Ok(()); + } @@ - tokio::time::sleep(Duration::from_millis(10)).await; + tokio::time::sleep(Duration::from_millis(750)).await;
253-255: Improve UX on no-op cancel.Print a message when there’s nothing to cancel.
- if result { - println!("Export cancelled."); - } + if result { + println!("Export cancelled."); + } else { + println!("No export in progress."); + }src/ipld/util.rs (2)
32-34: Prefer explicit Mutex::new for clarity.Avoid relying on Into for Mutex construction; it’s less idiomatic and may surprise readers.
-pub static CHAIN_EXPORT_STATUS: LazyLock<Mutex<ExportStatus>> = - LazyLock::new(|| ExportStatus::default().into()); +pub static CHAIN_EXPORT_STATUS: LazyLock<Mutex<ExportStatus>> = + LazyLock::new(|| Mutex::new(ExportStatus::default()));
298-300: Use consistent epoch access.Elsewhere this module and callers use block.epoch. Mixing block.uncached.epoch bypasses invariants and is inconsistent.
- update_epoch(block.uncached.epoch); + update_epoch(block.epoch);
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
CHANGELOG.md(1 hunks)src/blocks/header.rs(1 hunks)src/chain/mod.rs(2 hunks)src/cli/subcommands/snapshot_cmd.rs(3 hunks)src/ipld/util.rs(6 hunks)src/rpc/methods/chain.rs(6 hunks)src/rpc/mod.rs(1 hunks)src/rpc/types/mod.rs(2 hunks)
🧰 Additional context used
🧬 Code graph analysis (3)
src/rpc/methods/chain.rs (2)
src/ipld/util.rs (3)
cancel_export(57-61)end_export(52-55)start_export(43-50)src/chain/mod.rs (3)
export(52-52)export(58-67)export_v2(71-133)
src/ipld/util.rs (2)
src/blocks/header.rs (3)
cid(69-71)cid(304-306)new(282-288)src/blocks/tipset.rs (5)
cid(40-48)new(242-260)new(494-511)epoch(306-308)epoch(543-545)
src/cli/subcommands/snapshot_cmd.rs (1)
src/rpc/methods/chain.rs (18)
chain(375-375)chain(413-413)handle(145-147)handle(161-173)handle(187-202)handle(217-231)handle(246-278)handle(291-299)handle(312-323)handle(336-439)handle(452-465)handle(478-487)handle(500-543)handle(556-580)handle(596-605)handle(620-625)handle(640-678)handle(693-709)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: Build forest binaries on Linux AMD64
- GitHub Check: tests-release
- GitHub Check: tests
- GitHub Check: Build Ubuntu
- GitHub Check: cargo-publish-dry-run
- GitHub Check: Build MacOS
- GitHub Check: All lint checks
🔇 Additional comments (5)
src/blocks/header.rs (1)
239-239: Clarify necessity of publicuncachedfield (src/blocks/header.rs:239)
- Currently only used internally (e.g., src/ipld/util.rs:299); if it’s not consumed by external crates, restrict visibility to
pub(crate)or expose via a getter.- Fully public access allows direct mutation, risking cached CID/signature state becoming inconsistent.
- Please document the snapshot-export use case that requires this field to be fully public.
CHANGELOG.md (1)
38-39: Changelog entry looks goodEntry correctly reflects the new CLI subcommands under “Added”.
src/rpc/mod.rs (1)
86-88: Method registration looks correctBoth ForestChainExportStatus and ForestChainExportCancel are properly registered under the chain vertical.
src/chain/mod.rs (1)
167-169: Progress tracking enablementEnabling track_progress(true) here is appropriate to surface export status. Ensure the tracker correctly bounds values [0,1] to avoid NaN/Inf in RPC (see related comment in chain RPC).
src/cli/subcommands/snapshot_cmd.rs (1)
119-123: No changes required: DateTime::from_timestamp(secs, nanos) is provided by chrono 0.4.x.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
src/cli/subcommands/snapshot_cmd.rs (3)
164-166: Previous concern remains: Duration::MAX in timeout.As flagged in the previous review,
Duration::MAXcan overflow or misbehave in timer math. Consider using a large finite timeout.Apply this diff to use a finite timeout:
- let export_result = client - .call(ForestChainExport::request((params,))?.with_timeout(Duration::MAX)) - .await?; + let export_result = client + .call(ForestChainExport::request((params,))?.with_timeout(Duration::from_secs(6 * 60 * 60))) + .await?;
172-178: Previous concern remains: Persist before checksum, avoid let-chains.As flagged in the previous review:
- Persisting the file after writing the checksum can leave an orphaned checksum file if persist fails.
- The if-let chain syntax may pose MSRV concerns depending on the project's minimum Rust version.
- The
clone()on line 172 is unnecessary sinceexport_resultis only used once more in the match statement below.Apply this diff:
- if !dry_run && let ApiExportResult::Done(hash_opt) = export_result.clone() { - if let Some(hash) = hash_opt { - save_checksum(&output_path, hash).await?; - } - - temp_path.persist(output_path)?; - } + if !dry_run { + match &export_result { + ApiExportResult::Done(hash_opt) => { + // Persist first to avoid orphaned checksum on failure + temp_path.persist(&output_path)?; + if let Some(hash) = hash_opt { + save_checksum(&output_path, hash.clone()).await?; + } + } + ApiExportResult::Cancelled => { + // No file to persist on cancel + } + } + }
239-244: Previous concern remains: Duration::MAX in timeout.As flagged in the previous review,
Duration::MAXcan overflow or misbehave in timer math. For a simple status query, a shorter timeout is more appropriate.Apply this diff:
- let result = client - .call(ForestChainExportStatus::request(())?.with_timeout(Duration::MAX)) - .await?; + let result = client + .call(ForestChainExportStatus::request(())?.with_timeout(Duration::from_secs(30))) + .await?;
🧹 Nitpick comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)
246-256: Add user feedback when no export is running.When
resultisfalse(no export is currently running), the command exits silently. Users would benefit from explicit feedback in this case.Apply this diff:
let result = client .call( ForestChainExportCancel::request(())?.with_timeout(Duration::from_secs(30)), ) .await?; - if result { + if result { println!("Export cancelled."); + } else { + println!("No export is currently running."); } Ok(())
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/cli/subcommands/snapshot_cmd.rs(3 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
src/cli/subcommands/snapshot_cmd.rs (1)
src/rpc/methods/chain.rs (16)
handle(145-147)handle(161-173)handle(187-202)handle(217-231)handle(246-278)handle(291-299)handle(312-323)handle(336-439)handle(452-465)handle(478-487)handle(500-543)handle(556-580)handle(596-605)handle(620-625)handle(640-678)handle(693-709)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: tests-release
- GitHub Check: tests
- GitHub Check: Build forest binaries on Linux AMD64
- GitHub Check: Build MacOS
- GitHub Check: Build Ubuntu
- GitHub Check: cargo-publish-dry-run
- GitHub Check: All lint checks
🔇 Additional comments (3)
src/cli/subcommands/snapshot_cmd.rs (3)
10-10: LGTM!The new import is necessary to handle the export result variants (Done/Cancelled).
46-53: LGTM!The new command variants are well-structured and align with the PR objectives.
180-188: LGTM!The user feedback messages for completion and cancellation are appropriate.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/tool/subcommands/api_cmd/test_snapshots_ignored.txt(1 hunks)
🧰 Additional context used
🪛 GitHub Actions: Script linters
src/tool/subcommands/api_cmd/test_snapshots_ignored.txt
[error] 1-1: git diff --exit-code reported differences in test_snapshots_ignored.txt; exit code 1. This indicates the snapshots are not sorted/added as expected by 'make sort-lists'.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: tests
- GitHub Check: tests-release
- GitHub Check: cargo-publish-dry-run
- GitHub Check: Build Ubuntu
- GitHub Check: Build MacOS
- GitHub Check: Build forest binaries on Linux AMD64
- GitHub Check: All lint checks
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (4)
src/cli/subcommands/snapshot_cmd.rs (4)
214-241: Add overall timeout to prevent indefinite polling.The polling loop has no maximum duration, so if the server becomes unresponsive while an export is running, this will poll indefinitely.
Apply this diff:
+ let start = std::time::Instant::now(); + let max_wait = Duration::from_secs(12 * 60 * 60); // 12 hours max loop { + if start.elapsed() > max_wait { + pb.abandon_with_message("Export timed out"); + anyhow::bail!("Export status polling timed out after {} hours", max_wait.as_secs() / 3600); + } let result = client
175-191: Persist file before writing checksum; also avoid if-let chains.The current order (checksum → persist) can orphan the checksum file if persist fails. Additionally, the if-let chain at line 175 may not compile on older Rust versions.
Apply this diff to reorder and restructure:
- if !dry_run && let ApiExportResult::Done(hash_opt) = export_result.clone() { - if let Some(hash) = hash_opt { - save_checksum(&output_path, hash).await?; - } - - temp_path.persist(output_path)?; - } + if !dry_run { + match export_result.clone() { + ApiExportResult::Done(hash_opt) => { + // Persist first to prevent orphaned checksum on failure + temp_path.persist(&output_path)?; + if let Some(hash) = hash_opt { + save_checksum(&output_path, hash).await?; + } + } + ApiExportResult::Cancelled => { /* no file to persist */ } + } + }
227-228: Add completion message when export finishes.When
!result.exportingbecomes true (export finished), the code returns silently without updating the progress bar or informing the user.Apply this diff:
if !result.exporting { + pb.finish_with_message("Export completed"); return Ok(()); }
230-230: Clamp progress value defensively.If the server returns an invalid progress value (e.g., negative or > 1.0), the cast could produce unexpected results.
Apply this diff:
- let position = (result.progress * 10000.0).trunc() as u64; + let position = (result.progress.clamp(0.0, 1.0) * 10000.0).trunc() as u64;
🧹 Nitpick comments (3)
src/cli/subcommands/snapshot_cmd.rs (3)
236-236: Reduce polling frequency.A 10ms sleep interval allows up to 100 polls per second, which is unnecessarily aggressive and wastes CPU/network resources.
Apply this diff to use a more reasonable interval:
- tokio::time::sleep(Duration::from_millis(10)).await; + tokio::time::sleep(Duration::from_millis(500)).await;
247-247: Consider structured output format.The Debug format (
{:?}) isn't user-friendly or easily parsable. Consider supporting a--format jsonflag (as suggested by @hanabi1224) or using a custom Display implementation.Example for custom Display:
println!("Export status:"); println!(" Progress: {:.1}%", result.progress * 100.0); println!(" Exporting: {}", result.exporting); println!(" Cancelled: {}", result.cancelled); println!(" Started: {}", result.start_time);
257-259: Provide feedback when no export is in progress.When
resultis false (no active export), the command prints nothing, leaving the user uncertain about what happened.Apply this diff:
if result { println!("Export cancelled."); + } else { + println!("No export in progress."); }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/cli/subcommands/snapshot_cmd.rs(3 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
src/cli/subcommands/snapshot_cmd.rs (1)
src/rpc/methods/chain.rs (16)
handle(145-169)handle(212-224)handle(238-253)handle(268-282)handle(297-329)handle(342-350)handle(363-374)handle(387-490)handle(503-516)handle(529-538)handle(551-594)handle(607-631)handle(647-656)handle(671-676)handle(691-729)handle(744-760)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: cargo-publish-dry-run
- GitHub Check: Build MacOS
- GitHub Check: Build Ubuntu
- GitHub Check: tests
- GitHub Check: tests-release
- GitHub Check: Build forest binaries on Linux AMD64
- GitHub Check: All lint checks
🔇 Additional comments (3)
src/cli/subcommands/snapshot_cmd.rs (3)
10-10: LGTM!The import of
ApiExportResultis necessary for handling the new export result variants (Done/Cancelled).
46-53: LGTM!The new subcommand variants align with the PR objectives for exposing export status and cancellation controls.
164-169: LGTM!The export RPC call correctly uses a 30-second timeout, which is appropriate since the export operation runs asynchronously on the server.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (3)
src/cli/subcommands/snapshot_cmd.rs (3)
164-166: Address the Duration::MAX timeout concern.The previous review comment about avoiding
Duration::MAXremains unaddressed. UsingDuration::MAXcan overflow or misbehave in timer math.Apply this diff to use a large finite timeout:
- let export_result = client - .call(ForestChainExport::request((params,))?.with_timeout(Duration::MAX)) - .await?; + let export_result = client + .call(ForestChainExport::request((params,))?.with_timeout(Duration::from_secs(6 * 60 * 60))) + .await?;Also applies to lines 241-243.
172-178: Persist the file before writing the checksum.The previous review comment about reordering persist and checksum operations remains unaddressed. If
persistfails after writing the checksum, an orphaned checksum file will remain. Additionally, the if-let chain pattern may have MSRV implications.Apply this diff to persist first, then write the checksum:
- if !dry_run && let ApiExportResult::Done(hash_opt) = export_result.clone() { - if let Some(hash) = hash_opt { - save_checksum(&output_path, hash).await?; - } - - temp_path.persist(output_path)?; - } + if !dry_run { + match export_result.clone() { + ApiExportResult::Done(hash_opt) => { + temp_path.persist(&output_path)?; + if let Some(hash) = hash_opt { + save_checksum(&output_path, hash).await?; + } + } + _ => {} + } + }
190-238: Add completion message and defensive progress handling.The previous review comment identified several concerns that remain partially unaddressed:
Lines 224-225: When the export finishes normally (
!result.exporting), no completion message is printed and the progress bar is not updated to 100%. Users won't see clear feedback that the export succeeded.Line 227: The progress calculation lacks defensive bounds checking. If the server returns an invalid progress value (e.g., negative or > 1.0), the cast could produce unexpected results.
No overall timeout: The polling loop could run indefinitely if the server becomes unresponsive.
Apply this diff to address these concerns:
pb.set_style( ProgressStyle::with_template( "[{elapsed_precise}] [{wide_bar}] {percent}% {msg} ", ) .expect("indicatif template must be valid") .progress_chars("#>-"), ); + let start = std::time::Instant::now(); + let max_wait = Duration::from_secs(12 * 60 * 60); // 12 hours max loop { + if start.elapsed() > max_wait { + pb.abandon_with_message("Export timed out"); + anyhow::bail!("Export status polling timed out after {} hours", max_wait.as_secs() / 3600); + } let result = client .call( ForestChainExportStatus::request(())? .with_timeout(Duration::from_secs(30)), ) .await?; if result.cancelled { pb.set_message("Export cancelled"); pb.abandon(); return Ok(()); } if !result.exporting { + pb.set_position(10000); + pb.finish_with_message("Export completed"); return Ok(()); } - let position = (result.progress * 10000.0).trunc() as u64; + let position = (result.progress.clamp(0.0, 1.0) * 10000.0).trunc() as u64; pb.set_position(position); if position == 10000 { break; } tokio::time::sleep(Duration::from_millis(10)).await; } pb.finish_with_message("Export completed"); return Ok(());
🧹 Nitpick comments (2)
src/cli/subcommands/snapshot_cmd.rs (2)
239-246: Consider adding JSON format support.A previous comment on line 50 suggested supporting
--format jsonto facilitate bash scripting. This would allow users to programmatically parse the export status.Based on learnings from past review comments.
248-258: Provide feedback when there's no export to cancel.When
resultisfalse, the user receives no feedback. Consider printing a message to indicate that no export is currently in progress.Apply this diff to improve user feedback:
if result { println!("Export cancelled."); + } else { + println!("No export in progress."); }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/cli/subcommands/snapshot_cmd.rs(3 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
src/cli/subcommands/snapshot_cmd.rs (1)
src/rpc/methods/chain.rs (16)
handle(145-169)handle(212-224)handle(238-253)handle(268-282)handle(297-329)handle(342-350)handle(363-374)handle(387-490)handle(503-516)handle(529-538)handle(551-594)handle(607-631)handle(647-656)handle(671-676)handle(691-729)handle(744-760)
🔇 Additional comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)
46-53: LGTM!The new subcommand variants align well with the PR objectives and linked issue requirements. The
--waitflag forExportStatusprovides the desired polling behavior.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
src/cli/subcommands/snapshot_cmd.rs (2)
164-166: Duration::MAX timeout issue previously flagged.The use of
Duration::MAXfor the export timeout was already discussed in previous review comments. See the existing comment thread for details.
195-243: Multiple refinements needed for the wait loop.Several issues need attention:
Line 229-230: When the export completes naturally (
!result.exporting && !result.cancelled), the code returns without updating the progress bar to 100% or printing a completion message. Users will see the progress stop at some percentage < 100% without confirmation.Line 232: The progress calculation doesn't validate the server's response. If
result.progressis negative or > 1.0 (due to a bug or API mismatch), the cast tou64could produce unexpected values.Line 238: The 10ms sleep is quite aggressive. Consider increasing to 100-500ms to reduce RPC server load while still providing responsive updates.
No overall timeout: If the server becomes unresponsive during the export, this loop will poll indefinitely. Consider adding a maximum wait time (e.g., 12 hours) with a timeout message.
Apply this diff to address these concerns:
pb.set_style( ProgressStyle::with_template( "[{elapsed_precise}] [{wide_bar}] {percent}% {msg} ", ) .expect("indicatif template must be valid") .progress_chars("#>-"), ); + let start = std::time::Instant::now(); + let max_wait = Duration::from_secs(12 * 60 * 60); // 12 hours loop { + if start.elapsed() > max_wait { + pb.abandon_with_message("Export timed out"); + anyhow::bail!("Export status polling timed out after {} hours", max_wait.as_secs() / 3600); + } let result = client .call( ForestChainExportStatus::request(())? .with_timeout(Duration::from_secs(30)), ) .await?; if result.cancelled { pb.set_message("Export cancelled"); pb.abandon(); return Ok(()); } if !result.exporting { + pb.set_position(10000); + pb.finish_with_message("Export completed"); return Ok(()); } - let position = (result.progress * 10000.0).trunc() as u64; + let position = (result.progress.clamp(0.0, 1.0) * 10000.0).trunc() as u64; pb.set_position(position); if position == 10000 { break; } - tokio::time::sleep(Duration::from_millis(10)).await; + tokio::time::sleep(Duration::from_millis(100)).await; } pb.finish_with_message("Export completed"); return Ok(());
🧹 Nitpick comments (2)
src/cli/subcommands/snapshot_cmd.rs (2)
46-53: Consider adding--format jsonoutput option.The new subcommands look good. However, as discussed in the previous comment thread, adding a
--format jsonflag would facilitate bash scripting and automation. Currently, line 249 uses debug formatting ({:?}) for the non-wait case.Consider adding a format flag to both
ExportStatusandExportCancelcommands to support JSON output for better scriptability.
244-251: Improve non-wait status output formatting.Line 249 uses debug formatting (
{:?}) which is not user-friendly for a CLI tool. Consider either:
- Formatting the output in a human-readable way (e.g., "Export Status: In Progress (45%)" or "Export Status: Not Running")
- Implementing the
--format jsonoption discussed earlier to provide structured outputExample for human-readable output:
- println!("{:?}", result); + if result.exporting { + println!("Export in progress: {:.1}%", result.progress * 100.0); + } else if result.cancelled { + println!("Export cancelled"); + } else { + println!("No export in progress"); + }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/cli/subcommands/snapshot_cmd.rs(4 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
src/cli/subcommands/snapshot_cmd.rs (2)
src/rpc/client.rs (2)
client(114-114)request(272-285)src/rpc/reflect/mod.rs (1)
request(250-260)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: All lint checks
- GitHub Check: Build Ubuntu
- GitHub Check: Build forest binaries on Linux AMD64
- GitHub Check: Build MacOS
- GitHub Check: Check
- GitHub Check: cargo-publish-dry-run
- GitHub Check: tests-release
- GitHub Check: tests
🔇 Additional comments (3)
src/cli/subcommands/snapshot_cmd.rs (3)
10-10: LGTM!The import of
ApiExportResultis necessary for the new export status functionality.
173-183: LGTM!The persist-before-checksum logic correctly addresses the previous review concern. The implementation properly handles both
DoneandCancelledcases.
185-194: LGTM!Clear user feedback for both completion and cancellation scenarios.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
src/rpc/methods/chain.rs (1)
503-521: Guard progress calculation against division by zero.When
mutex.initial_epochis 0 (e.g., at startup or if not properly initialized), the division(mutex.epoch as f64) / (mutex.initial_epoch as f64)producesNaN, which will fail JSON serialization and break the API response.Apply this diff to safely compute progress:
- let progress = if mutex.initial_epoch == 0 { - 0.0 - } else { - 1.0 - ((mutex.epoch as f64) / (mutex.initial_epoch as f64)) - }; + let progress = if mutex.initial_epoch > 0 { + let p = 1.0 - ((mutex.epoch as f64) / (mutex.initial_epoch as f64)); + if p.is_finite() { p.clamp(0.0, 1.0) } else { 0.0 } + } else if mutex.exporting { + 0.0 + } else { + 1.0 + };This handles the zero case and clamps the result to
[0.0, 1.0]to prevent out-of-range values.
🧹 Nitpick comments (3)
src/cli/subcommands/snapshot_cmd.rs (2)
195-243: Refine wait-mode completion handling, progress validation, and add polling timeout.The wait mode implementation has several areas that need refinement (some flagged in previous reviews):
Line 230: When the export finishes (
!result.exporting), the code returns immediately without updating the progress bar to 100% or printing a completion message. Users won't see clear confirmation.Line 232: The progress calculation
(result.progress * 10000.0).trunc() as u64doesn't validateresult.progress. If the server returns invalid values (e.g., negative, > 1.0, or NaN), the cast could produce unexpected results.Missing overall timeout: The polling loop has no maximum duration. If the server becomes unresponsive during a long export, this will poll indefinitely.
Apply this diff to address these concerns:
pb.set_style( ProgressStyle::with_template( "[{elapsed_precise}] [{wide_bar}] {percent}% {msg} ", ) .expect("indicatif template must be valid") .progress_chars("#>-"), ); + let start = std::time::Instant::now(); + let max_wait = Duration::from_secs(12 * 60 * 60); // 12 hours max loop { + if start.elapsed() > max_wait { + pb.abandon_with_message("Export timed out"); + anyhow::bail!("Export status polling timed out after {} hours", max_wait.as_secs() / 3600); + } let result = client .call( ForestChainExportStatus::request(())? .with_timeout(Duration::from_secs(30)), ) .await?; if result.cancelled { pb.set_message("Export cancelled"); pb.abandon(); return Ok(()); } if !result.exporting { + pb.set_position(10000); + pb.finish_with_message("Export completed"); return Ok(()); } - let position = (result.progress * 10000.0).trunc() as u64; + let position = (result.progress.clamp(0.0, 1.0) * 10000.0).trunc() as u64; pb.set_position(position); if position == 10000 { break; } tokio::time::sleep(Duration::from_millis(10)).await; } pb.finish_with_message("Export completed"); return Ok(());
244-251: Consider adding JSON output format for programmatic use.The current implementation prints the debug representation of
ApiExportStatus. As noted in previous discussion, supporting a--format jsonflag would facilitate bash scripting and programmatic consumption of the status output.Consider adding a format flag to the
ExportStatusvariant:ExportStatus { /// Wait until it completes and print progress. #[arg(long)] wait: bool, + /// Output format (json, text) + #[arg(long, default_value = "text")] + format: OutputFormat, },Then conditionally serialize the result:
match format { OutputFormat::Json => println!("{}", serde_json::to_string_pretty(&result)?), OutputFormat::Text => println!("{:?}", result), }src/rpc/methods/chain.rs (1)
421-438: Cancellation mechanism can miss or leak signals; consider CancellationToken.The current global
CANCEL_EXPORT.notify_waiters()pattern has race conditions flagged in previous reviews:
- Signal loss: If
notify_waiters()fires before the export task reaches theCANCEL_EXPORT.notified()await point, the signal is missed.- Signal leakage: Switching to
notify_one()as a stopgap still allows a late notification to linger and cancel the next export.Recommended: Use a per-export
tokio_util::sync::CancellationToken:
- Store an
Arc<Mutex<Option<CancellationToken>>>for the active export- On
start_export, create a new token, store it, and clone it into thetokio::select!- In
ForestChainExportCancel, lock, take the token, callcancel(), and return true if a token existedThis ensures each export has its own cancellation lifecycle without signal loss or leakage.
As a minimal stopgap (still imperfect), replace
notify_waiters()withnotify_one()to reduce (but not eliminate) signal loss:- CANCEL_EXPORT.notify_waiters(); + CANCEL_EXPORT.notify_one();But please prioritize moving to
CancellationTokento fully resolve both issues.Also applies to: 473-482, 534-538
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
src/cli/subcommands/snapshot_cmd.rs(4 hunks)src/rpc/methods/chain.rs(6 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
src/rpc/methods/chain.rs (2)
src/ipld/util.rs (3)
cancel_export(57-61)end_export(52-55)start_export(43-50)src/chain/mod.rs (3)
export(52-52)export(58-67)export_v2(71-133)
src/cli/subcommands/snapshot_cmd.rs (2)
src/rpc/client.rs (2)
client(114-114)request(272-285)src/rpc/reflect/mod.rs (1)
request(250-260)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: cargo-publish-dry-run
- GitHub Check: Build MacOS
- GitHub Check: Build Ubuntu
- GitHub Check: All lint checks
- GitHub Check: tests
- GitHub Check: tests-release
- GitHub Check: Build forest binaries on Linux AMD64
🔇 Additional comments (1)
src/rpc/methods/chain.rs (1)
385-385: Breaking API change: return type nowApiExportResult.Both
ForestChainExport(Line 385) andChainExport(Line 609) have changed their return type fromOption<String>toApiExportResult. This is a breaking change for RPC clients that expect the old signature.Ensure that:
- This change is documented in the CHANGELOG
- Clients are notified if this is part of a public API
- API versioning is considered if backward compatibility is required
Based on the PR description, this appears to be an intentional change as part of the feature implementation. Verify that dependent clients and integrations are updated accordingly.
Also applies to: 609-609
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)
164-166: ReplaceDuration::MAXwith a finite RPC timeout.Using
Duration::MAXstill risks overflow in the JSON-RPC timeout machinery (prior feedback flagged this as well). Please switch to a realistic upper bound (e.g., a named constant such as 6 h) and reuse it for the other export calls in this file.- let export_result = client - .call(ForestChainExport::request((params,))?.with_timeout(Duration::MAX)) + const EXPORT_TIMEOUT: Duration = Duration::from_secs(6 * 60 * 60); // 6h cap + let export_result = client + .call(ForestChainExport::request((params,))?.with_timeout(EXPORT_TIMEOUT))Please apply the same timeout constant to the non-wait
export-statuscall and the diff export path below.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/cli/subcommands/snapshot_cmd.rs(4 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: Build MacOS
- GitHub Check: Build Ubuntu
- GitHub Check: cargo-publish-dry-run
- GitHub Check: tests-release
- GitHub Check: tests
- GitHub Check: Build forest binaries on Linux AMD64
- GitHub Check: All lint checks
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)
204-275: Honor--format jsonin--waitmode or reject the combination.Currently
formatis ignored when--waitis set, always rendering a TTY bar. Either:
- Stream JSON status objects each tick, or
- Error out on
--wait --format jsonwith a clear message.If you want, I can draft a minimal JSON-streaming loop (one-line JSON per tick) to keep it script-friendly. Would you prefer that?
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/cli/subcommands/snapshot_cmd.rs(5 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
src/cli/subcommands/snapshot_cmd.rs (2)
src/rpc/client.rs (2)
client(114-114)request(272-285)src/rpc/reflect/mod.rs (1)
request(250-260)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: Build MacOS
- GitHub Check: cargo-publish-dry-run
- GitHub Check: Build Ubuntu
- GitHub Check: Build forest binaries on Linux AMD64
- GitHub Check: tests-release
- GitHub Check: tests
- GitHub Check: All lint checks
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
src/chain/mod.rs(1 hunks)src/rpc/methods/chain.rs(7 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- src/chain/mod.rs
🧰 Additional context used
🧬 Code graph analysis (1)
src/rpc/methods/chain.rs (1)
src/ipld/util.rs (3)
cancel_export(57-61)end_export(52-55)start_export(43-50)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: cargo-publish-dry-run
- GitHub Check: Build MacOS
- GitHub Check: Build Ubuntu
- GitHub Check: Build forest binaries on Linux AMD64
- GitHub Check: tests-release
- GitHub Check: tests
- GitHub Check: All lint checks
🔇 Additional comments (2)
src/rpc/methods/chain.rs (2)
424-493: Export cancellation structure is sound, but depends on CancellationToken fix.The pattern of racing the export against cancellation using
tokio::select!is correct, and the lifecycle (start_export → select → end_export) is properly sequenced. However, this implementation depends on fixing the globalCANCELLATION_TOKENissue flagged above.After applying the per-export token fix, verify that:
- Multiple sequential exports work correctly
- Cancelling one export doesn't affect subsequent exports
- Attempting to cancel when no export is running returns
false
536-545: Cancellation logic is correct but depends on token fix.The logic correctly detects an in-progress export by attempting to acquire the lock. However, with the current global
CANCELLATION_TOKEN, callingcancel()breaks all future exports (see earlier comment).After applying the per-export token fix, this logic will work as intended.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)
233-254: Add missing completion check in polling loop.The loop checks for cancellation (lines 240-244) and breaks when
position >= 10000(lines 249-251), but doesn't explicitly check if the export has completed naturally. Ifresult.exportingbecomes false while progress is less than 1.0 (due to rounding or server behavior), the loop will continue polling indefinitely.Apply this diff to add the completion check:
loop { let result = client .call( ForestChainExportStatus::request(())? .with_timeout(Duration::from_secs(30)), ) .await?; if result.cancelled { pb.set_message("Export cancelled"); pb.abandon(); return Ok(()); } + if !result.exporting { + pb.set_position(10000); + pb.finish_with_message("Export completed"); + return Ok(()); + } let position = (result.progress.clamp(0.0, 1.0) * 10000.0).trunc() as u64; pb.set_position(position); if position >= 10000 { break; } tokio::time::sleep(Duration::from_millis(500)).await; } pb.finish_with_message("Export completed");Note: This issue was raised in a previous review but appears to still be present in the code.
🧹 Nitpick comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)
53-60: Consider documenting JSON output format.While the text format is self-explanatory, it would be helpful to document what fields are included in the JSON output (e.g., exporting, progress, cancelled, start_time) so users know what to expect when using
--format json.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
CHANGELOG.md(1 hunks)src/cli/subcommands/snapshot_cmd.rs(5 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
src/cli/subcommands/snapshot_cmd.rs (2)
src/rpc/client.rs (2)
client(114-114)request(272-285)src/rpc/reflect/mod.rs (1)
request(250-260)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: Build Ubuntu
- GitHub Check: cargo-publish-dry-run
- GitHub Check: Build MacOS
- GitHub Check: All lint checks
- GitHub Check: tests-release
- GitHub Check: tests
- GitHub Check: Build forest binaries on Linux AMD64
🔇 Additional comments (3)
src/cli/subcommands/snapshot_cmd.rs (2)
23-27: LGTM: Format enum is properly defined.The enum correctly derives the necessary traits for clap integration and the variants are appropriately named.
269-280: LGTM: ExportCancel implementation handles all cases correctly.The implementation properly handles both success and failure cases, providing clear feedback to the user in each scenario.
CHANGELOG.md (1)
32-32: LGTM: Changelog entry is clear and follows conventions.The entry properly documents the new subcommands and references the related issue.
2 participants
Summary of changes
Changes introduced in this pull request:
forest-cli snasphot export-statusforest-cli snasphot export-cancelReference issue to close (if applicable)
Closes #6082
Other information and links
Change checklist
Summary by CodeRabbit
New Features
snapshot export-status(supports--waitwith live progress and JSON/Text output) andsnapshot export-cancel.Documentation
Tests