Skip to content

Support various formats for progress rate string #228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wsshin wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Support various formats for progress rate string #228

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
2b75515
Implement various rate formats including fraction
wsshin Jan 10, 2022
205e7ff
Fix names of exported AbstractRateFormat types
wsshin Jan 10, 2022
e86f0dc
Add rate_format to Progress
wsshin Jan 10, 2022
c719788
Make tests pass
wsshin Jan 10, 2022
f24a42a
Define default constructor for PercentRateFromat()
wsshin Jan 10, 2022
0f85b00
Improve formatting for PercentRateFormat
wsshin Jan 16, 2022
dd5d2ce
Simplify formatting for IntegerRateFormat
wsshin Jan 16, 2022
6ce9b93
Add tests for rate format
wsshin Jan 16, 2022
77e5ea2
Add usages for different rate formats in README
wsshin Jan 16, 2022
dce00d7
Merge branch 'master' into fraction
wsshin Apr 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,29 @@ The first incantation will use a minimum update interval of 1 second, and show t
final duration. If your computation runs so quickly that it never needs to show progress,
no extraneous output will be displayed.

Different formats for progress rate string are supported:
```julia
# Empty string
@showprogress 1 "Computing..." EmptyRateFormat() for i in 1:107
sleep(0.1)
end

# 2 digits below decimal point (e.g., "2.80%")
@showprogress 1 "Computing..." PercentRateFormat(2) for i in 1:107
sleep(0.1)
end

# Fraction (e.g., "3/107")
@showprogress 1 "Computing..." FractionRateFormat() for i in 1:107
sleep(0.1)
end

# Integer (e.g., "3 of 107")
@showprogress 1 "Computing..." IntegerRateFormat() for i in 1:107
sleep(0.1)
end
```

The `@showprogress` macro wraps a `for` loop, comprehension, `@distributed` for loop, or
`map`/`pmap`/`reduce` as long as the object being iterated over implements the `length`
method and will handle `continue` correctly.
Expand Down
46 changes: 43 additions & 3 deletions src/ProgressMeter.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ using Printf: @sprintf
using Distributed

export Progress, ProgressThresh, ProgressUnknown, BarGlyphs, next!, update!, cancel, finish!, @showprogress, progress_map, progress_pmap, ijulia_behavior
export EmptyRateFormat, PercentRateFormat, FractionRateFormat, IntegerRateFormat

"""
`ProgressMeter` contains a suite of utilities for displaying progress
Expand Down Expand Up @@ -46,6 +47,33 @@ function BarGlyphs(s::AbstractString)
return BarGlyphs(glyphs...)
end

abstract type AbstractRateFormat end
function rate_string end

struct EmptyRateFormat <: AbstractRateFormat end
rate_string(c::Int, n::Int, ::EmptyRateFormat) = ""

struct PercentRateFormat <: AbstractRateFormat
digits::Int # number of digits below decimal point
end
PercentRateFormat() = PercentRateFormat(0)
function rate_string(c::Int, n::Int, p::PercentRateFormat)
str = string(round(100c/n, digits=p.digits)) # e.g., round(10, digits=0) = 10.0
i, f = split(str, ".") # integral, fractional parts

rate_str = lpad(i, 3)
p.digits≠0 && (rate_str *= "." * rpad(f, p.digits, "0"))
rate_str *= "%"

return rate_str
end

struct FractionRateFormat <: AbstractRateFormat end
rate_string(c::Int, n::Int, ::FractionRateFormat) = lpad(c, length(string(n))) * "/" * string(n)

struct IntegerRateFormat <: AbstractRateFormat end
rate_string(c::Int, n::Int, ::IntegerRateFormat) = lpad(c, length(string(n))) * " of " * string(n)

"""
`prog = Progress(n; dt=0.1, desc="Progress: ", color=:green,
output=stderr, barlen=tty_width(desc), start=0)` creates a progress meter for a
Expand All @@ -66,6 +94,7 @@ mutable struct Progress <: AbstractProgress
tlast::Float64
printed::Bool # true if we have issued at least one status update
desc::String # prefix to the percentage, e.g. "Computing..."
rate_format::AbstractRateFormat
Copy link
Collaborator

@martinholters martinholters Feb 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we introduce a type parameter to Progress so that rate_format can be concretely typed? Otherwise, the calls to rate_string would do dynamic dispatch. But maybe that is insignificant performance-wise compared to the IO. Could you benchmark an empty loop to see whether this PR incurs any relevant performance hit?

Copy link
Contributor

@DilumAluthge DilumAluthge Feb 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooof. Yeah having the dynamic dispatch inside a hot loop sounds like a bad idea.

Instead of having rate_format::AbstractRateFormat, perhaps we could have a concretely-typed format_string::String?

Copy link
Author

@wsshin wsshin Feb 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the master branch:

julia> p = ProgressMeter.Progress(10000000);

julia> p.rate_format  # master doesn't support rate format
ERROR: type Progress has no field rate_format
...

julia> @btime ProgressMeter.next!($p)
Progress: 100%|███████████████████████████████████████████████████████████████████████████████████████| Time: 0:00:02
  16.335 ns (0 allocations: 0 bytes)

With the present PR:

julia> p = ProgressMeter.Progress(10000000);

julia> p.rate_format  # present PR supports rate format
PercentRateFormat(0)

julia> @btime ProgressMeter.next!($p)
Progress: 100%|███████████████████████████████████████████████████████████████████████████████████████| Time: 0:00:02
  15.517 ns (0 allocations: 0 bytes)

There doesn't seem any noticeable performance degradation.

Copy link
Owner

@timholy timholy Feb 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More informative might be something like this:

julia> p = ProgressMeter.Progress(10000000; dt=0.001);

julia> @bprofile ProgressMeter.next!($p)
Progress: 100%|█████████████████████████████████████████████████████████████████████████████████| Time: 0:00:02
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  5.342 ns … 624.207 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     6.455 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   8.209 ns ±  15.537 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▄█▄▄▂                                                       ▁
  ██████▆▅▆▇▆▆▆▅▅▅▄▅▃▃▁▄▃▁▃▄▃▃▃▃▃▁▁▁▃▃▄▁▄▁▃▁▄▃▁▃▁▃▁▁▄▁▄▅▅▄▄▄▄ █
  5.34 ns      Histogram: log(frequency) by time      55.2 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

Presumably the dispatch is only triggered when it prints?

barlen::Union{Int,Nothing} # progress bar size (default is available terminal width)
barglyphs::BarGlyphs # the characters to be used in the bar
color::Symbol # default to green
Expand All @@ -82,6 +111,7 @@ mutable struct Progress <: AbstractProgress
function Progress(n::Integer;
dt::Real=0.1,
desc::AbstractString="Progress: ",
rate_format=PercentRateFormat(),
color::Symbol=:green,
output::IO=stderr,
barlen=nothing,
Expand All @@ -96,7 +126,7 @@ mutable struct Progress <: AbstractProgress
counter = start
tinit = tsecond = tlast = time()
printed = false
new(n, reentrantlocker, dt, counter, tinit, tsecond, tlast, printed, desc, barlen, barglyphs, color, output, offset, 0, start, enabled, showspeed, 1, 1, Int[])
new(n, reentrantlocker, dt, counter, tinit, tsecond, tlast, printed, desc, rate_format, barlen, barglyphs, color, output, offset, 0, start, enabled, showspeed, 1, 1, Int[])
end
end

Expand All @@ -105,8 +135,16 @@ Progress(n::Integer, dt::Real, desc::AbstractString="Progress: ",
offset::Integer=0) =
Progress(n, dt=dt, desc=desc, barlen=barlen, color=color, output=output, offset=offset)

Progress(n::Integer, dt::Real, desc::AbstractString="Progress: ",
rate_format::AbstractRateFormat=PercentRateFormat(),
barlen=nothing, color::Symbol=:green, output::IO=stderr;
offset::Integer=0) =
Progress(n, dt=dt, desc=desc, rate_format=rate_format, barlen=barlen, color=color, output=output, offset=offset)

Progress(n::Integer, desc::AbstractString, offset::Integer=0) = Progress(n, desc=desc, offset=offset)

Progress(n::Integer, desc::AbstractString, rate_format::AbstractRateFormat, offset::Integer=0) =
Progress(n, desc=desc, rate_format=rate_format, offset=offset)

"""
`prog = ProgressThresh(thresh; dt=0.1, desc="Progress: ",
Expand Down Expand Up @@ -273,7 +311,8 @@ function updateProgress!(p::Progress; showvalues = (), truncate_lines = false, v
bar = barstring(barlen, percentage_complete, barglyphs=p.barglyphs)
elapsed_time = t - p.tinit
dur = durationstring(elapsed_time)
msg = @sprintf "%s%3u%%%s Time: %s" p.desc round(Int, percentage_complete) bar dur
rate_str = rate_string(p.counter, p.n, p.rate_format)
msg = @sprintf "%s%s%s Time: %s" p.desc rate_str bar dur
if p.showspeed
sec_per_iter = elapsed_time / (p.counter - p.start)
msg = @sprintf "%s (%s)" msg speedstring(sec_per_iter)
Expand Down Expand Up @@ -308,7 +347,8 @@ function updateProgress!(p::Progress; showvalues = (), truncate_lines = false, v
else
eta = "N/A"
end
msg = @sprintf "%s%3u%%%s ETA: %s" p.desc round(Int, percentage_complete) bar eta
rate_str = rate_string(p.counter, p.n, p.rate_format)
msg = @sprintf "%s%s%s Time: %s" p.desc rate_str bar eta
if p.showspeed
sec_per_iter = elapsed_time / (p.counter - p.start)
msg = @sprintf "%s (%s)" msg speedstring(sec_per_iter)
Expand Down
14 changes: 14 additions & 0 deletions test/test.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,20 @@ end
println("\nTesting changing the description")
testfunc5B(107, 0.01, 0.05, "Step 1...", 50)

function testfunc5C(n, dt, tsleep, desc, rate_format)
p = ProgressMeter.Progress(n, dt, desc, rate_format)
for i = 1:n
sleep(tsleep)
ProgressMeter.next!(p)
end
end

println("\nTesting changing the progress rate format")
testfunc5C(107, 0.01, 0.05, "Computing...", EmptyRateFormat())
testfunc5C(107, 0.01, 0.05, "Computing...", PercentRateFormat())
testfunc5C(107, 0.01, 0.05, "Computing...", PercentRateFormat(2))
testfunc5C(107, 0.01, 0.05, "Computing...", FractionRateFormat())
testfunc5C(107, 0.01, 0.05, "Computing...", IntegerRateFormat())

function testfunc6(n, dt, tsleep)
ProgressMeter.@showprogress dt for i in 1:n
Expand Down