Skip to content

Benchmark 3.5.24 (pooled) vs 2023.03

Jump to bottom
Ihar Suvorau edited this page Jul 24, 2023 · 2 revisions

Benchmarking results for v3.5.24 (pooled resourced) against the version in Spring 2023

Metric names

Short name Full name
AED Absolute Event Distance
CAR Case Arrival Rate
CED Circadian Event Distribution
RED Relative Event Distribution
CTD Cycle Time Distance
NGD N-Gram Distance (size 2)
NGD(3) N-Gram Distance (size 3)

Delta values for each metric

Negative value means improvement:

comparison

Measurements

name metric distance_2023.03 distance_3.5.24 distance_diff
AC_CRE AED 117.32 228.88732023010553 111.56732023010554
AC_CRE CAR 110.38 286.9748743718593 176.5948743718593
AC_CRE CED 3.11 2.812594557045716 -0.29740544295428384
AC_CRE CTD 62.23 39.68341708542714 -22.54658291457286
AC_CRE NGD 0.24 0.1716584711667411 -0.06834152883325889
AC_CRE NGD 3) 0.2174573055028463
AC_CRE RED 48.19 28.852165228507506 -19.337834771492492
BPIC12 AED 313.3 212.43076429603056 -100.86923570396945
BPIC12 CAR 336.42 290.41834677419354 -46.00165322580648
BPIC12 CED 2.1 2.5602918496808296 0.4602918496808295
BPIC12 CTD 93.45 20.57627688172045 -72.87372311827956
BPIC12 NGD 0.56 0.6425556365886539 0.08255563658865384
BPIC12 NGD 3) 0.7269584578426203
BPIC12 RED 96.82 34.93385013032818 -61.88614986967181
BPIC17 AED 314.92 321.75398987004104 6.833989870041023
BPIC17 CAR 390.04 389.0881236442516 -0.9518763557484249
BPIC17 CED 1.65 2.145270131124884 0.49527013112488394
BPIC17 CTD 102.85 46.01098156182213 -56.83901843817787
BPIC17 NGD 0.37 0.3592826984897469 -0.010717301510253074
BPIC17 NGD 3) 0.5645599851539871
BPIC17 RED 132.31 31.089469529963246 -101.22053047003675
CALL AED 61.76 82.16359499788796 20.403594997887957
CALL CAR 61.68 84.65810111541262 22.97810111541262
CALL CED 4.72 18.65638395546567 13.936383955465672
CALL CTD 8.18 4.641477250948676 -3.5385227490513236
CALL NGD 0.08 0.1252496425237234 0.04524964252372339
CALL NGD 3) 0.1657212499120336
CALL RED 0.0 3.531925777675508 3.531925777675508

Raw CSV files with measurements

Configuration

version: 4
common:
  train_log_path: <train_log_path>
  test_log_path: <test_log_path>
  num_final_evaluations: 10
  evaluation_metrics:
    - 3_gram_distance
    - 2_gram_distance
    - absolute_event_distribution
    - relative_event_distribution
    - circadian_event_distribution
    - arrival_event_distribution
    - cycle_time_distribution
  log_ids:
    case: case_id
    activity: activity
    resource: resource
    start_time: start_time
    end_time: end_time
    enabled_time: enabled_time
preprocessing:
  multitasking: false
  enable_time_concurrency_threshold: 0.5
  concurrency_df: 0.75
  concurrency_l2l: 0.9
  concurrency_l1l: 0.9
control_flow:
  optimization_metric: n_gram_distance
  num_iterations: 30
  num_evaluations_per_iteration: 5
  gateway_probabilities: discovery
  discovery_algorithm: sm3
  epsilon:
    - 0.1
    - 1.0
  eta:
    - 0.2
    - 0.6
  replace_or_joins:
    - true
    - false
  prioritize_parallelism:
    - true
    - false
resource_model:
  optimization_metric: circadian_event_distribution
  num_iterations: 40
  num_evaluations_per_iteration: 5
  resource_profiles:
    discovery_type: pool
    granularity:
      - 15
      - 60
    confidence:
      - 0.1
      - 1.0
    support:
      - 0.1
      - 1.0
    participation: 0.4
extraneous_activity_delays:
  optimization_metric: relative_event_distribution
  num_iterations: 1
Clone this wiki locally