@@ -233,3 +233,35 @@ Figure 7 corresponds to `hamming-1/c-pareto.pdf` (left)
233233 and ` mathematics-1/c-pareto.pdf ` (right).
234234Figure 8 corresponds to ` baseline-pareto2.pdf ` .
235235Figure 9 corresponds to ` cost-vs-time.pdf ` .
236+
237+ Since this evaluation measures real running time,
238+ you should expect to see a deviation in results
239+ from the figures in the paper.
240+ To demonstrate this variance,
241+ we ran our evaluation on other machines
242+ and plotted the results;
243+ the figures and specs for each machine are provided below.
244+ The first row of each table are
245+ the figures from the submitted version
246+ of the paper.
247+
248+ ### Figure 7
249+
250+ | OS | CPU | RAM (GB) | NMSE | Mathematics |
251+ | --| --| --| --| --|
252+ | Ubuntu 20.04.6 LTS | AMD EPYC 7702P | 512 | ![ Figure 7] ( ./infra/figures/config-1/clang-comparison-hamming.jpg ) | ![ Figure 7] ( ./infra/figures/config-1/clang-comparison-mathematics.jpg ) |
253+ | Ubuntu 22.04.2 LTS | Intel i5-8279U | 16 | ![ Figure 7] ( ./infra/figures/config-2/hamming-c-pareto.jpg ) | ![ Figure 7] ( ./infra/figures/config-2/mathematics-c-pareto.jpg ) |
254+
255+ ### Figure 8
256+
257+ | OS | CPU | RAM (GB) | Figure 8 |
258+ | --| --| --| --|
259+ | Ubuntu 20.04.6 LTS | AMD EPYC 7702P | 512 | ![ Figure 8] ( ./infra/figures/config-1/herbie-comparison.jpg ) |
260+ | Ubuntu 22.04.2 LTS | Intel i5-8279U | 16 | ![ Figure 8] ( ./infra/figures/config-2/baseline-pareto2.jpg ) |
261+
262+ ### Figure 9
263+
264+ | OS | CPU | RAM (GB) | Figure 9 |
265+ | --| --| --| --|
266+ | Ubuntu 20.04.6 LTS | AMD EPYC 7702P | 512 | ![ Figure 9] ( ./infra/figures/config-1/cost-vs-time.jpg ) |
267+ | Ubuntu 22.04.2 LTS | Intel i5-8279U | 16 | ![ Figure 9] ( ./infra/figures/config-2/cost-vs-time.jpg ) |
0 commit comments