Question regarding the paper #6
Description
Hello. Thank you so much for the great research!
I had a question while reading the paper, so I'm leaving it here.
I read section 5 "Analysis of Generated Code Edits" with great interest and was wondering if I could find more detailed information in the subsection "Comparing CODEX and CODEGEN", since it only shows the number of code edits that each model generated (i.e. optimized successfully). For example, I would like to see a list of problems that CODEX successfully optimized and the detailed %OPT/SpeedUp/%RTR for each problem.
Again, thank you so much for this great work.