no code implementations • 1 Nov 2023 • Atsushi Shirafuji, Taku Matsumoto, Md Faizul Ibne Amin, Yutaka Watanobe
We classify errors for 95, 631 code pairs and identify 3. 47 errors on average, which are submitted by various levels of programmers on an online judge system.
no code implementations • 26 Sep 2023 • Atsushi Shirafuji, Md. Mostafizer Rahman, Md Faizul Ibne Amin, Yutaka Watanobe
The experimental results show that the fine-tuned CodeT5 achieves a pass@100 of 91. 95% and an average edit distance of the most similar correct program of 6. 84, which indicates that at least one correct program can be suggested by generating 100 candidate programs.