Data processing and validation for the paper titled "Automated Grading and Feedback Tools for Programming Education: A Systematic Review" published in the ACM Transactions on Computing Education.
All the data is stored within the 'data' directory, including:
- raw data of all the decisions and annotations made by the two screeners
- further annotation of data specifically relating to the performance of automated assessment tools compared when compared to humans
- processed data ready for visualisation
- subsets of data utilised for specific visualisations or other work
The data extraction notebook processes the annotated data into a dataset that can be visualised for the results and discussion. The results and discussion notebooks are used to output the visualisations and the data used within the results and discussions sections.
To replicate the data processing and visualisation steps, run the notepads from start to finish in the following order:
- data extraction
- results
- discussion