Sound data and notebooks/code related to the offline experiments #4
Description
In the paper, we talk about offline experiments establishing our techniques; however, I can't find these offline experiments + sound data in our repo. Ideally, they would be here and fully replicatable. Even if we cannot release the sound recordings (e.g., due to privacy concerns), we could still release our experimental testbed code.
We then used these insights to design three experiments: two quantitative evaluations on real-world datasets and a field study. On sounds recorded by hearing people in multiple contexts, ProtoSound outperformed the best baseline model by a 9.7% accuracy margin (XX% vs. XX%). The average accuracy (88.9%) was close to the ground truth obtained by manual human labeling (91.3%). On sounds recorded by DHH people in and around their homes, ProtoSound’s average accuracy was 90.4%. In comparison, the dataset’s label accuracy rated by a hearing person was 94.5%.