openwpm/OpenWPM#468 introduced job retries so that failed crawl jobs got retried n times. While this made running crawls much smoother, it also introduced data duplication, where the data for a job / crawl record / site visit could be (possibly partially) written but not recorded as successful, hence retried and resulting in a duplicated data for a site visit.
Feel free to close if you feel like it would be better to solve the root cause of the data duplication (related: openwpm/OpenWPM#476) or flag the final crawl_history entries as final instead of adding a helper.