Skip to content

[Project idea] Find lack of diff coverage using fuzzing #31

@untitaker

Description

@untitaker

We can employ cargo-fuzz or cargo-afl to find two jsonschemas that are not compatible with each other, but produce no diff by the tool.

afl produces a random bytestring. we

  1. parse that bytestring into a lhs schema, a rhs schema, and a payload that we should validate against both schemas
  2. if that parsing already fails, just pass the test
  3. if the payload validates against both schemas, pass the test
  4. if the payload validates against lhs but not rhs, AND json-schema-diff produces no breaking changes, crash

on steps 2 and 3, we will discard a lot of payloads. afl will complain about this and say that coverage does not increase. but it will likely still find a lot of cases, if running long enough.

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions