Skip to content

Conversation

@andreasjansson
Copy link
Member

@andreasjansson andreasjansson commented Oct 8, 2025

Important

Improved boolean and JSON handling, enhanced input generation for model fuzzing, and added tests for image output matching.

  • Behavior:
    • Updated boolean() in ai.py to use a more specific system prompt and added thinking=True to call().
    • Enhanced json_object() in ai.py to handle JSON outputs more robustly, including stripping code block markers.
    • Modified call() in ai.py to support a thinking parameter and updated model version.
    • Improved output_matches_prompt() in match_outputs.py to better handle media description matching.
    • Added normalize_suffix() in match_outputs.py to standardize file extensions.
    • Updated make_fuzz_inputs() in predict.py to generate diverse inputs while respecting schema constraints.
  • Tests:
    • Added test_non_matching_images.py to test image similarity and matching.
    • Updated test_output_matches_prompt.py to include more comprehensive image prompt matching tests.
    • Enhanced test_predict.py to test various scenarios for make_fuzz_inputs() including handling of null values and fixed inputs.
  • Misc:
    • Renamed make_predict_inputs() to make_fuzz_inputs() across multiple files.
    • Updated predict.py to include a new function make_fuzz_system_prompt() for generating system prompts for fuzzing.

This description was created by Ellipsis for e8b7c0d. You can customize this summary. It will automatically update as commits are pushed.

@andreasjansson andreasjansson merged commit 6a384d8 into main Oct 13, 2025
4 checks passed
@andreasjansson andreasjansson deleted the better-boolean branch October 13, 2025 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants