Skip to content

Conversation

@qti-hungjuiw
Copy link
Contributor

Description

  • The change allows users to better debug unit tests by adding the following command flags:
    • --dump_onnx: Dump input onnx model
    • --dump_json: Dump json qnn graph with provider_option dump_json_qnn_graph
    • --dump_dlc: Dump dlc with provider_option qnn_ir_backend_path
    • --verbose: Use the log level ORT_LOGGING_LEVEL_VERBOSE
  • Developers can use the command flags above to save the artifacts of QNN-EP testcases to a directory named with <TestSuite>_<TestName>
        .
        ├── QnnCPUBackendTests_BatchNorm2D_fp32         # RunQnnModelTest
        │   ├── dumped_f32_model.onnx                   # float32 ONNX model
        │   ├── QNNExecutionProvider_QNN_XXXX_X_X.dlc
        │   └── QNNExecutionProvider_QNN_XXXX_X_X.json
        ├── QnnHTPBackendTests_BatchNorm_FP16           # TestFp16ModelAccuracy
        │   ├── dumped_f16_model.onnx                   # float16 ONNX model
        │   ├── dumped_f32_model.onnx                   # float32 ONNX model
        │   ├── QNNExecutionProvider_QNN_XXXX_X_X.dlc
        │   └── QNNExecutionProvider_QNN_XXXX_X_X.json
        └── QnnHTPBackendTests_BatchNorm2D_U8U8S32      # TestQDQModelAccuracy
            ├── dumped_f32_model.onnx                   # float32 ONNX model
            ├── dumped_qdq_model.onnx                   # QDQ ONNX model
            ├── QNNExecutionProvider_QNN_XXXX_X_X.dlc
            └── QNNExecutionProvider_QNN_XXXX_X_X.json
    
        # All artifact files are placed under the current working directory from which the test binary is invoked.
    

Motivation and Context

  • The Json qnn graph/dlc are helpful for backend to debug performance/accuracy issues
  • By comparing the onnx and Json qnn graph/dlc, we can locate the issue about graph manipulation.

…test.exe

- The change allows users to better debug unit tests by adding the
following command flags:
    - --dump_onnx: Dump input onnx model
    - --dump_json: Dump json qnn graph with provider_option
      dump_json_qnn_graph
    - --dump_dlc: Dump dlc with provider_option qnn_ir_backend_path
    - --verbose: Use the log level ORT_LOGGING_LEVEL_VERBOSE
- Save onnx, dlc, json to the directory named with
  <TestSuite>_<Testname>
- Skip datatypes check on QnnIr.dll to prevent generated dlc from
  partitioned
@qti-hungjuiw
Copy link
Contributor Author

Hi @edgchen1, could you take a look at this PR for QNN-EP's test cases and let me know if everything looks okay?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant