模型可用性检查器

模型可用性检查器会分析 ONNX 模型，评估其与 ORT Mobile、NNAPI 和 CoreML 一起使用的适用性。

目录

• 用法
• 与 NNAPI 和 CoreML 一起使用
• 建议

用法

python -m onnxruntime.tools.check_onnx_model_mobile_usability --help
usage: check_onnx_model_mobile_usability.py [-h] [--log_level {debug,info}] model_path

Analyze an ONNX model to determine how well it will work in mobile scenarios.

positional arguments:
  model_path            Path to ONNX model to check

optional arguments:
  -h, --help            show this help message and exit
  --log_level {debug,info}
                        Logging level (default: info)

与 NNAPI 和 CoreML 一起使用

该脚本将检查模型中的操作符是否受 ORT 的 NNAPI 执行提供程序 (EP) 和 CoreML EP 支持。根据支持的操作符数量及其在模型中的位置，它将评估使用 NNAPI 或 CoreML 是否可能带来益处。始终建议进行性能测试以进行验证。

此检查的示例输出如下：

INFO:  Checking resnet50-v1-7.onnx for usability with ORT Mobile.
INFO:  Checking NNAPI
INFO:  1 partitions with a total of 121/122 nodes can be handled by the NNAPI EP.
INFO:   Partition sizes: [121]
INFO:  Unsupported nodes due to operator=0
INFO:   Caveats that have not been checked and may result in a node not actually being supported:
     ai.onnx:Conv:Only 2D Conv is supported. Weights and bias should be constant.
     ai.onnx:Gemm:If input B is not constant, transB should be 1.
     ai.onnx:GlobalAveragePool:Only 2D Pool is supported.
     ai.onnx:MaxPool:Only 2D Pool is supported.
INFO:  Unsupported nodes due to input having a dynamic shape=1
INFO:  NNAPI should work well for this model as there is one partition covering 99.2% of the nodes in the model.
INFO:  Model should perform well with NNAPI as is: YES

如果模型具有动态输入形状，则会进行额外检查，以评估将形状设置为固定大小是否有助于性能。有关更多信息，请参阅 onnxruntime.tools.make_dynamic_shape_fixed。

此检查的示例输出：

INFO:  Checking resnet50-v1-7.onnx for usability with ORT Mobile.
...
INFO:  Checking CoreML MLProgram
INFO:  2 partitions with a total of 120/122 nodes can be handled by the CoreML MLProgram EP.
INFO:   Partition sizes: [119, 1]
INFO:  Unsupported nodes due to operator=1
INFO:   Unsupported ops: ai.onnx:Flatten
INFO:   Caveats that have not been checked and may result in a node not actually being supported:
     ai.onnx:Conv:Only 1D/2D Conv is supported. Bias if provided must be constant.
     ai.onnx:Gemm:Input B must be constant.
     ai.onnx:GlobalAveragePool:Only 2D Pool is supported currently. 3D and 5D support can be added if needed.
     ai.onnx:MaxPool:Only 2D Pool is supported currently. 3D and 5D support can be added if needed.
INFO:  Unsupported nodes due to input having a dynamic shape=1
INFO:  CoreML MLProgram can be considered for this model as there are two partitions covering 98.4% of the nodes. Performance testing is required to validate.
INFO:  Model should perform well with CoreML MLProgram as is: MAYBE
INFO:  --------
INFO:  Checking if model will perform better if the dynamic shapes are fixed...
INFO:  Partition information if the model was updated to make the shapes fixed:
INFO:  2 partitions with a total of 121/122 nodes can be handled by the CoreML MLProgram EP.
INFO:   Partition sizes: [120, 1]
INFO:  Unsupported nodes due to operator=1
INFO:   Unsupported ops: ai.onnx:Flatten
INFO:   Caveats that have not been checked and may result in a node not actually being supported:
     ai.onnx:Conv:Only 1D/2D Conv is supported. Bias if provided must be constant.
     ai.onnx:Gemm:Input B must be constant.
     ai.onnx:GlobalAveragePool:Only 2D Pool is supported currently. 3D and 5D support can be added if needed.
     ai.onnx:MaxPool:Only 2D Pool is supported currently. 3D and 5D support can be added if needed.
INFO:  CoreML MLProgram can be considered for this model as there are two partitions covering 99.2% of the nodes. Performance testing is required to validate.
INFO:  Model should perform well with CoreML MLProgram if modified to have fixed input shapes: MAYBE
INFO:  Shapes can be altered using python -m onnxruntime.tools.make_dynamic_shape_fixed

诊断输出提供了有关提出这些建议的深入信息。

这包括：

• 有关 NNAPI 和 CoreML EP 支持或不支持的单个操作符的信息
• 有关支持的操作符被分成多少组（也称为分区）的信息
  • 组越多，性能越差，因为每次在支持和不支持的节点组之间切换时，我们都必须在 NPU（神经网络处理单元）和 CPU 之间切换。

建议

最后，脚本将提供使用哪个 EP 的建议。

INFO:  As NNAPI or CoreML may provide benefits with this model it is recommended to compare the performance of the model using the NNAPI EP on Android, and the CoreML EP on iOS, against the performance using the CPU EP.