onnxruntime/orttraining/tools/scripts/sqldb_to_tensors.py
Suffian Khan 9fa0d8392a
Extend node debugging utilities to push tensors and node placement to SQL database (#8672)
* adding support for tracing to sqldb instead of files

* use compiled statements

* script to pull tensors from db

* link sqlite3

* remove node info redundant with onnx graph

* addressing PR comments

* address PR comments and include program counter

* third party notice

* use find_pacakge

* add to cgmanifests.json

* address thread safety and add pid suffix

* build fi

* python script to select on devicetype

* remove unpopulated and redundant Shape and Type fields

* comment

* comment

* PR comments

* add graph execution counter to session state

* move increment to inference session

* std::endl to \n

* ifdef on graph execution counter

* add ifdef to inference session

* move DEBUG_NODE_INPUTS_OUTPUTS to CMakeLists.txt
2021-08-21 00:40:12 -07:00

19 lines
720 B
Python

# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
import sqlite3
import onnx
from onnx import numpy_helper
connection = sqlite3.connect('<path-to-sqldb-from-tracing>', detect_types=sqlite3.PARSE_DECLTYPES)
def convert_tensor_proto_to_numpy_array(blob):
tensor_proto = onnx.TensorProto()
tensor_proto.ParseFromString(blob)
return numpy_helper.to_array(tensor_proto)
sqlite3.register_converter("TensorProto", convert_tensor_proto_to_numpy_array)
for step, name, value, device, producer, consumers in connection.execute(
'Select Step, Name, Value, DeviceType, TracedProducer, TracedConsumers from Tensors'):
print(step, name, value.shape, consumers)