mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-05-17 21:10:43 +00:00
d834ec895a
### Description `lintrunner` is a linter runner successfully used by pytorch, onnx and onnx-script. It provides a uniform experience running linters locally and in CI. It supports all major dev systems: Windows, Linux and MacOs. The checks are enforced by the `Python format` workflow. This PR adopts `lintrunner` to onnxruntime and fixed ~2000 flake8 errors in Python code. `lintrunner` now runs all required python lints including `ruff`(replacing `flake8`), `black` and `isort`. Future lints like `clang-format` can be added. Most errors are auto-fixed by `ruff` and the fixes should be considered robust. Lints that are more complicated to fix are applied `# noqa` for now and should be fixed in follow up PRs. ### Notable changes 1. This PR **removed some suboptimal patterns**: - `not xxx in` -> `xxx not in` membership checks - bare excepts (`except:` -> `except Exception`) - unused imports The follow up PR will remove: - `import *` - mutable values as default in function definitions (`def func(a=[])`) - more unused imports - unused local variables 2. Use `ruff` to replace `flake8`. `ruff` is much (40x) faster than flake8 and is more robust. We are using it successfully in onnx and onnx-script. It also supports auto-fixing many flake8 errors. 3. Removed the legacy flake8 ci flow and updated docs. 4. The added workflow supports SARIF code scanning reports on github, example snapshot:  5. Removed `onnxruntime-python-checks-ci-pipeline` as redundant ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Unified linting experience in CI and local. Replacing https://github.com/microsoft/onnxruntime/pull/14306 --------- Signed-off-by: Justin Chu <justinchu@microsoft.com>
52 lines
1.4 KiB
Python
Executable file
52 lines
1.4 KiB
Python
Executable file
import collections
|
|
import json
|
|
import sys
|
|
|
|
actual = sys.argv[1]
|
|
expect = sys.argv[2]
|
|
|
|
with open(actual) as file_actual:
|
|
json_actual = json.loads(file_actual.read())
|
|
|
|
with open(expect) as file_expect:
|
|
json_expect = json.loads(file_expect.read())
|
|
|
|
|
|
def almost_equal(x, y, threshold=0.05):
|
|
return abs(x - y) < threshold
|
|
|
|
|
|
# loss curve tail match
|
|
loss_tail_length = 4
|
|
loss_tail_matches = collections.deque(maxlen=loss_tail_length)
|
|
logged_steps = len(json_actual["steps"])
|
|
for i in range(logged_steps):
|
|
step_actual = json_actual["steps"][i]
|
|
step_expect = json_expect["steps"][i]
|
|
|
|
is_match = step_actual["step"] == step_expect["step"]
|
|
is_match = is_match if almost_equal(step_actual["loss"], step_expect["loss"]) else False
|
|
loss_tail_matches.append(is_match)
|
|
|
|
print(
|
|
"step {} loss actual {:.6f} expected {:.6f} match {}".format(
|
|
step_actual["step"],
|
|
step_actual["loss"],
|
|
step_expect["loss"],
|
|
is_match if logged_steps - i <= loss_tail_length else "n/a",
|
|
)
|
|
)
|
|
|
|
success = all(loss_tail_matches)
|
|
|
|
# performance match
|
|
threshold = 0.95
|
|
is_performant = json_actual["samples_per_second"] >= threshold * json_expect["samples_per_second"]
|
|
success = success if is_performant else False
|
|
print(
|
|
"samples_per_second actual {:.3f} expected {:.3f} in-range {}".format(
|
|
json_actual["samples_per_second"], json_expect["samples_per_second"], is_performant
|
|
)
|
|
)
|
|
|
|
assert success
|