mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-05-17 21:10:43 +00:00
d834ec895a
### Description `lintrunner` is a linter runner successfully used by pytorch, onnx and onnx-script. It provides a uniform experience running linters locally and in CI. It supports all major dev systems: Windows, Linux and MacOs. The checks are enforced by the `Python format` workflow. This PR adopts `lintrunner` to onnxruntime and fixed ~2000 flake8 errors in Python code. `lintrunner` now runs all required python lints including `ruff`(replacing `flake8`), `black` and `isort`. Future lints like `clang-format` can be added. Most errors are auto-fixed by `ruff` and the fixes should be considered robust. Lints that are more complicated to fix are applied `# noqa` for now and should be fixed in follow up PRs. ### Notable changes 1. This PR **removed some suboptimal patterns**: - `not xxx in` -> `xxx not in` membership checks - bare excepts (`except:` -> `except Exception`) - unused imports The follow up PR will remove: - `import *` - mutable values as default in function definitions (`def func(a=[])`) - more unused imports - unused local variables 2. Use `ruff` to replace `flake8`. `ruff` is much (40x) faster than flake8 and is more robust. We are using it successfully in onnx and onnx-script. It also supports auto-fixing many flake8 errors. 3. Removed the legacy flake8 ci flow and updated docs. 4. The added workflow supports SARIF code scanning reports on github, example snapshot:  5. Removed `onnxruntime-python-checks-ci-pipeline` as redundant ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Unified linting experience in CI and local. Replacing https://github.com/microsoft/onnxruntime/pull/14306 --------- Signed-off-by: Justin Chu <justinchu@microsoft.com>
74 lines
2.3 KiB
Python
74 lines
2.3 KiB
Python
#!/usr/bin/env python3
|
|
# Copyright (c) Microsoft Corporation. All rights reserved.
|
|
# Licensed under the MIT License.
|
|
|
|
# This file replaces https URLs in deps.txt to local file paths. It runs after we download the dependencies from Azure
|
|
# DevOps Artifacts
|
|
|
|
import argparse
|
|
import csv
|
|
import os
|
|
from dataclasses import dataclass
|
|
from pathlib import Path
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class Dep:
|
|
name: str
|
|
url: str
|
|
sha1_hash: str
|
|
|
|
|
|
def parse_arguments():
|
|
parser = argparse.ArgumentParser()
|
|
# The directory that contains downloaded zip files
|
|
parser.add_argument("--new_dir", required=False)
|
|
|
|
return parser.parse_args()
|
|
|
|
|
|
def main():
|
|
SCRIPT_DIR = os.path.dirname(os.path.realpath(__file__)) # noqa: N806
|
|
REPO_DIR = os.path.normpath(os.path.join(SCRIPT_DIR, "..", "..")) # noqa: N806
|
|
|
|
args = parse_arguments()
|
|
new_dir = None
|
|
if args.new_dir:
|
|
new_dir = Path(args.new_dir)
|
|
else:
|
|
BUILD_BINARIESDIRECTORY = os.environ.get("BUILD_BINARIESDIRECTORY") # noqa: N806
|
|
if BUILD_BINARIESDIRECTORY is None:
|
|
raise NameError("Please specify --new_dir or set the env var BUILD_BINARIESDIRECTORY")
|
|
new_dir = Path(BUILD_BINARIESDIRECTORY) / "deps"
|
|
|
|
# Here we intentionally do not check if new_dir exists, because it might be used in a docker container instead.
|
|
|
|
deps = []
|
|
|
|
csv_file_path = Path(REPO_DIR) / "cmake" / "deps.txt"
|
|
|
|
# Read the whole file into memory first
|
|
with csv_file_path.open("r", encoding="utf-8") as f:
|
|
depfile_reader = csv.reader(f, delimiter=";")
|
|
for row in depfile_reader:
|
|
if len(row) != 3:
|
|
continue
|
|
# Lines start with "#" are comments
|
|
if row[0].startswith("#"):
|
|
continue
|
|
deps.append(Dep(row[0], row[1], row[2]))
|
|
|
|
# Write updated content back
|
|
with csv_file_path.open("w", newline="", encoding="utf-8") as f:
|
|
depfile_writer = csv.writer(f, delimiter=";")
|
|
for dep in deps:
|
|
if dep.url.startswith("https://"):
|
|
new_url = new_dir / dep.url[8:]
|
|
depfile_writer.writerow([dep.name, new_url.as_posix(), dep.sha1_hash])
|
|
else:
|
|
# Write the original thing back
|
|
depfile_writer.writerow([dep.name, dep.url, dep.sha1_hash])
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|