mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-06-29 03:30:52 +00:00
d834ec895a
### Description `lintrunner` is a linter runner successfully used by pytorch, onnx and onnx-script. It provides a uniform experience running linters locally and in CI. It supports all major dev systems: Windows, Linux and MacOs. The checks are enforced by the `Python format` workflow. This PR adopts `lintrunner` to onnxruntime and fixed ~2000 flake8 errors in Python code. `lintrunner` now runs all required python lints including `ruff`(replacing `flake8`), `black` and `isort`. Future lints like `clang-format` can be added. Most errors are auto-fixed by `ruff` and the fixes should be considered robust. Lints that are more complicated to fix are applied `# noqa` for now and should be fixed in follow up PRs. ### Notable changes 1. This PR **removed some suboptimal patterns**: - `not xxx in` -> `xxx not in` membership checks - bare excepts (`except:` -> `except Exception`) - unused imports The follow up PR will remove: - `import *` - mutable values as default in function definitions (`def func(a=[])`) - more unused imports - unused local variables 2. Use `ruff` to replace `flake8`. `ruff` is much (40x) faster than flake8 and is more robust. We are using it successfully in onnx and onnx-script. It also supports auto-fixing many flake8 errors. 3. Removed the legacy flake8 ci flow and updated docs. 4. The added workflow supports SARIF code scanning reports on github, example snapshot:  5. Removed `onnxruntime-python-checks-ci-pipeline` as redundant ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Unified linting experience in CI and local. Replacing https://github.com/microsoft/onnxruntime/pull/14306 --------- Signed-off-by: Justin Chu <justinchu@microsoft.com>
105 lines
3.2 KiB
YAML
105 lines
3.2 KiB
YAML
name: Lint
|
|
|
|
on:
|
|
push:
|
|
branches:
|
|
- master
|
|
- main
|
|
pull_request:
|
|
|
|
jobs:
|
|
optional-lint:
|
|
name: Optional Lint
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v3
|
|
- name: misspell # Check spellings as well
|
|
uses: reviewdog/action-misspell@v1
|
|
with:
|
|
github_token: ${{ secrets.github_token }}
|
|
locale: "US"
|
|
reporter: github-pr-check
|
|
level: info
|
|
filter_mode: diff_context
|
|
- name: shellcheck # Static check shell scripts
|
|
uses: reviewdog/action-shellcheck@v1
|
|
with:
|
|
github_token: ${{ secrets.github_token }}
|
|
reporter: github-pr-check
|
|
level: info
|
|
filter_mode: file
|
|
|
|
lint-python-format:
|
|
# Required workflow
|
|
name: Python format
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v3
|
|
- name: Setup Python
|
|
uses: actions/setup-python@v4
|
|
with:
|
|
# Version range or exact version of Python to use, using SemVer's version range syntax. Reads from .python-version if unset.
|
|
python-version: "3.10"
|
|
- name: Install dependencies
|
|
run: |
|
|
python -m pip install -r requirements-dev.txt
|
|
python -m pip install lintrunner lintrunner-adapters
|
|
lintrunner init
|
|
- name: Run lintrunner on all files
|
|
run: |
|
|
set +e
|
|
if ! lintrunner --force-color --all-files --tee-json=lint.json -v; then
|
|
echo ""
|
|
echo -e "\e[1m\e[36mYou can reproduce these results locally by using \`lintrunner -m main\`.\e[0m"
|
|
exit 1
|
|
fi
|
|
- name: Produce SARIF
|
|
if: always()
|
|
run: |
|
|
python -m lintrunner_adapters to-sarif lint.json lintrunner.sarif
|
|
- name: Upload SARIF file
|
|
if: always()
|
|
continue-on-error: true
|
|
uses: github/codeql-action/upload-sarif@v2
|
|
with:
|
|
# Path to SARIF file relative to the root of the repository
|
|
sarif_file: lintrunner.sarif
|
|
category: lintrunner
|
|
checkout_path: ${{ github.workspace }}
|
|
|
|
lint-cpp:
|
|
name: Lint C++
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@master
|
|
- name: Install ninja
|
|
run: python -m pip install --upgrade ninja
|
|
- name: Generate compile_commands.json
|
|
run: |
|
|
python tools/ci_build/build.py \
|
|
--cmake_generator "Ninja" \
|
|
--build_dir build \
|
|
--update \
|
|
--cmake_extra_defines CMAKE_EXPORT_COMPILE_COMMANDS=ON
|
|
- name: Generate ONNX protobuf files
|
|
run: cmake --build build/Debug --config Debug --target onnx_proto
|
|
- uses: reviewdog/action-cpplint@master
|
|
with:
|
|
github_token: ${{ secrets.github_token }}
|
|
reporter: github-pr-check
|
|
level: warning
|
|
flags: --linelength=120 --exclude=java/src/main/native/*.c
|
|
filter: "-runtime/references"
|
|
|
|
lint-js:
|
|
name: Lint JavaScript
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v3
|
|
- uses: reviewdog/action-eslint@v1
|
|
with:
|
|
reporter: github-pr-check
|
|
level: error
|
|
filter_mode: file
|
|
eslint_flags: "--ext .ts --ext .tsx"
|
|
workdir: "js/"
|