onnxruntime/tools/ci_build/github/linux
Justin Chu d834ec895a
Adopt linrtunner as the linting tool - take 2 (#15085)
### Description

`lintrunner` is a linter runner successfully used by pytorch, onnx and
onnx-script. It provides a uniform experience running linters locally
and in CI. It supports all major dev systems: Windows, Linux and MacOs.
The checks are enforced by the `Python format` workflow.

This PR adopts `lintrunner` to onnxruntime and fixed ~2000 flake8 errors
in Python code. `lintrunner` now runs all required python lints
including `ruff`(replacing `flake8`), `black` and `isort`. Future lints
like `clang-format` can be added.

Most errors are auto-fixed by `ruff` and the fixes should be considered
robust.

Lints that are more complicated to fix are applied `# noqa` for now and
should be fixed in follow up PRs.

### Notable changes

1. This PR **removed some suboptimal patterns**:

	- `not xxx in` -> `xxx not in` membership checks
	- bare excepts (`except:` -> `except Exception`)
	- unused imports
	
	The follow up PR will remove:
	
	- `import *`
	- mutable values as default in function definitions (`def func(a=[])`)
	- more unused imports
	- unused local variables

2. Use `ruff` to replace `flake8`. `ruff` is much (40x) faster than
flake8 and is more robust. We are using it successfully in onnx and
onnx-script. It also supports auto-fixing many flake8 errors.

3. Removed the legacy flake8 ci flow and updated docs.

4. The added workflow supports SARIF code scanning reports on github,
example snapshot:
	

![image](https://user-images.githubusercontent.com/11205048/212598953-d60ce8a9-f242-4fa8-8674-8696b704604a.png)

5. Removed `onnxruntime-python-checks-ci-pipeline` as redundant

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Unified linting experience in CI and local.

Replacing https://github.com/microsoft/onnxruntime/pull/14306

---------

Signed-off-by: Justin Chu <justinchu@microsoft.com>
2023-03-24 15:29:03 -07:00
..
docker Gradle clean up (#14973) 2023-03-10 10:50:32 -08:00
ort_minimal Adopt linrtunner as the linting tool - take 2 (#15085) 2023-03-24 15:29:03 -07:00
tvm upgrade protobuf to 3.20.2 and onnx to 1.13 (#14279) 2023-01-31 12:55:09 -08:00
build_cuda_c_api_package.sh Remove 37 and 50 from CUDA compute archs (#14874) 2023-03-03 12:24:21 -08:00
build_linux_arm64_python_package.sh Remove 37 and 50 from CUDA compute archs (#14874) 2023-03-03 12:24:21 -08:00
build_yocto.sh
copy_strip_binary.sh include coreml_provider_factory.h in macos build instead of coreml_ex… (#12138) 2022-07-11 18:27:01 -07:00
create_package.sh
extract_and_bundle_gpu_package.sh Remove git and python packages from the docker images used by Zip-Nuget-Java-Nodejs Packaging Pipeline (#11651) 2022-06-03 20:00:54 -07:00
java_copy_strip_binary.sh Change native folder name for java macos arm64 (#12335) 2022-07-27 15:13:07 -07:00
java_linux_final_test.sh Update protobuf-java to version 3.21.7 (#13630) 2022-11-17 15:04:42 -08:00
run_build.sh Gradle clean up (#14973) 2023-03-10 10:50:32 -08:00
run_dockerbuild.sh Update package pipelines to support TRT 8.5 (#13998) 2022-12-16 15:01:50 -08:00
run_python_dockerbuild.sh Remove root in run_python_dockerbuild.sh (#15169) 2023-03-23 06:32:36 -07:00
run_python_tests.sh Add license header to some files. (#13074) 2022-09-23 18:46:02 -07:00
test_custom_ops_pytorch_export.sh
upload_code_coverage_data.sh Pin version of post to dashboard scripts' dependencies and update them to work with recent version. (#10353) 2022-01-21 19:35:58 -08:00
upload_ortsrv_binaries.sh
yocto_build_toolchain.sh