Olivia Jain
4666a49106
Add Component Governance ( #8794 )
...
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
2021-08-20 17:41:18 -07:00
Olivia Jain
9cefd1303b
Integrate Anubis ( #8603 )
...
* copy over changes
* Update build_image.sh
* allow for configurable trt
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* reflect previous changes
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* model_list.json
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* checkout trt 7.1
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update post.py
* Update post.py
* Update post.py
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update model_list.json
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update post.py
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update post.py
* Update post.py
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update model_list.json
* Update post.py
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update post.py
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update post.py
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update start_job.ps1
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update run_mem_test_docker.sh
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Separate anubis files
* revert to old pipeline
* Update post.py
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* build off master Dockerfile
* Delete Dockerfile.custom-trt-perf
* Delete install_common_deps.sh
* uncomment
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml
* pass in trt container version
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update post.py
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update post.py
* remove sudo
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* add back build number
* allow python 3.8
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* python 3.8 fix trtexec
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* remove prev py38
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* add perf dependencies
* Update start_job.ps1
2021-08-16 13:20:28 -07:00
Chi Lo
31f291f0af
Add TRT EP memory leak test into trt perf script ( #8155 )
...
* Add memory check for TRT perf
* Revise test app
* Add memory check for TRT perf
* Revise test app
* add test cases
* Modify script and add pipeline YAML
* remove redundant code
* temporarily change
* Change YAML
* revise test app
* fix minor bug
* code refactor
* small fix
* temporarily change for test
* prepare result log
* rm container when it exits
* code refactor
2021-07-13 09:39:08 -07:00
Olivia Jain
b2247ece25
Make Perf Test Configurable ( #7836 )
...
- Allow anyone to kick off a perf test here. Customize: branch, eps, model selection, cuda version.
- Only run shape inference when required.
- Kill errored out memory processes.
- Remove warmup run.
- Clean up script.
- Standalone_TRT is it's own "EP" vs as an additional run with TRT EP
2021-06-18 11:11:19 -07:00
Olivia Jain
29172d8f54
Setup EP Dashboard ( #7321 )
...
* setting up dashboard
* posting to ort dashboard
* creating separate docker file
* including common deps
* tracking latency over time
2021-05-11 10:33:39 -07:00
Olivia Jain
fb40602ea2
Mem trt ( #6868 )
...
* adding trt comparison and memory consumption
* creating separate docker file
2021-04-05 22:16:12 -07:00
Olivia Jain
db05d53b94
Setup perf in docker and add features ( #6582 )
...
* setup scripts to run in docker
* percent threshold for accuracy
* branch testing
2021-02-25 09:31:03 -08:00
stevenlix
53eb948f4c
Upgrade TensorRT to v7.2.2 ( #6452 )
...
* upgrade to TensorRT 7.2.2
* extend GPU tensorrt CI timeout to 150 minutes
* update docker image name
* disable user interaction to avoid tensorrt container stuck when install tzdata
* upgrade to libssl1.1 for ubuntu20.04
* remove libicu60 from ubuntu20.04
* add libicu66 for ubuntu20.04
* debug
* llvm
* llvm
* disable ReverseSequenceTest.InvalidInput
* disable ReverseSequenceTest.InvalidInput
* fix issues
* fix issues
* Update linux-gpu-tensorrt-ci-pipeline.yml
* disable warning 4458 for TensorRT parser
* update onnx-tensorrt submodule
* disable warnings for TensorRT parser
* update onnx-tensorrt submodule to include latest bug fixes
* update setup_env_trt
* update pool for win trt ci pipeline'
Co-authored-by: George Wu <jywu@microsoft.com>
2021-02-18 04:30:47 -08:00
Chi Lo
92292de135
Tensorrt perf tool ( #5436 )
...
* Add YAML file for pipeline
* Modify typo
* Add working directory
* Modify and test
* Modfiy and test
* Modify and test
* Modify and test
* Modify
* Modify
* Modify
* Modify
* Make sure to copy all the result files
* Add clearn up
* Modify
* Modify agent pool name
* Upload only specific artifacts
* Modify
* Integrated CI Pipeline for running TRT perf as well as added the “large amount of models” into perf model target
* Fix bug
* Fix bug
* Add reading the information regarding previously known failing models
and then skip testing them during benchmark/validation
* Modify the script file for CI
* Replace print with logger.info
* Fix bug
* Fix bug
* Refine the code
* Modify the script so that it can capture script segmentation fault while
running ORT
* Fix bug
* fix bug
* fix bug
* Add debug info
* fix bug
* Refine perf code
* Refine the code
* fix bug
* Code refactoring
* change many-models path
* remove metadata after validation/benchmark are done
* Update README.md
* Fix bug so that metadata doesn't hold stale value
* Remove hardcode and update README
* Add arguments to the script to make it run correctly
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines
* Fix bug so that metadata doesn't hold stale value
* Fix small bug of finding test dataset directory for FP16 test data, as
well as modification of some output information
* use -i random for perf test of TRT changes
Co-authored-by: Olivia Jain <oljain@microsoft.com>
2020-11-06 12:27:42 -08:00
