Commit graph

809 commits

Author SHA1 Message Date
Baiju Meswani
249c4dec7f
Update orttraining release pipelines to use torch 1.11.0 (#11018)
* Update orttraining release pipelines to use torch 1.11.0

* Change requirements_torch...txt to requirements.txt

* Update cuda cmake architectures and clean up old files
2022-03-31 21:51:06 -07:00
Changming Sun
8e6dbad287 FIX: Nuget pipeline doesn't report binary size for Linux ARM64
In #10652 #10637 #10624, we changed the RID. But I forgot to update this part.
2022-03-31 18:32:05 -07:00
Yulong Wang
179406bd25
[JS] upgrade package-lock.json from v1 to v2 (#11039)
* upgrade package-lock.json from v1 to v2

* upgrade requirement of nodejs version to 16.x
2022-03-30 13:30:28 -07:00
leqiao-1
8ddc45f52d
Add linux and macos arm64 java aritifacts (#10981) 2022-03-25 16:23:17 -07:00
Olivia Jain
de384805cd
Custom parameters (#10964)
* get inputs independently for trtexec

* track one process only

* remove engine and profile files

* change time to commit time

* add runtime option for io binding

* move to commit date

* fixes

* add option for graph optimization

* cleanup docker script

* note second time creation

* allow for parameters to be configured from pipeline at runtime

* uncomment

* include optional arguments at runtime

* post second session creation

* update cmake version

* Revert "update cmake version"

This reverts commit 09a1364eae68610724c8e90eeea777b7ee03f74b.

* Move data format import
2022-03-23 09:47:24 -07:00
Yi Zhang
0efbe92296
fix coverage report error in master build (#10969)
* fix error in master

* check NNAPI_EP_MASTER

* Revert "check NNAPI_EP_MASTER"

This reverts commit 59c9043b7c9bbcb4b495d2dd121ef6d4271be408.

* rm coverage in PR build
2022-03-23 16:00:57 +08:00
raviskolli
480c793125
Update training packages to Pytorch 1.11.0 (#10851)
* Update ortmodule training packages to Pytorch 1.11.0

Co-authored-by: Harshitha Venkata <havenka@microsoft.com>
Co-authored-by: Baiju Meswani <bmeswani@microsoft.com>
2022-03-22 16:45:51 -07:00
Sunghoon
6d19c295d0
use lf as eol for node package (#10965) 2022-03-21 15:50:03 -07:00
Sunghoon
b34d9f6867
[js/wasm] Add WebAssembly static library build into web CI pipeline (#10959)
* add webassembly static library build into ci

* add webassembly static library build into ci

* skip publishing on static lib

* fix type
2022-03-21 15:49:49 -07:00
leqiao-1
a6ea278502
add python3.10 support (#10848)
* add python3.10 support

* upgrade numpy version in build pipeline

* add python 3.10 path

* upgrade torch version in build pipeline

* update docker run arguments

* change torch version

* fix typo

* fix permission issue

* change python version

* remove python3.10 for openvino build

* remove python 3.10 for openvino build
2022-03-21 09:46:02 +08:00
Yi Zhang
c1e37e4ebf
Android CI Pipeline: Fix post coverage bug (#10949) 2022-03-19 11:17:08 -07:00
Yi Zhang
3897b93606
optimize Android CI (#10938) 2022-03-19 11:00:21 +08:00
Changming Sun
a3eeb7b0f7 Update orttraining-linux-external-custom-ops.yml for Azure Pipelines 2022-03-17 23:36:53 -07:00
Sunghoon
740870a285
[js] Create npm packaging pipeline (#10886)
* create npm packaging pipeline

* fix indentations

* Update npm-packaging-pipeline.yml for Azure Pipelines

* Update npm-packaging-pipeline.yml for Azure Pipelines

* Update npm-packaging-pipeline.yml for Azure Pipelines

* react-native-ci as a template

* fix typos

* fix template paths

* add a depencendy

* change a stage name

* set different artifact name for each package

* fix typo

* Update npm-packaging-pipeline.yml for Azure Pipelines

Set a build Id for node npm package as a parameter

* Update npm-packaging-pipeline.yml for Azure Pipelines

Set a build Id for node npm package as a parameter

* Update npm-packaging-pipeline.yml for Azure Pipelines
2022-03-17 16:44:00 -07:00
Chi Lo
aebbb90b79
Integrate C-API tests into Pipelines for release packages (#10794)
* add c-api test for package

* fix bug for running c-api test for package

* refine run application script

* remove redundant code

* include CUDA test

* Remove testing CUDA EP temporarily

* fix bug

* Code refactor

* try to fix YAML bug

* try to fix YAML bug

* try to fix YAML bug

* fix bug for multiple directories in Pipelines

* fix bug

* add comments and fix bug

* Update c-api-noopenmp-packaging-pipelines.yml

* Remove failOnStandardError flag in Pipelines
2022-03-15 10:18:38 -07:00
Edward Chen
1a62306db7
Use separate build directories for full and mobile iOS packages. (#10835) 2022-03-10 19:33:06 -08:00
Chun-Wei Chen
5202efd11e
remove unused six in code and CIs (#10832) 2022-03-10 15:38:44 -08:00
Changming Sun
cc3a3476ed
Uninstall onnxruntime-training before running local tests (#10827)
* Uninstall onnxruntime-training before running local tests
2022-03-09 18:45:04 -08:00
Edward Chen
c147c9dda6
Remove ORT_ENABLE_RUNTIME_OPTIMIZATION_IN_MINIMAL_BUILD. (#10778)
Remove ORT_ENABLE_RUNTIME_OPTIMIZATION_IN_MINIMAL_BUILD as it is now implied by ORT_EXTENDED_MINIMAL_BUILD.
Remove related CMake option.
2022-03-08 16:18:49 -08:00
Sunghoon
68c8f5a1ef
Change a pipeline vmImage from windows-latest to windows-2019 (#10804) 2022-03-08 10:49:59 -08:00
liqun Fu
da885a72e8
update with onnx 1.11 release (#10441) 2022-03-07 21:10:55 -08:00
dependabot[bot]
4d943c9bd3
Bump numpy from 1.16.6 to 1.21.0 in /tools/ci_build/github/linux/docker/scripts/manylinux (#10387)
* Bump numpy in /tools/ci_build/github/linux/docker/scripts/manylinux
2022-03-07 20:39:49 -08:00
Changming Sun
d8bf9a479b
Remove python 3.6 from training pipelines (#10780)
Because the numpy we use doesn't support python 3.6. And inference pipelines already removed python 3.6.
2022-03-07 09:57:24 -08:00
PeixuanZuo
55af7a96a7
update the amd ci pipeline (#10723)
* [TEST] test to get amd pipeline information

* [FIX] lower the threshold

* [UPDATE] add retry task

* [UPDATE] add retry task

* [ERROR] error to occur retry

* [FIX] error

* [UPDATE] update retryCountOnTaskFailure to 1 time

* [UPDATE] add showmeminfo
2022-03-07 18:39:42 +08:00
Olivia Jain
12eb660415
Compare TRT vs ORT-TRT Accurately (#10565)
* get inputs independently for trtexec

* track one process only

* remove engine and profile files

* change time to commit time

* add runtime option for io binding

* move to commit date

* fixes

* add option for graph optimization

* cleanup docker script

* include remaining changes

* choose graph optimization option

* add space in option
2022-03-04 10:14:18 -08:00
Edward Chen
395a7242d6
[iOS packaging] Minor updates. (#10755)
* Change storage container, simplify build definition parameters.
* Remove explicit version from Objective-C docs.
* Increase timeout.
* Use real storage account.
* Get static website URL with az cli.
2022-03-04 16:02:53 +10:00
Guoyu Wang
7aa706854f
Pipeline changes to build full ORT package for Android (#10654)
* Add android package build settings for full build
Co-authored-by: gwang0000 <62914304+gwang0000@users.noreply.github.com>
Co-authored-by: Scott McKay <skottmckay@gmail.com>
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
2022-03-04 15:35:54 +10:00
Baiju Meswani
f9b6eef05f
orttraining packaging pipeline for rocm 5.0.1 (#10725) 2022-03-02 12:32:14 -08:00
Sunghoon
a7f6442c45
[js] release pipeline for web and react native (#10656)
* skip browserstack test at release pipeline

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* pool name as a parameter to run at lotus

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* create a packaging pipeline for web

* Update web-packaging-pipeline.yml for Azure Pipelines

* make web-ci-pipeline as a template

* make web-ci-pipeline as a template

* make web-ci-pipeline as a template

* make web-ci-pipeline as a template

* change a paramter name checking a pipeline

* make a pool name changable for react native pipeline

* disable code sign validation for react native

* fix react native package.json publish

* fix indentation

* remove unnecessary comment

* test onnxruntime-common package publish

* ts and js files use lf as eol for windows

* use Linux style of ending line break

* change newLine at only tsconfig.json

* restore a commented code

* fix git restore directory for npm packaging

* fix a typo

* force eol to lf on windows for js directory in CI
2022-03-01 21:38:33 -08:00
Yulong Wang
f4b2d3af2b
Upgrade emsdk to 3.1.3 (#10577) 2022-02-28 23:52:41 -08:00
Edward Chen
ffde44cd09
[iOS Packaging] Add full ORT build iOS package. (#10626)
Add C/C++ and Objective-C packages with full ORT builds.
2022-02-28 15:39:07 -08:00
Scott McKay
f2ca43fe0d
Enable CoreML in the macos package (#10675)
* packaging pipeline change

* Enable CoreML on macos

Co-authored-by: Guoyu Wang <wanggy@outlook.com>
2022-02-28 09:12:37 +10:00
Changming Sun
81831201a8
Change C# tests to use C# 5.0 (#10686)
.NET Core 2.1 has reached end of support on August 21, 2021. Use C# 5.0 instead. Our CI build machines do no have C# 6.0 yet. Later I will do it.
2022-02-26 00:28:30 -08:00
Changming Sun
56be66a0ab Update c-api-cpu.yml: change nuget linux arm64 RID 2022-02-24 11:15:51 -08:00
Alexey Gladyshev
7dc7529ec8
[TVM EP] Integrate tests for TVM EP into public onnxruntime CI (#10505)
* add support for bool type

* add TVM EP support for tests

* include TVM EP in python test pool

* fix pylint

* moved technical imports to a separate file

* clean up post build actions & move _ld_preload.py extension to CMake level

* add files for include TVM EP into CI

* implement custom logger for TVM

* replace TVM logging with ONNX RT logging

* update link for TVM EP tutorial

* clean up TVM EP cmake

* add pybind auto enabling for TVM EP

* fix blank spaces

* code review fixes

* replace print with comment

* add list of EP without TVM EP

* enable onnx tests

* disable contrib ops and ml ops

* reuse Dockerfile.ubuntu

* Move install_tvm_test_dependencies.sh out of Docker context dir, update build definition.

Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
2022-02-24 16:24:23 +01:00
Yulong Wang
893ee65e54
[js/web] fix lint error when run without ort-web TS types (#10429)
* [js/web] fix lint error when run without ort-web TS types

* update CI to run linter before 'npm ci' in /js/web
2022-02-17 22:34:38 -08:00
Sunghoon
1af4c170ef
[js/react_native] publish onnxruntime-common npm package as web and node do (#10566)
* apply the same policy for onnxruntime-common as web and node

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* remove old comment
2022-02-17 15:25:27 -08:00
leqiao-1
8d06e5a9df
Add openvino base image option (#10581)
* add selectable python package build pipeline

* update tensorrt version

* update tensorrt version

* Update Dockerfile.ubuntu_openvino

* Update install_ubuntu.sh

* add parameters for openvino base image

* fix syntax error
2022-02-17 17:10:01 +08:00
leqiao-1
f22cd3af5d
Leqiao/add selectable pipeline (#10560)
* add selectable python package build pipeline

* update tensorrt version

* update tensorrt version
2022-02-16 09:07:29 +08:00
Sunghoon
dd33ce0fdc
[js/react_native] Create ONNX Runtime React Native pipeline (#10474)
* Pipeline for ONNX Runtime react native

* Fix a test failure

* test with custom built binaries

* add onnxruntime-common package back

* don't bob build when bootstrap

* revise Android test

* rename example to e2e

* remove onnxruntime packages from package.json

* remove release-it package

* upgrade gradle version to the same as CI

* add a pipeline for react native

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* android and ios mobile build for react native e2e

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* use android aar package template

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* use android aar package template

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* publish ios test results

* add e2e tests and publish a npm package

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* remove aar from npm package

* wait for view displayed

* change a waiting logic

* increase wait time for app launching

* give more time to launch an app

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* disable metro server on testing

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* test ios simulator launching

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* fix iOS e2e test

* use a publishing version of npm packages

* make pretty

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* make only one onnxruntime-common package after packaging

* make a powershell script of packaging universal

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Add a warning for file changes during a test

* clean up

* fix lint errors

* fix js npm packaging

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* resolve comments

* fix a typo
2022-02-09 21:37:05 -08:00
Changming Sun
4f13c8ac39
Update orttraining-linux-ci-pipeline.yml (#10462) 2022-02-03 13:46:16 -08:00
Olivia Jain
a1d9a71b8b
Improve Perf System (#10404)
* move table names to one location

* remove session metadata

* reload trt inputs

* fix posting names

* Update linux-gpu-tensorrt-daily-perf-pipeline.yml for Azure Pipelines

* remove comments

* Split up anubis job and perf run

* add trt environ variables

* No embedded links
2022-02-01 16:01:34 -08:00
Edward Chen
c43c1691ad
Enable transpose optimizer in minimal extended build (#10349)
Enable transpose optimizer and infrastructure it depends on in a minimal extended build.
2022-01-31 09:41:04 -08:00
Changming Sun
ec4362f8f3
Enable more static analysis warnings and enable the analyzer for training cpu (#10176) 2022-01-27 11:17:20 -08:00
Changming Sun
5576e3553d
Remove python 3.6 from our python packaging pipeline (#10395) 2022-01-26 10:21:57 -08:00
Guoyu Wang
4af116649c
[QDQ] Hookup NNAPI GetCapability/Compile with shared QDQ selectors (#10347)
* add qdqgroup as input for NodeUnit

* minor update

* hookup nnapi_ep

* minor update

* update compiler setting

* Add a simple UT

* Pipeline change to add build minimal extended with NNAPI for Android

* move GetAllNodeUnits to node_unit.h, add UT for NodeUnits, minor updates

* minor updates

* address CR comments

Co-authored-by: gwang0000 <62914304+gwang0000@users.noreply.github.com>
2022-01-25 17:13:46 -08:00
Dmitri Smirnov
7e092a7e3f
Reduce number of memory allocations based on a customer profiling case (#10193)
Add abseil and inlined containers typedefs
Introduce TensorShapeVector for shape building.
Use gsl::span<const T> to make interfaces accept different types of vector like args.
Introduce InineShapeVectorT for shape capacity typed instantiations
Refactor cuda slice along with provider shared interfaces
Refactor Concat, Conv, Pad
Build with Conv Einsum and ConvTranspose refactored.
Remove TesnorShape::GetDimsAsVector()
Refactor SliceIterator and SliceIteratorBase
Refactor broadcast
Refactor Pads for twice as long
Remove memory planner intermediate shapes vector
Refactor orttraining
Fix passing TenshroShapeVector to tests
Remove abseil copy and submodule, use FetchContent_Declare/Fetch
Path with separate command
Make RocmAsyncBuffer accept anything convertible to span. Adjust Linux GPU pipeline.
2022-01-24 10:40:46 -08:00
PeixuanZuo
3dfadf9031
[FIX] Add condition in amd ci pipeline yaml to stop test in time when onnxruntime build failed (#10335)
* [FIX] Add condition in amd ci pipeline yaml to stop test in time when onnxruntime build failed.
2022-01-24 15:34:48 +08:00
Edward Chen
6876641c1e
Pin version of post to dashboard scripts' dependencies and update them to work with recent version. (#10353) 2022-01-21 19:35:58 -08:00
Edward Chen
bfabef081d
Remove unused pipeline orttraining-linux-gpu-perf-test-ci-pipeline.yml and unused send_perf_metrics tool. (#10326) 2022-01-21 14:31:34 -08:00