Commit graph

781 commits

Author SHA1 Message Date
Sunghoon
a7f6442c45
[js] release pipeline for web and react native (#10656)
* skip browserstack test at release pipeline

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* pool name as a parameter to run at lotus

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* create a packaging pipeline for web

* Update web-packaging-pipeline.yml for Azure Pipelines

* make web-ci-pipeline as a template

* make web-ci-pipeline as a template

* make web-ci-pipeline as a template

* make web-ci-pipeline as a template

* change a paramter name checking a pipeline

* make a pool name changable for react native pipeline

* disable code sign validation for react native

* fix react native package.json publish

* fix indentation

* remove unnecessary comment

* test onnxruntime-common package publish

* ts and js files use lf as eol for windows

* use Linux style of ending line break

* change newLine at only tsconfig.json

* restore a commented code

* fix git restore directory for npm packaging

* fix a typo

* force eol to lf on windows for js directory in CI
2022-03-01 21:38:33 -08:00
Yulong Wang
f4b2d3af2b
Upgrade emsdk to 3.1.3 (#10577) 2022-02-28 23:52:41 -08:00
Edward Chen
ffde44cd09
[iOS Packaging] Add full ORT build iOS package. (#10626)
Add C/C++ and Objective-C packages with full ORT builds.
2022-02-28 15:39:07 -08:00
Scott McKay
f2ca43fe0d
Enable CoreML in the macos package (#10675)
* packaging pipeline change

* Enable CoreML on macos

Co-authored-by: Guoyu Wang <wanggy@outlook.com>
2022-02-28 09:12:37 +10:00
Changming Sun
81831201a8
Change C# tests to use C# 5.0 (#10686)
.NET Core 2.1 has reached end of support on August 21, 2021. Use C# 5.0 instead. Our CI build machines do no have C# 6.0 yet. Later I will do it.
2022-02-26 00:28:30 -08:00
Changming Sun
56be66a0ab Update c-api-cpu.yml: change nuget linux arm64 RID 2022-02-24 11:15:51 -08:00
Alexey Gladyshev
7dc7529ec8
[TVM EP] Integrate tests for TVM EP into public onnxruntime CI (#10505)
* add support for bool type

* add TVM EP support for tests

* include TVM EP in python test pool

* fix pylint

* moved technical imports to a separate file

* clean up post build actions & move _ld_preload.py extension to CMake level

* add files for include TVM EP into CI

* implement custom logger for TVM

* replace TVM logging with ONNX RT logging

* update link for TVM EP tutorial

* clean up TVM EP cmake

* add pybind auto enabling for TVM EP

* fix blank spaces

* code review fixes

* replace print with comment

* add list of EP without TVM EP

* enable onnx tests

* disable contrib ops and ml ops

* reuse Dockerfile.ubuntu

* Move install_tvm_test_dependencies.sh out of Docker context dir, update build definition.

Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
2022-02-24 16:24:23 +01:00
Yulong Wang
893ee65e54
[js/web] fix lint error when run without ort-web TS types (#10429)
* [js/web] fix lint error when run without ort-web TS types

* update CI to run linter before 'npm ci' in /js/web
2022-02-17 22:34:38 -08:00
Sunghoon
1af4c170ef
[js/react_native] publish onnxruntime-common npm package as web and node do (#10566)
* apply the same policy for onnxruntime-common as web and node

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* remove old comment
2022-02-17 15:25:27 -08:00
leqiao-1
8d06e5a9df
Add openvino base image option (#10581)
* add selectable python package build pipeline

* update tensorrt version

* update tensorrt version

* Update Dockerfile.ubuntu_openvino

* Update install_ubuntu.sh

* add parameters for openvino base image

* fix syntax error
2022-02-17 17:10:01 +08:00
leqiao-1
f22cd3af5d
Leqiao/add selectable pipeline (#10560)
* add selectable python package build pipeline

* update tensorrt version

* update tensorrt version
2022-02-16 09:07:29 +08:00
Sunghoon
dd33ce0fdc
[js/react_native] Create ONNX Runtime React Native pipeline (#10474)
* Pipeline for ONNX Runtime react native

* Fix a test failure

* test with custom built binaries

* add onnxruntime-common package back

* don't bob build when bootstrap

* revise Android test

* rename example to e2e

* remove onnxruntime packages from package.json

* remove release-it package

* upgrade gradle version to the same as CI

* add a pipeline for react native

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* android and ios mobile build for react native e2e

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* use android aar package template

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* use android aar package template

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* publish ios test results

* add e2e tests and publish a npm package

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* remove aar from npm package

* wait for view displayed

* change a waiting logic

* increase wait time for app launching

* give more time to launch an app

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* disable metro server on testing

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* test ios simulator launching

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* fix iOS e2e test

* use a publishing version of npm packages

* make pretty

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* make only one onnxruntime-common package after packaging

* make a powershell script of packaging universal

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Add a warning for file changes during a test

* clean up

* fix lint errors

* fix js npm packaging

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* Update mac-react-native-ci-pipeline.yml for Azure Pipelines

* resolve comments

* fix a typo
2022-02-09 21:37:05 -08:00
Changming Sun
4f13c8ac39
Update orttraining-linux-ci-pipeline.yml (#10462) 2022-02-03 13:46:16 -08:00
Olivia Jain
a1d9a71b8b
Improve Perf System (#10404)
* move table names to one location

* remove session metadata

* reload trt inputs

* fix posting names

* Update linux-gpu-tensorrt-daily-perf-pipeline.yml for Azure Pipelines

* remove comments

* Split up anubis job and perf run

* add trt environ variables

* No embedded links
2022-02-01 16:01:34 -08:00
Edward Chen
c43c1691ad
Enable transpose optimizer in minimal extended build (#10349)
Enable transpose optimizer and infrastructure it depends on in a minimal extended build.
2022-01-31 09:41:04 -08:00
Changming Sun
ec4362f8f3
Enable more static analysis warnings and enable the analyzer for training cpu (#10176) 2022-01-27 11:17:20 -08:00
Changming Sun
5576e3553d
Remove python 3.6 from our python packaging pipeline (#10395) 2022-01-26 10:21:57 -08:00
Guoyu Wang
4af116649c
[QDQ] Hookup NNAPI GetCapability/Compile with shared QDQ selectors (#10347)
* add qdqgroup as input for NodeUnit

* minor update

* hookup nnapi_ep

* minor update

* update compiler setting

* Add a simple UT

* Pipeline change to add build minimal extended with NNAPI for Android

* move GetAllNodeUnits to node_unit.h, add UT for NodeUnits, minor updates

* minor updates

* address CR comments

Co-authored-by: gwang0000 <62914304+gwang0000@users.noreply.github.com>
2022-01-25 17:13:46 -08:00
Dmitri Smirnov
7e092a7e3f
Reduce number of memory allocations based on a customer profiling case (#10193)
Add abseil and inlined containers typedefs
Introduce TensorShapeVector for shape building.
Use gsl::span<const T> to make interfaces accept different types of vector like args.
Introduce InineShapeVectorT for shape capacity typed instantiations
Refactor cuda slice along with provider shared interfaces
Refactor Concat, Conv, Pad
Build with Conv Einsum and ConvTranspose refactored.
Remove TesnorShape::GetDimsAsVector()
Refactor SliceIterator and SliceIteratorBase
Refactor broadcast
Refactor Pads for twice as long
Remove memory planner intermediate shapes vector
Refactor orttraining
Fix passing TenshroShapeVector to tests
Remove abseil copy and submodule, use FetchContent_Declare/Fetch
Path with separate command
Make RocmAsyncBuffer accept anything convertible to span. Adjust Linux GPU pipeline.
2022-01-24 10:40:46 -08:00
PeixuanZuo
3dfadf9031
[FIX] Add condition in amd ci pipeline yaml to stop test in time when onnxruntime build failed (#10335)
* [FIX] Add condition in amd ci pipeline yaml to stop test in time when onnxruntime build failed.
2022-01-24 15:34:48 +08:00
Edward Chen
6876641c1e
Pin version of post to dashboard scripts' dependencies and update them to work with recent version. (#10353) 2022-01-21 19:35:58 -08:00
Edward Chen
bfabef081d
Remove unused pipeline orttraining-linux-gpu-perf-test-ci-pipeline.yml and unused send_perf_metrics tool. (#10326) 2022-01-21 14:31:34 -08:00
Olivia Jain
eee627fde9
Track Session Creation Time (#10281)
* add back previous changes lost in merge

* post session to dashboard

* post session creation time to dashboard

* fix trt 8 functionality:

* add component governance

* Remove hardcoded values

* Update linux-gpu-tensorrt-daily-perf-pipeline.yml for Azure Pipelines

* cleanup errors

* post results only once

* checkout 8.0 GA

* try build 8.0 without building shared lib

* add back build_shared_lib, not the problem

* add upload_time to table

* use identifier to post

* Shorten to TRT x.x

* shorten commit hash using rev_parse

* use shortened commit hash

* use nvidia's default TRT_VERSION
2022-01-21 13:20:53 -08:00
Baiju Meswani
2affd6e71e
orttraining packaging and ci pipelines to use cuda 11.3 (#10252) 2022-01-13 13:36:33 -08:00
Edward Chen
d43ef67d2b
Move binary size check to separate pipeline (#10254)
Move binary size check(s) to a separate pipeline. In the future, other binary size-related builds can go here.
Add publishing of build artifacts for easier analysis.
Add optional build with debug info.
2022-01-12 19:21:20 -08:00
Olivia Jain
4048ed326c
Update EP Perf Pipeline (#10149)
* migrate to 1ES Hosted Pool

* migrate to Kusto database

* refactor and organize ep names with ORT prefix

* standardize TRT benchmarking with save/load engine, input binding, and workspace

* Add TRT 8.2 to ep perf pipeline

* update model_list.json with full onnx zoo

* add anubis credentials

* add anubis credentials

* clarify trt variables

* get system info from docker image

* remove unwanted commenting
2022-01-11 16:12:32 -08:00
Edward Chen
792db33f01
Enable loading of ORT format model graph runtime optimizations (#9901)
Initial implementation of load/replay of runtime optimizations in an ORT format model.
2022-01-04 12:09:07 -08:00
Edward Chen
3bc91c2151
Move reduced ops files into build directory (#10030)
In a reduced ops build, some source files get updated. This change moves the updated files into the build directory. This way, it is easier to simultaneously manage different build directories (with possibly different reduced ops configurations) based on a single source directory.
2021-12-28 19:04:20 -08:00
Sheil Kumar
ce1a9ca618
Fix Microsoft.AI.MachineLearning NuGet App failure with multiple binaries copied to same destination (#10076)
* Include onnxruntime binary when not using pacakge referene or uap app.

* Remove the lib\uap10.0 build from the nuget package - causing conflicts

* Add UWP test

* remove build files

* remove local change

* reset mimalloc and onnx-tensorrt

* change username to Microsoft

Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
2021-12-21 12:34:03 -08:00
Changming Sun
4e9e01cb3c
Fix SDL warnings in CPU EP (#9975) 2021-12-19 20:54:29 -08:00
George Wu
16274beb6f
update TensorRT EP to use TensorRT 8.2 (#9981)
* update base image from 11.4.0 to 11.4.2

* update Linux TRT GPU pipeline to TRT 8.2

* update onnx-tensorrt to 8.2-GA

* disable failing TensorRT 8.2 tests.

* update pad test.

* fix

* update win trt ci pipeline to trt 8.2

* test run with cuda 11.4 and cudnn 8.2

* increase timeout

* revert

* revert

* update packaging pipelines to use trt 8.2

* fix typo

* update trt gpu perf pipeline to trt 8.2

* increase timeout

* delete deprecated ci-perf-pipeline.yml

* bump timeout

* adjust timeout packaging
2021-12-15 15:59:31 -08:00
Changming Sun
20f8a06f1f
Remove OpenMP code (#10032) 2021-12-15 00:58:42 -08:00
Xavier Dupré
42c176b60c
Update default opset to 14 in ORTModule (#9743)
* update to torch 1.10
* update torchvision version
* update torchtext version
* remove deprecated option enable_onnx_checker
* add unit test to test gradient of GatherElements
* add ORTMODULE_ONNX_OPSET_VERSION in a docker file
2021-12-09 12:45:35 +01:00
Tang, Cheng
8db49e3d0f
add ortmodule and eager mode test (#9888)
* add ortmodule and eager mode test

* add ortmodule dependency

* fix eager pipeline

* skip tthe ortmodule test for windows due to win ci issue

* remove useless win ci change

* add torch

Co-authored-by: Abhishek Jindal <abjindal@microsoft.com>
2021-12-02 19:49:18 -08:00
Abhishek Jindal
06e63218be
changing commit for windows build for eager mode (#9912) 2021-12-02 14:07:46 -08:00
Scott McKay
912e50f61c
Add CI minimal build with all options disabled. Fix python binding code if sparse tensors are disabled. (#9898)
* Add 2 builds to validate the cmake defines for excluding optional components work in both full and minimal builds.

* Create empty config for no-ops build

* Create empty config for no-ops build - attempt #2

* Create empty config for no-ops build - attempt #3

* Update python binding code to work when sparse tensors are disabled.
2021-12-03 06:56:51 +10:00
RandySheriffH
9345894c82
Add build option to enable cuda profiling (#9875) 2021-11-29 22:44:50 -08:00
Yulong Wang
6eb0c8d420
fix build break in release pipeline for Node.js binding test (#9850) 2021-11-24 13:54:43 -08:00
Scott McKay
fb4a8e12fc
Limit inclusion of Xamarin mobile target frameworks. (#9834)
- Only set them as targets for the ORT nuget package
    - Use OrtPackageId as the condition for inclusion, if installed
      - need to do the nuget restore via msbuild so that this property is set correctly
  - Add desktop-only version of the C# sln as there is no way to exclude the mobile specific csproj's from an sln
    - use this when applicable if someone is running build.py with the `--build_nuget` flag

Other
  - remove attempt to include symbols in the nuget package as nuget doesn't support symbols in native packages
  - update build.py to use `nuget` and not a windows specific path and filename for a linux build with `--build_nuget`
2021-11-23 11:29:53 +10:00
Changming Sun
88d8d2d02a
Remove Copy NuGet Package to Azure Blob Store build step (#8991) 2021-11-22 08:03:02 -08:00
Chi Lo
7242627fec
Integrate TensorRT into GPU Python package (#9785)
* add use_tensorrt build option

* Add use_tensorrt to running tests

* add use_tensorrt for Windows

* make trt ep to skip backend test

* make trt ep to skip backend test

* Fix bug

* Add/Modify description

* modify for debug

* swtich pool to test

* modify to debug

* modify to debug

* add vobersity

* refine the code

* refine the code

* refine the code

* fix flake8 warning

* refine the code

* add pre_load check for trt as well as add cupti lib to cuda depedencies

* modify script to make trt build path the same as cuda

* show error message when user wants to run TensorRT but TensorRT is not installed in the env

* fix bug

* fix bug

* add trt lib for manylinux

* include cuda_dependencies for trt

* rewrite the condition to throw exception

* make code more compact
2021-11-18 13:26:51 -08:00
Chun-Wei Chen
ac57afc3a6
Update ONNX to 1.10 globally in CIs (#9751)
* Bump ONNX 1.10.2 globally

* load ONNX_VERSION from VERSION_NUMBER

* /
2021-11-15 15:28:26 -08:00
raviskolli
9f4e8cf6a0
Update training pipelines to pytorch 1.10 (#9709)
* Update training pipelines to pytorch 1.10

* Fixed a typo in cuda version.

* Downgraded gcc to 8 for cuda 10.2
2021-11-15 11:21:55 -08:00
Edward Chen
9f69d8bbae
Disable partial runtime optimization implementation by default (#9748)
* Only serialize runtime optimization records container if non-empty.

* Remove runtime optimizations from onnxruntime/core/flatbuffers/schema/README.md as it's not completely implemented yet.

* Disable partial runtime optimization implementation by default.
2021-11-12 17:37:29 -08:00
Edward Chen
997266a620
Add build.py option to disable ORT format model runtime optimization (#9723)
ORT format model runtime optimization implementation is in progress.
This change adds a build.py option to disable the partial runtime optimization implementation, adds CI builds to test it, and disables runtime optimizations in mobile package builds.
2021-11-11 18:05:45 -08:00
Edward Chen
962feff7f1
Update linux-ci-pipeline.yml to run debug unit tests. (#9722) 2021-11-10 16:43:40 -08:00
Yulong Wang
31dc768e07
update ONNX Runtime Web CI to use same script for package versioning (#9698)
* use ps script for package generating

* allow e2e test has no common pkg
2021-11-10 12:52:34 -08:00
Changming Sun
53afaefe3b
Refactor Windows CI pipeline yaml files (#9672) 2021-11-08 11:11:49 -08:00
Yulong Wang
c6fddb263f
Add Node.js binding support to packaging pipeline (#9577) 2021-11-05 15:29:40 -07:00
Edward Chen
3e5dbfd94f
Increase binary size limit and publish binary size data even if limit is exceeded. (#9675) 2021-11-05 09:41:20 -07:00