Commit graph

929 commits

Author SHA1 Message Date
Changming Sun
87b1fddd97
Add Linux/MacOS ARM64 support to nuget packaging pipeline (#9570) 2021-10-27 19:00:43 -07:00
Changming Sun
aa76520e60
Update macOS build agents to macOS 11 (#9562) 2021-10-27 10:00:04 -07:00
Scott McKay
b5a652c578
Add Xamarin support (#9436)
Add Xamarin support to the ORT nuget packages.
  - Update C# code to support Xamarin builds for iOS and Android
  - refactor some things to split out common code
  - include iOS and Android ORT native shared library in native nuget package
2021-10-27 20:07:07 +10:00
Yulong Wang
90555bf96d
[node.js binding] enable CI for macOS arm64 (#9532)
* nodejs aggr

* add dependency

* no unzip

* fix aggregation

* add arm64 for mac

* mac arm64 build

* fix commandline

* add check for multi-CMAKE_OSX_ARCHITECTURES

* fix
2021-10-26 16:42:19 -07:00
Yulong Wang
bf4c3fa3d6
[node.js binding] aggregate binaries for multiple platforms in single NPM package (#9501) 2021-10-25 20:16:10 -07:00
Changming Sun
406f1629c1
Remove Featurizers code (#9300) 2021-10-20 10:20:35 -07:00
Yulong Wang
5b65f1cb44
fixes SDL Native Rules warning in Node.js binding CI (#9402) 2021-10-18 13:05:46 -07:00
Changming Sun
6ecb990fae Update win-ci-pipeline.yml 2021-10-18 10:43:19 -07:00
Abhishek Jindal
87e726d1a0
Abjindal/merge eager with external custom ops (#8986)
* switching to pytorch nightly build

* adding eager mode

* enable pybind and remove install step

* removing auditwheel repair process

* installing package

* adding auditwheel back

* disabling auditwheel repair for eager mode

* typo correction
2021-10-14 13:19:45 -07:00
Jeff Daily
3e879aab6b
work around ucx in rocm ci Dockerfile (#9360) 2021-10-14 09:49:31 -07:00
Suffian Khan
70cf61fa84
disable bart-l for now (#9305) 2021-10-07 16:55:54 -07:00
Changming Sun
4f4875b0e8 Add "workspace: clean: all" to anybuild build yaml file 2021-10-06 22:49:37 -07:00
Suffian Khan
510b58c877
Increase AMD CI pipeline timeout to 120 min (#9280)
* increase timeout

* add timeout

* add timeout

* rename
2021-10-06 10:43:09 -07:00
Changming Sun
334980e016 Delete nocontribops pipelines 2021-10-06 10:30:32 -07:00
Sheil Kumar
8f6fd014e4
Force Windows AI NuGet pipeline to use Windows SDK 19041 (#9255)
* Force Windows AI Nuget pipeline to use 19041 Windows SDK as 22000 casues a downlevel regression by importing LoadLibraryW

* move into quotes

Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
2021-10-01 21:46:14 -07:00
Yulong Wang
448325b254
[js/web] name ort web for consistency (#9240) 2021-09-30 22:53:26 -07:00
Yulong Wang
8c57d51928
support WebAssembly SIMD for qgemm (#9191)
* support WebAssembly SIMD for qgemm

* remove '--experimental-wasm-bulk-memory' for test
2021-09-30 12:40:56 -07:00
Changming Sun
e1b84eefcc Revert "Revert "linux trt package pipeline (#7537)""
This reverts commit b606005858.
2021-09-30 11:39:23 -07:00
Edward Chen
5326397a6a
[iOS] Facilitate usage of pods with custom builds (#9216)
Refactor iOS framework build/pod package creation into a separate script that can be used with custom builds.
Add documentation.
2021-09-30 08:44:00 -07:00
Changming Sun
b606005858 Revert "linux trt package pipeline (#7537)"
This reverts commit faea7a222d.
2021-09-28 19:09:04 -07:00
RandySheriffH
058108bef9
Execution Provider Profiler (#8406)
* implement cuda provider

* define profiler common

* call start after register

* add memcpy event

* add cuda correlation

* format code

* add cupti to test path

* switch to CUpti_ActivityKernel3

* reset cupti path

* fix test case

* fix trt pipeline

* add namespace

* format code

* exclude training from testing

* remove mutex
2021-09-28 13:59:52 -07:00
Suffian Khan
6f580f07de
Switch AMD CI pipeline to use environment image from onnxruntimecibuildenvironment (#9206)
* shift docker image reference for amd ci pipeline

* fix service endpoint

* reduce perf tolerance
2021-09-28 13:06:16 -07:00
Changming Sun
1104e8d3e5
Linux Anybuild build pipeline (#9091) 2021-09-28 11:22:27 -07:00
Sheil Kumar
f16bb37fae
make uwp store apps link to statically-linked crt desktop builds (#9182)
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
2021-09-27 15:37:15 -07:00
baijumeswani
c30cc9190a
Change the agent pool for orttraining-distributed pipeline (#9179) 2021-09-26 21:26:44 -07:00
Suffian Khan
47888392ab
Fix nightly CI pipeline to generate ROCm 4.2 wheels and add ROCm 4.3.1 wheels (#9101)
* make work for both rocm 4.2 and rocm 4.3.1

* fix rocm 4.3.1 docker image reference

* fix CUDA_VERSION to ROCM_VERSION

* fix ReduceConsts conflict def

* add ifdef to miopen_common.h as well

* trailing ws
2021-09-19 23:36:03 -07:00
Yulong Wang
0183492931
simplify Web CI pipeline (#9094) 2021-09-17 13:00:07 -07:00
Guoyu Wang
438175cb34
Build shared host protoc in iOS CI pipelines (#9087)
* iOS build, share host protoc build

* Change android ci

* ios packaging ci

* checkout submodule

* revert

* update package pipeline

* minor update
2021-09-16 17:16:50 -07:00
Maajid khan
7fc28cd539
[OpenVINO-EP] UEP v3.1 Release with OpenVINO 2021.4.1 (#9081)
* 2021.4.1 Docker and ci changes

* OV version change

* Removing Imagescaler op from the op's list

Reverting this change which was added in last
PR. Imagescaler is now deprecated. so removing
it from the supported list. Also this
op is causing regression in the performance
of the FP16 models.

Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com>

* Re-writing the help message for num_of_threads

Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com>

Co-authored-by: Aravind Gunda <aravindx.gunda@intel.com>
2021-09-16 17:09:07 -07:00
Yulong Wang
498461485f
optimize WebAssembly CI pipeline (#9058) 2021-09-16 01:07:36 -07:00
Guoyu Wang
bee5c26580
Add CPU_ONLY runtime option to NNAPI EP (#9066)
* Add NNAPI cpu only option

* update java

* Update comments
2021-09-15 15:50:18 -07:00
Suffian Khan
e758870b18
Upgrade ROCm CI pipeline for ROCm 4.3.1 and permit run inside container (#9070)
* try to run inside 4.3.1 container

* no \ in container run command

* remove networking options

* try with adding video render groups

* add job to build docker image

* try without 1st stage

* change alpha, beta to float

* try adding service connection

* retain huggingface directory

* static video and render gid

* use runtime expression for variables

* install torch-ort

* pin sacrebleu==1.5.1

* update curves for rocm 4.3.1

* try again

* disable determinism and only check tail of loss curve and with a much larger threshold of 0.05

* disable RoBERTa due to high run variablity on ROCm 4.3.1

* put reduction unit tests back in
2021-09-15 12:32:02 -07:00
Changming Sun
4930320647
Delete linux-pytorch-custom-ops-ci-pipeline.yml (#9023) 2021-09-14 21:51:21 +00:00
Suffian Khan
4322f7e647
Fix ROCm wheels CI pipeline break by installing latest protobuf from source (#9047)
* install protobuf from source

* fix rm command in Dockerfile

* fix options on rm command

* fix cd into protobuf source directory

* try again

* remove strip step

* debug list the files

* ls on /usr

* more debug

* more debug

* adjust LD_LIBRARY_PATH

* try remove protobuf before ORT build
2021-09-14 12:07:00 -07:00
Guoyu Wang
cf70635d2a
Add Android executable drop in the Package pipeline (#9050)
* add copy executable for android job

* minor fix

* Variable fix

* Move to use tgz because zip is not part of the docker image

* update compression
2021-09-14 11:45:33 -07:00
Yulong Wang
be80698698
[js/web] a bugfix and add tests for wasm proxy worker (#9048)
* [js/web] add tests for wasm proxy worker

* fix script src override
2021-09-14 10:38:58 -07:00
Guoyu Wang
c709380c52
Add full iOS job in package pipeline (#9036)
* Add full ios xcframework job

* create zip file of the xcframework
2021-09-13 15:54:11 -07:00
baijumeswani
1422a9ba6b
Remove previous temporary fixes and address TODOs (#9020) 2021-09-13 10:10:07 -07:00
Edward Chen
29d6573f3d
Increase timeouts for Mac CI builds. (#9024)
Increase timeouts for "orttraining-mac-ci-pipeline" and "iOS CI Pipeline" CI builds.
2021-09-10 12:57:08 -07:00
Chi Lo
0367e1f1c2
Update Nuget Packge Pipline to CUDA11.4 and TensorRT8 on Windows (#9000)
* Update to CUDA11.4 and TensorRT-8.0.3.4

* update trt pool, remove cudnn from setup_env_gpu.bat

* revert pool

* test gpu package pipeline on t4

* back out changes

* back out changes

Co-authored-by: George Wu <jywu@microsoft.com>
2021-09-09 06:56:37 -07:00
Guoyu Wang
1533f574e4
Add full Android job in package pipeline (#9009)
* Add full Android job in package pipeline

* Address CR comments
2021-09-08 21:12:59 -07:00
Yulong Wang
2e8792ca42
[js/web] fix karma launch with chrome headless (#8998) 2021-09-08 11:52:41 -07:00
Ashwini Khade
ec63d10303
add model local function support (#8540)
* updates for picking pnnx commit

* add tests filter to c# tests

* plus test fixes

* fix versioning for contrib ops

* fix tests

* test filter for optional ops

* more versioning related updates

* fix test

* fix layernorm spec

* more updates

* update docs

* add more test filters

* more filters

* update binary size threshold

* update docs

* draft - enable model local function

* enable model local functions in ORT

* update to latest rel onnx commit

* plus tests

* plus more updates

* plus updates

* test updates

* Fix for nested functions + shape inference

* plus bug fix and updates per review

* plus fixes per review

* plus test updates

* plus updates per review

* plus fixes

* fix a test
2021-09-08 11:47:01 -07:00
George Wu
a30d9f5317
fix windows gpu pipelines that use cuda 10.2 (training, reduced_ops and 10.2 validation) (#8994)
* build for arch 52

* arch 52

* gpu arch 52
2021-09-07 22:01:06 -07:00
Changming Sun
0bb56a18cf
Add TRT header file to ORT GPU nuget package (#8962) 2021-09-07 09:50:09 -07:00
Scott McKay
eebcc20f10
Add netstandard2.0 framework to nuget managed package. (#8960)
* Add netstandard2.0 to nuget managed package.
Re-does PR that was backed out due to packaging pipeline changes.
Allows deprecation of netstandard1.1 in the following release as netstandard2 is the preferred lowest level framework.
2021-09-04 08:01:46 +10:00
Olivia Jain
a0c9408f0d
Make TRT Version Configurable (#8864)
* copy changes from trt_and_mem

* second edits

* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines

* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines

* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines

* change to cuda 11.4

* build with cuda 11.4

* Update Dockerfile.ubuntu_cuda11_1_tensorrt7_2

* add cmake extra defines

* cmake architectures

* fix cmake arch

* Delete ubuntu-18.04.Dockerfile

* Rename Dockerfile.ubuntu_cuda11_1_tensorrt7_2 to Dockerfile.ubuntu_cuda11_4_tensorrt7_2

* Update linux-gpu-tensorrt-ci-perf-pipeline.yml

* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines

* removing previous ort args

* rename to cuda 11.4

* remove cuda 10_2

* delete trt 7.1

* remove 7.1

* Passing in cuda architecture to reduce build time

* always add submodule sync due to recursive cloning

* fix run command

* add and

* take away unused arms and share python installation script

* Update linux-gpu-tensorrt-ci-perf-pipeline.yml

* Update Dockerfile.tensorrt

* cleanup file

* install python directly on dockerfile - move to scripts in future

* Update Dockerfile.custom-trt-perf

* adding cuda 11.1 for missing Libnvrtc.so.11.1

* Delete install_python.sh
2021-09-03 13:32:27 -07:00
Chi Lo
1f576e1766
Detect necessary files inside GPU packages (#8955)
* Rename files

* Update YAML files

* Update validation script and YAML
2021-09-03 13:28:28 -07:00
liqun Fu
a7f5bd226b
retarget torch181 to torch182 (#8947)
Co-authored-by: liqun <liqun@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
2021-09-03 09:44:42 -07:00
Changming Sun
1a34775fe9
Fix the benchmark code (#8926) 2021-09-02 10:36:24 -07:00