Commit graph

316 commits

Author SHA1 Message Date
Jian Chen
43e1e89453
Update aarch64 building pool to aiinfra-linux-ARM64-CPU-2019 (#12243)
* Setting new pool for arm64

* Setting defualt pool name

* adding DockerInstaller stage

* try to install docker from apt-get

* change to specific

* adding chmod to docker.sock

* install dotnet sdk

* specic dotnet 3.1.x

* add manuall step to install dotnet

* typo bass

* remove inputs

* change dotnet installation dir

* skipComponentGovernanceDetection on arm64 linux

* variables typo

* variables:
    - name: skipComponentGovernanceDetection
      value: true

* update variables

* skipComponentGovernanceDetection set to true

* moving varliables

* moving the variables again

* setting condition on cgd

* indentation

* indentation again

* conditional variable

* if

* remove cgd

* conditionl on cgd

* condition

* parameters

* clean up
2022-07-20 12:08:02 -04:00
Edward Chen
6e051016c1
Add Python package to perf test pipeline. (#12135) 2022-07-12 10:50:24 -07:00
Scott McKay
bfe1eca10c
Add targets files for new .net6 frameworks (#12016)
* Add net6 targets.
Remove maccatalyst as we don't have a native build targetting that.

* Set platform in macos targets

* Add targetFramework entries

* Move NativeLib.DllName definition and set using preprocessor values for simplicity. Couldn't get it to build with the preprocessor based setup when it was in a separate file.

Update the nuspec generation to set platform version for .net6 targets. TODO: Validate versions. I copied them from the managed nuget package the packaging pipeline generated prior to adding targets. Possibly w could/should lower some of the versions.

Hopefully the need to specify a version goes away when the release version of VS2022 supports .net6.

* Try android 31.1 as https://github.com/actions/virtual-environments/blob/main/images/win/Windows2022-Readme.md suggests that should be available on the CI machines

* Fix patch version mismatch
Add some extra debug info in case it helps

* Debug nuget location in CI

* Add workspace entry back in

* Add steps

* One more attempt with hardcoded nuget.exe path and original android31.0 version

* Better fix - found explicit nuget download and updated version there.

* flake8 fixes

* Fix black complaints.

* Exit Microsoft_ML_OnnxRuntime_CheckPrerequisites for net6 iOS.

* Removed outdated comment
2022-07-01 09:13:55 -07:00
Yulong Wang
bd973bcf1e
[js/rn] upgrade dependencies for e2e test (#11863)
* [js/rn] upgrade dependencies for e2e test

* use JDK11 only for gradle

* expand variable
2022-06-27 14:56:49 -07:00
Scott McKay
f72288b453
Fix a couple of typos (#11943)
Fix couple of typos
2022-06-27 10:32:14 +10:00
Gary Miguel
4bf22e2a40
Update ONNX to 1.12 (#11924)
Follow-ups that need to happen after this and before the next ORT release:
* Support SequenceMap with https://github.com/microsoft/onnxruntime/pull/11731
* Support signal ops with https://github.com/microsoft/onnxruntime/pull/11778

Follow-ups that need to happen after this but don't necessarily need to happen before the release:
* Implement LayerNormalization kernel for opset version 17: https://github.com/microsoft/onnxruntime/issues/11916

Fixes #11640
2022-06-21 17:19:52 -07:00
Scott McKay
3b1224dc08
Add .net6 support to the C# nuget package. (#11908)
* Add .net6 support to the C# nuget package.

Currently requires jumping through a lot of hoops due to .net 6 only being supported in the preview release of VS 2022.

Build existing targets using msbuild.
Add .net6 targets and build using dotnet.
Create nuget package with combined targets.

A few misc automated changes from VS to spacing and adding a couple of properties.
2022-06-22 08:08:24 +10:00
Yi Zhang
8bb0062873
add manylinux_2_27 CPU wheel (#11886)
* add manylinux_2_27

* minor refactory

* change base image

* minor refactor

* add tests

* fix condition
2022-06-17 19:38:38 +08:00
Changming Sun
10478a09ca Revert "add manylinux_2_27 wheel (#11832)"
This reverts commit bbace23d0c.
2022-06-16 18:28:12 -07:00
George Wu
df5ee6aa4e
[TensorRT EP] support TensorRT 8.4 (#11866)
* update trt 8.4ga

* trt 8.4 linux ci pipeline

* fix cmake

* placeholder_builder

* trt 8.4 windows pipeline

* gpu package pipeline

* trt 8.4.1.5 , packaging pipeline updates

* python packaging

* ctest timeout

* python packaging test

* bump timeout

* python format

* format

* revert

* newline

* enable trt python tests

* typo

* python format

* disable on windows
2022-06-16 07:46:40 -07:00
Scott McKay
d64f23fec0
EP factory creation cleanup and enhancements. (#11798)
* Rework the EP factory creation setup so we're not cut-and-pasting function declarations in multiple places.
Convert append EP for SNPE to be generic, and also use for XNNPACK.
Add XNNPACK to C# API

* Don't need stub for MIGraphX as it's using provider bridge.

* Remove old 'create' functions that aren't applicable now that the EPs are built as separate libraries.

* Only use EPs that require the layout transform if the opset is supported by the layout transformer.

* Update wasm registration of xnnpack.
2022-06-16 07:01:41 +10:00
Yi Zhang
bbace23d0c
add manylinux_2_27 wheel (#11832)
* add manylinux_2_27
2022-06-15 10:26:51 +08:00
Yi Zhang
b4f1e769c0
Add Mac Silicon/M1 Wheel (#11591) 2022-06-07 08:58:20 -07:00
Changming Sun
d5e34acb82
Remove git and python packages from the docker images used by Zip-Nuget-Java-Nodejs Packaging Pipeline (#11651) 2022-06-03 20:00:54 -07:00
Changming Sun
ec05313cd9
Split the GPU pipeline to 3 different machine pools (#11724) 2022-06-03 10:57:32 -07:00
Changming Sun
48ae27d578
Update protobuf-java to 3.20.1 (#10420) 2022-05-11 07:52:12 -07:00
Yulong Wang
3437967e63
[js/rn] fix CI packaging for react native E2E test (#11463)
* [js/rn] fix ORTRN packaging in CI

* fix env var setting
2022-05-09 18:09:52 -07:00
Changming Sun
3b16fb2000
Delete java-test-final-jar-step.yml (#8894) 2022-05-09 11:25:03 -07:00
Yulong Wang
308b605047
[wasm] increase timeout for Web Assembly static lib CI (#11306)
* [wasm] increase timeout for Web Assembly static lib CI

* update config format
2022-05-03 11:29:40 -07:00
Scott McKay
af249943a1 Increase the timeout so the packaging pipeline stops failing.
TODO: Someone should investigate why the AARCH64 build takes 3+ hours and reduce it if possible. Assuming it's using an emulator given the x64 build with the same arguments takes 13 minutes.
2022-04-20 09:36:37 -07:00
Yi Zhang
94032357e2
use int storage (#11185) 2022-04-15 09:56:36 +08:00
Changming Sun
8237568b65
Fix the rocm packaging pipeline package upload problem (#11174)
In #11114 , I changed the script to use azcopy instead of azure blob storage's python APIs. However, it doesn't work for the AMD rocm pipeline, because:

1. The machines do not have azcopy installed
2. The machines are not in Azure, so they don't have Azure managed identity. So they still need to use SAS.

Therefore in this PR I get the old python file back, but only use it in the AMD pipeline.
2022-04-11 13:59:44 -07:00
ytaous
eec5187801
Remove Rocm 4.2 from CI Build (#11130)
* remove rocm42 CI

* update torch to v1.11.0

Co-authored-by: Ethan Tao <ettao@microsoft.com@orttrainingdev7.d32nl1ml4oruzj4qz3bqlggovf.px.internal.cloudapp.net>
2022-04-07 11:42:09 -07:00
Changming Sun
26fceca90f
Update tools/ci_build/upload_python_package_to_azure_storage.py to not use the azure blob storage python package (#11114) 2022-04-06 14:30:51 -07:00
Changming Sun
588a66e221
Add cleanup steps to the build jobs which run in Linux CPU machine pool (#11078) 2022-03-31 22:34:12 -07:00
Baiju Meswani
249c4dec7f
Update orttraining release pipelines to use torch 1.11.0 (#11018)
* Update orttraining release pipelines to use torch 1.11.0

* Change requirements_torch...txt to requirements.txt

* Update cuda cmake architectures and clean up old files
2022-03-31 21:51:06 -07:00
Changming Sun
8e6dbad287 FIX: Nuget pipeline doesn't report binary size for Linux ARM64
In #10652 #10637 #10624, we changed the RID. But I forgot to update this part.
2022-03-31 18:32:05 -07:00
Yulong Wang
179406bd25
[JS] upgrade package-lock.json from v1 to v2 (#11039)
* upgrade package-lock.json from v1 to v2

* upgrade requirement of nodejs version to 16.x
2022-03-30 13:30:28 -07:00
leqiao-1
8ddc45f52d
Add linux and macos arm64 java aritifacts (#10981) 2022-03-25 16:23:17 -07:00
raviskolli
480c793125
Update training packages to Pytorch 1.11.0 (#10851)
* Update ortmodule training packages to Pytorch 1.11.0

Co-authored-by: Harshitha Venkata <havenka@microsoft.com>
Co-authored-by: Baiju Meswani <bmeswani@microsoft.com>
2022-03-22 16:45:51 -07:00
Sunghoon
6d19c295d0
use lf as eol for node package (#10965) 2022-03-21 15:50:03 -07:00
Sunghoon
b34d9f6867
[js/wasm] Add WebAssembly static library build into web CI pipeline (#10959)
* add webassembly static library build into ci

* add webassembly static library build into ci

* skip publishing on static lib

* fix type
2022-03-21 15:49:49 -07:00
leqiao-1
a6ea278502
add python3.10 support (#10848)
* add python3.10 support

* upgrade numpy version in build pipeline

* add python 3.10 path

* upgrade torch version in build pipeline

* update docker run arguments

* change torch version

* fix typo

* fix permission issue

* change python version

* remove python3.10 for openvino build

* remove python 3.10 for openvino build
2022-03-21 09:46:02 +08:00
Sunghoon
740870a285
[js] Create npm packaging pipeline (#10886)
* create npm packaging pipeline

* fix indentations

* Update npm-packaging-pipeline.yml for Azure Pipelines

* Update npm-packaging-pipeline.yml for Azure Pipelines

* Update npm-packaging-pipeline.yml for Azure Pipelines

* react-native-ci as a template

* fix typos

* fix template paths

* add a depencendy

* change a stage name

* set different artifact name for each package

* fix typo

* Update npm-packaging-pipeline.yml for Azure Pipelines

Set a build Id for node npm package as a parameter

* Update npm-packaging-pipeline.yml for Azure Pipelines

Set a build Id for node npm package as a parameter

* Update npm-packaging-pipeline.yml for Azure Pipelines
2022-03-17 16:44:00 -07:00
Chi Lo
aebbb90b79
Integrate C-API tests into Pipelines for release packages (#10794)
* add c-api test for package

* fix bug for running c-api test for package

* refine run application script

* remove redundant code

* include CUDA test

* Remove testing CUDA EP temporarily

* fix bug

* Code refactor

* try to fix YAML bug

* try to fix YAML bug

* try to fix YAML bug

* fix bug for multiple directories in Pipelines

* fix bug

* add comments and fix bug

* Update c-api-noopenmp-packaging-pipelines.yml

* Remove failOnStandardError flag in Pipelines
2022-03-15 10:18:38 -07:00
Chun-Wei Chen
5202efd11e
remove unused six in code and CIs (#10832) 2022-03-10 15:38:44 -08:00
Changming Sun
cc3a3476ed
Uninstall onnxruntime-training before running local tests (#10827)
* Uninstall onnxruntime-training before running local tests
2022-03-09 18:45:04 -08:00
Sunghoon
68c8f5a1ef
Change a pipeline vmImage from windows-latest to windows-2019 (#10804) 2022-03-08 10:49:59 -08:00
liqun Fu
da885a72e8
update with onnx 1.11 release (#10441) 2022-03-07 21:10:55 -08:00
Changming Sun
d8bf9a479b
Remove python 3.6 from training pipelines (#10780)
Because the numpy we use doesn't support python 3.6. And inference pipelines already removed python 3.6.
2022-03-07 09:57:24 -08:00
Guoyu Wang
7aa706854f
Pipeline changes to build full ORT package for Android (#10654)
* Add android package build settings for full build
Co-authored-by: gwang0000 <62914304+gwang0000@users.noreply.github.com>
Co-authored-by: Scott McKay <skottmckay@gmail.com>
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
2022-03-04 15:35:54 +10:00
Baiju Meswani
f9b6eef05f
orttraining packaging pipeline for rocm 5.0.1 (#10725) 2022-03-02 12:32:14 -08:00
Sunghoon
a7f6442c45
[js] release pipeline for web and react native (#10656)
* skip browserstack test at release pipeline

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* pool name as a parameter to run at lotus

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* Update web-ci-pipeline.yml for Azure Pipelines

* create a packaging pipeline for web

* Update web-packaging-pipeline.yml for Azure Pipelines

* make web-ci-pipeline as a template

* make web-ci-pipeline as a template

* make web-ci-pipeline as a template

* make web-ci-pipeline as a template

* change a paramter name checking a pipeline

* make a pool name changable for react native pipeline

* disable code sign validation for react native

* fix react native package.json publish

* fix indentation

* remove unnecessary comment

* test onnxruntime-common package publish

* ts and js files use lf as eol for windows

* use Linux style of ending line break

* change newLine at only tsconfig.json

* restore a commented code

* fix git restore directory for npm packaging

* fix a typo

* force eol to lf on windows for js directory in CI
2022-03-01 21:38:33 -08:00
Yulong Wang
f4b2d3af2b
Upgrade emsdk to 3.1.3 (#10577) 2022-02-28 23:52:41 -08:00
Edward Chen
ffde44cd09
[iOS Packaging] Add full ORT build iOS package. (#10626)
Add C/C++ and Objective-C packages with full ORT builds.
2022-02-28 15:39:07 -08:00
Scott McKay
f2ca43fe0d
Enable CoreML in the macos package (#10675)
* packaging pipeline change

* Enable CoreML on macos

Co-authored-by: Guoyu Wang <wanggy@outlook.com>
2022-02-28 09:12:37 +10:00
Changming Sun
56be66a0ab Update c-api-cpu.yml: change nuget linux arm64 RID 2022-02-24 11:15:51 -08:00
Yulong Wang
893ee65e54
[js/web] fix lint error when run without ort-web TS types (#10429)
* [js/web] fix lint error when run without ort-web TS types

* update CI to run linter before 'npm ci' in /js/web
2022-02-17 22:34:38 -08:00
leqiao-1
8d06e5a9df
Add openvino base image option (#10581)
* add selectable python package build pipeline

* update tensorrt version

* update tensorrt version

* Update Dockerfile.ubuntu_openvino

* Update install_ubuntu.sh

* add parameters for openvino base image

* fix syntax error
2022-02-17 17:10:01 +08:00
leqiao-1
f22cd3af5d
Leqiao/add selectable pipeline (#10560)
* add selectable python package build pipeline

* update tensorrt version

* update tensorrt version
2022-02-16 09:07:29 +08:00