Commit graph

44 commits

Author SHA1 Message Date
Jian Chen
43e1e89453
Update aarch64 building pool to aiinfra-linux-ARM64-CPU-2019 (#12243)
* Setting new pool for arm64

* Setting defualt pool name

* adding DockerInstaller stage

* try to install docker from apt-get

* change to specific

* adding chmod to docker.sock

* install dotnet sdk

* specic dotnet 3.1.x

* add manuall step to install dotnet

* typo bass

* remove inputs

* change dotnet installation dir

* skipComponentGovernanceDetection on arm64 linux

* variables typo

* variables:
    - name: skipComponentGovernanceDetection
      value: true

* update variables

* skipComponentGovernanceDetection set to true

* moving varliables

* moving the variables again

* setting condition on cgd

* indentation

* indentation again

* conditional variable

* if

* remove cgd

* conditionl on cgd

* condition

* parameters

* clean up
2022-07-20 12:08:02 -04:00
Scott McKay
bfe1eca10c
Add targets files for new .net6 frameworks (#12016)
* Add net6 targets.
Remove maccatalyst as we don't have a native build targetting that.

* Set platform in macos targets

* Add targetFramework entries

* Move NativeLib.DllName definition and set using preprocessor values for simplicity. Couldn't get it to build with the preprocessor based setup when it was in a separate file.

Update the nuspec generation to set platform version for .net6 targets. TODO: Validate versions. I copied them from the managed nuget package the packaging pipeline generated prior to adding targets. Possibly w could/should lower some of the versions.

Hopefully the need to specify a version goes away when the release version of VS2022 supports .net6.

* Try android 31.1 as https://github.com/actions/virtual-environments/blob/main/images/win/Windows2022-Readme.md suggests that should be available on the CI machines

* Fix patch version mismatch
Add some extra debug info in case it helps

* Debug nuget location in CI

* Add workspace entry back in

* Add steps

* One more attempt with hardcoded nuget.exe path and original android31.0 version

* Better fix - found explicit nuget download and updated version there.

* flake8 fixes

* Fix black complaints.

* Exit Microsoft_ML_OnnxRuntime_CheckPrerequisites for net6 iOS.

* Removed outdated comment
2022-07-01 09:13:55 -07:00
Scott McKay
f72288b453
Fix a couple of typos (#11943)
Fix couple of typos
2022-06-27 10:32:14 +10:00
Scott McKay
3b1224dc08
Add .net6 support to the C# nuget package. (#11908)
* Add .net6 support to the C# nuget package.

Currently requires jumping through a lot of hoops due to .net 6 only being supported in the preview release of VS 2022.

Build existing targets using msbuild.
Add .net6 targets and build using dotnet.
Create nuget package with combined targets.

A few misc automated changes from VS to spacing and adding a couple of properties.
2022-06-22 08:08:24 +10:00
Changming Sun
d5e34acb82
Remove git and python packages from the docker images used by Zip-Nuget-Java-Nodejs Packaging Pipeline (#11651) 2022-06-03 20:00:54 -07:00
Changming Sun
48ae27d578
Update protobuf-java to 3.20.1 (#10420) 2022-05-11 07:52:12 -07:00
Scott McKay
af249943a1 Increase the timeout so the packaging pipeline stops failing.
TODO: Someone should investigate why the AARCH64 build takes 3+ hours and reduce it if possible. Assuming it's using an emulator given the x64 build with the same arguments takes 13 minutes.
2022-04-20 09:36:37 -07:00
Changming Sun
8e6dbad287 FIX: Nuget pipeline doesn't report binary size for Linux ARM64
In #10652 #10637 #10624, we changed the RID. But I forgot to update this part.
2022-03-31 18:32:05 -07:00
leqiao-1
8ddc45f52d
Add linux and macos arm64 java aritifacts (#10981) 2022-03-25 16:23:17 -07:00
Sunghoon
6d19c295d0
use lf as eol for node package (#10965) 2022-03-21 15:50:03 -07:00
Guoyu Wang
7aa706854f
Pipeline changes to build full ORT package for Android (#10654)
* Add android package build settings for full build
Co-authored-by: gwang0000 <62914304+gwang0000@users.noreply.github.com>
Co-authored-by: Scott McKay <skottmckay@gmail.com>
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
2022-03-04 15:35:54 +10:00
Edward Chen
ffde44cd09
[iOS Packaging] Add full ORT build iOS package. (#10626)
Add C/C++ and Objective-C packages with full ORT builds.
2022-02-28 15:39:07 -08:00
Scott McKay
f2ca43fe0d
Enable CoreML in the macos package (#10675)
* packaging pipeline change

* Enable CoreML on macos

Co-authored-by: Guoyu Wang <wanggy@outlook.com>
2022-02-28 09:12:37 +10:00
Changming Sun
56be66a0ab Update c-api-cpu.yml: change nuget linux arm64 RID 2022-02-24 11:15:51 -08:00
Edward Chen
6876641c1e
Pin version of post to dashboard scripts' dependencies and update them to work with recent version. (#10353) 2022-01-21 19:35:58 -08:00
Changming Sun
20f8a06f1f
Remove OpenMP code (#10032) 2021-12-15 00:58:42 -08:00
Scott McKay
fb4a8e12fc
Limit inclusion of Xamarin mobile target frameworks. (#9834)
- Only set them as targets for the ORT nuget package
    - Use OrtPackageId as the condition for inclusion, if installed
      - need to do the nuget restore via msbuild so that this property is set correctly
  - Add desktop-only version of the C# sln as there is no way to exclude the mobile specific csproj's from an sln
    - use this when applicable if someone is running build.py with the `--build_nuget` flag

Other
  - remove attempt to include symbols in the nuget package as nuget doesn't support symbols in native packages
  - update build.py to use `nuget` and not a windows specific path and filename for a linux build with `--build_nuget`
2021-11-23 11:29:53 +10:00
Changming Sun
88d8d2d02a
Remove Copy NuGet Package to Azure Blob Store build step (#8991) 2021-11-22 08:03:02 -08:00
Yulong Wang
31dc768e07
update ONNX Runtime Web CI to use same script for package versioning (#9698)
* use ps script for package generating

* allow e2e test has no common pkg
2021-11-10 12:52:34 -08:00
Yulong Wang
c6fddb263f
Add Node.js binding support to packaging pipeline (#9577) 2021-11-05 15:29:40 -07:00
Changming Sun
cc73bcc243 Suppress component governance component warnings for ios 2021-10-28 14:25:29 -07:00
Changming Sun
87b1fddd97
Add Linux/MacOS ARM64 support to nuget packaging pipeline (#9570) 2021-10-27 19:00:43 -07:00
Changming Sun
aa76520e60
Update macOS build agents to macOS 11 (#9562) 2021-10-27 10:00:04 -07:00
Scott McKay
b5a652c578
Add Xamarin support (#9436)
Add Xamarin support to the ORT nuget packages.
  - Update C# code to support Xamarin builds for iOS and Android
  - refactor some things to split out common code
  - include iOS and Android ORT native shared library in native nuget package
2021-10-27 20:07:07 +10:00
Guoyu Wang
438175cb34
Build shared host protoc in iOS CI pipelines (#9087)
* iOS build, share host protoc build

* Change android ci

* ios packaging ci

* checkout submodule

* revert

* update package pipeline

* minor update
2021-09-16 17:16:50 -07:00
Guoyu Wang
cf70635d2a
Add Android executable drop in the Package pipeline (#9050)
* add copy executable for android job

* minor fix

* Variable fix

* Move to use tgz because zip is not part of the docker image

* update compression
2021-09-14 11:45:33 -07:00
Guoyu Wang
c709380c52
Add full iOS job in package pipeline (#9036)
* Add full ios xcframework job

* create zip file of the xcframework
2021-09-13 15:54:11 -07:00
Guoyu Wang
1533f574e4
Add full Android job in package pipeline (#9009)
* Add full Android job in package pipeline

* Address CR comments
2021-09-08 21:12:59 -07:00
Chi Lo
1f576e1766
Detect necessary files inside GPU packages (#8955)
* Rename files

* Update YAML files

* Update validation script and YAML
2021-09-03 13:28:28 -07:00
Changming Sun
fbb6f0f599 Fix an error in Nuget pipeline caused by merge conflict 2021-09-02 09:26:25 -07:00
Changming Sun
6299a60bf8
Nuget: splitting PDB files to a separated package (#8903) 2021-09-01 09:07:24 -07:00
Changming Sun
a9a0d3f6fa Update min supported macOS version to 10.14 2021-08-31 16:09:48 -07:00
Changming Sun
c6d9426ef2
Add binary size reporting back (#8883) 2021-08-30 19:48:38 -07:00
Guoyu Wang
6a1939252f
Fix Android java API failure (#8865)
* Fix Android Package break

* Without java fix -- pipeline should fail

* With java fix, should pass now

* address CR comments
2021-08-27 15:58:56 -07:00
Changming Sun
5f74f198c1
Merge CPU/GPU nuget pipeline (#8683)
Merge CPU/GPU nuget pipeline. The old GPU nuget pipeline will be only for DML.

TODO: the result GPU package contains PDB files for some of the DLLs, but not all. It is due to the refactoring of CUDA EP to pluggable DLLs. At that time we forgot to copy the PDB files. However, I can't add them in now. Because currently the package is already 220MB large. If the missed PDB files were added, then it will be oversize. nuget.org doesn't accept >250MB packages.
2021-08-12 13:21:29 -07:00
Changming Sun
0510688411
Update compliance tasks in python packaging pipeline and fix some compile warnings (#8471)
1. Update SDLNativeRules from v2 to v3. The new one allows us setting excluded paths.
2. Update TSAUpload from v1 to v2. And add a config file ".gdn/.gdntsa" for it.
3. Fix some parentheses warnings
4. Update cmake to the latest.
5. Remove "--x86" build option from pipeline yaml files. Now we can auto-detect cpu architecture from python. So we don't need to ask user to specify it.
2021-07-30 17:16:37 -07:00
Changming Sun
c716b56f26
Update C++ Standard from 14 to 17 (#8041)
Switched the code to C++17. To build ONNX Runtime on old distros like CentOS 7, you need to install a newer GCC from additionary repos. If you build onnxruntime with the newer GCC, typically the result binary can't be distributed to other places because it depends on the new GCC's runtime libraries, something that the stock OS doesn't have. But on RHEL/CentOS, it can be better. We use Red Hat devtoolset 8/9/10 with CentOS7 building our code. The new library features(like std::filesystem) that not exists in the old C++ runtime will be statically linked into the applications with some restrictions:

1. GCC has dual ABI, but we can only use the old one. It means std::string is still copy-on-write and std::list::size() is still O(n). Also, if you build onnxruntime on CentOS 7 and link it with some binaries that were built on CentOS 8 or Ubuntu with the new ABI and export C++ symbols directly(instead of using a C API), the it won't work.

2. We still can't use std::optional. It is a limitation coming from macOS. We will solve it when we got macOS 11 build machines. It won't be too long.

3. Please avoid to use C++17 in CUDA files(*.cu). Also, the *.h files that they include(like core/framework/float16.h). This is Because CUDA 10.2 doesn't support C++17. You are welcome to use the new features in any *.cc files.
2021-06-25 14:08:01 -07:00
Changming Sun
b313c4581c
Remove CC/CXX env settings from C API packaging pipeline (#8014) 2021-06-10 11:36:52 -07:00
Edward Chen
45a7352622
Update Mac CI builds to use macOS-10.15 image, Xcode 12.4. (#7437)
Update Mac CI builds to use macOS-10.15 image, Xcode 12.4.
2021-05-27 09:39:34 -07:00
Changming Sun
4161758058
Remove openmp related packaging pipeline (#6991)
1. Remove openmp related packaging pipelines and build jobs.
2. Set continueOnError to true for the TSAUpload tasks. Their service is unstable recently.
3. Update Ubuntu 16 docker images to Ubuntu 18, in prepare for getting C++17 support
4. Cherry-pick the changes in 1.7.1 to the master: updating CFLAGS/CXXFLAGS to strip out debug symbols
2021-03-12 10:02:59 -08:00
Changming Sun
0be5475de6
Update packaging pipelines(#6664) 2021-02-17 09:53:36 -08:00
RandySheriffH
9043df8b66
Deprecate OMP from nuget pipeline (release:1.7) (#6671)
* deprecate OMP from nuget

* remove omp build

* remove

* add openmp build

* add variants

* rename package

* move GPU to no omp pipeline

* reset path

* switch to abs path

* reset path

* add cpu package

* remove obsolete name

* set package name

Co-authored-by: Randy Shuai <rashuai@microsoft.com>
2021-02-17 00:03:44 -08:00
Changming Sun
8378a45ae7
Add python 3.8/3.9 support for Windows GPU and Linux ARM64 (#6615)
Add python 3.8/3.9 support for Windows GPU and Linux ARM64

Delete jemalloc from cgmanifest.json.

Add onnx node test to Nuphar pipeline.

Change $ANDROID_HOME/ndk-bundle to $ANDROID_NDK_HOME. The later one is more accurate.

Delete Java GPU packaging pipeline

Remove test data download step in Nuget Mac OS pipeline. Because these machines are out of control and out of our network, it's hard to make it reliable and the data secure.

Fix a doc problem in c-api-artifacts-package-and-publish-steps-windows.yml. It shouldn't copy C_API.md, because the file has been moved into a different branch.

Delete the CI build docker file for Ubuntu cuda 9.x and Ubuntu x86 32 bits

And, due to some internal restrictions, I need to rename some of the agent pools
2021-02-11 16:43:35 -08:00
Changming Sun
aa31ba5774
Merge CPU packaging pipelines (#6480)
1. Merge Nuget CPU pipeline, Java CPU pipeline, C-API pipeline into a single one.
2. Enable compile warnings for cuda files(*.cu) on Windows.
3. Enable static code analyze for the Windows builds in these jobs. For example, this is our first time scanning the JNI code.
4. Fix some warnings in the training code.
5. Enable code sign for Java. Previously we forgot it.
6. Update TPN.txt to remove Jemalloc.
2021-02-04 08:38:56 -08:00