Commit graph

69 commits

Author SHA1 Message Date
M. Zeeshan Siddiqui
5d99f179b9
Merge pull request #3486 from microsoft/sedymche/merge_master_ort_training
Merge from master into ort_training
2020-04-13 10:55:36 -07:00
Sergii Dymchenko
bf3df41424 Put back SubmoduleCheckoutMode parameter into mac-ci.yml. 2020-04-12 21:49:38 -07:00
edgchen1
cffdff6702
Publish unit test results from Linux and Mac builds (#3480)
* Added publish test results step to Linux and Mac builds.

* Fix test result file pattern.
2020-04-10 14:51:56 -07:00
Sergii Dymchenko
6ba7c99e50 Merge branch 'master' into ort_training 2020-04-09 12:42:04 -07:00
Changming Sun
a5fea26cb4 Disable model tests for Mac OS X builds 2020-04-02 15:14:32 -07:00
Xueyun Zhu
ccc3535e72 resolve conflict 2020-03-20 20:20:35 +00:00
Tiago Koji Castro Shibata
3bdb0b620a
Fix WCOS/Win32 linking bugs (#3126)
* Fix WCOS/Win32 linking bugs

* Remove unused NODEFAULTLIB flags

* Avoid plain target_link_libraries signature

* Avoid plain target_link_libraries signature

* Fix library list escaping

* Use library list instead of string

* Remove duplicate link to windowsapp.lib

* Remove Win32 build workarounds

* Specify CMake policies before initializing language

* Expose Win32 header definitions during build

* Force set API family

* Enable Win32 APIs in featurizer

* Use MT dynamic CRT

* Expose Win32 specific functions

* Disable app container globally

* Disable default wide functions in featurizers

* Add featurizers to test include path

* Workaround https://gitlab.kitware.com/cmake/cmake/issues/19428

* Revert pipeline debugging hacks

* Skip /FI in CUDA sources

* Default to Win32 builds

* Enable WCOS when using WinML

* Use generator expression to apply CMAKE_MSVC_RUNTIME_LIBRARY to C++ only
2020-03-19 08:52:40 -07:00
Edward Chen
80dd62a240 Enable CI for training. 2020-03-11 14:41:32 -07:00
Edward Chen
e542cfd0e0 Introduce training changes. 2020-03-11 14:39:03 -07:00
Tiago Koji Castro Shibata
a59243090a
Publish release symbols (#3152)
* Publish release symbols

* Publish symbols if IsReleaseBuild
2020-03-05 22:32:18 -08:00
smk2007
6cdd2b4934
Enable DML Nuget Package for x64 or x86 architectures (#3120)
* add dml gpu pipelines

* add x86 to the gpu dml dev build pipeline

* Enable DML x86 builds

* Fix uint64_t -> size_t warning

* fix warnings

* enable dml on x86 ci builds

* operatorHelper 773 error uint32_t vs uint64_t

* operatorHelper 773 error uint32_t vs uint64_t

* make x86 pipeline use the gpu pool

* more warnings

* fix x86 directml path

* make dml nuget package

* disable tf_pnasnet_large

* disable zfnet512

* make validation use wildcards

* disable x86 dml gpu tests

* add args.

* update gpu.yml

* change nupkg wildcard

* add debug statements

* package x86 dml nupkg

* dont drop managed nuget again from dml pipeline build

* Add DML EULA

* directml license should be renamed to not clobber the existing license

* casing on dml package....

* {} to ()

* fix license name

* disable dml from x86 ci

* typo and cr feedback

* remove featurizers

* ship the dml pdb as well
2020-03-02 20:18:46 -08:00
Hariharan Seshadri
d7f2cdcc7e
Fix target platform of managed OnnxRuntime dll and enable x86 .NET testing (#3056)
* WIP: Re-enable x86 .NET testing in Release pipelines

Enabling x86 testing will make sure that ORT packages doesn’t break x86 projects of customers

* Remove setting some env variables

* Comment out a test failing on x86 builds

* More changes

* Minor fix

* More changes

* More changes

* s

* s

* s

* Revert minor change

* More changes

* More changes

* More changes 2

* explicitly set platform target

* Delete bin and obj folders

* Clean output dirs

* Add back TargetFramwork

* Disable x86 .net framework tests

* Skip x86 tests in MKLML pipeline
2020-02-24 23:02:59 -08:00
Changming Sun
fb871978b5
Adjust build flags for the release pipelines (#3066)
1. Add LTCG back. It was set to default OFF in my previous PR to speed up Windows build. It is only needed in release pipelines.
2. Remove --use_featurizers from all the packaging pipelines
3. Make sure all the packages have openmp
2020-02-21 16:45:42 -08:00
Changming Sun
179603775f
Use CUDA 10.1 for Linux build (#3057)
Use CUDA 10.1 for Linux build
(Windows change is already in)

Please note, cublas 10.2.1.243 is for CUDA SDK 10.1.243, not CUDA 10.2.x. CUDA 10.2.89 need cublas 10.2.2.89. They match on the last part of the digits.

libcublas10-10.1.0.105 won't work!!!

The cuda docker image by viswamy is already using 10.1, no need to change.
2020-02-21 11:55:32 -08:00
Tiago Koji Castro Shibata
fb2182f3fc
Release ARM/ARM64 Nuget packages (#2987)
* Enable ARM64 release builds

* Add ARM release

* Skip C# dll signing in ARM

* Copy ARM binaries to Nuget

* Restore nuget packages before ARM packaging

* wip

* Use host protoc at C# build

* Set ProtocDirectory on cross-compiled builds

* wip

* Fix typo
2020-02-10 16:29:27 -08:00
jignparm
645d8fb213
Jignparm/upgrade macos vm image (#2928)
* update MacOS image to 10.14

* Update to macos 10.14
2020-01-28 20:24:45 -08:00
Changming Sun
e0c9cdaa73
Fix the nuget pipelines (#2901) 2020-01-23 20:02:18 -08:00
KeDengMS
f9f25ec047
Fix spurious component detection warning (#2857)
Fix spurious component detection warning
Use component detection template for all pipelines
2020-01-18 20:10:35 -08:00
Changming Sun
c4e4abce73
Run static code analyzer on most of our code (#2817) 2020-01-10 22:17:17 -08:00
Changming Sun
48e042868f
Update test data (#2356) 2020-01-10 10:52:23 -08:00
Hariharan Seshadri
ebfcad1c90
Add script for release Nuget validation (#2719)
* Initial commit

* Nits

* Disable a test temporarily

* Change working directory

* Test

* Add download python step

* Test update

* More changes

* Fix space issue

* Fix

* Verify nuget signing

* Fix

* Spaces

* PR feedback

* Nit

* Fix

* Fix

* Remove temporary changes
2020-01-08 18:42:22 +05:30
Changming Sun
b42cb61904
Packaging pipeline changes for VS 2019 (#2711) 2019-12-20 19:53:51 -08:00
Changming Sun
a46a28b7d8
Windows CI changes(#2650) 2019-12-13 12:23:49 -08:00
shahasad
eeb28a80c0
setup java ci mac (#2570) 2019-12-06 11:43:40 -08:00
Ryan Hill
854362cf05
Update win-x86-ci.yml (#2557)
Fix build pipeline break
2019-12-05 18:44:12 -08:00
Xiang Zhang
3e7aaf8fa1 User/xianz/telemetry (#2458)
* enabme telemetry

* enable telemetry

* set enable telemetry as default

* for debugging

* remove log and set disable telemetry as default back

* delete private file while testing

* resolve comment: mainly add license header, rename macro and update docs

* rewording in privacy.md
2019-12-03 23:34:53 -08:00
Ashwini Khade
4caf5c9c13
add additional test data set for nuget pipeline (#2448)
* add SAS token to download internal test data for nuget pipeline

* update azure endpoint

* fix keyvault download step

* fix variable declaration for secret group

* fix indentation

* fix yaml syntax for variables

* fix setting secrets for script

* fix env synctax

* Fix macos pipeline

* attempt to add secrets to windows download data

* fix mac and win data download

* fix windows data download

* update test data set url and location
2019-11-25 13:08:03 -08:00
Changming Sun
080a0a3186
Nuget pipeline changes (#2305)
1. refactor the pipeline, remove some duplicated code
2. Move Windows_py_GPU_Wheels job to Win-GPU-CUDA10. We'll deprecated the "Win-GPU" pool
3. Delete cpu-nocontribops-esrp-pipeline.yml and cpu-nocontribops-pipeline.yml
4. In Linux nuget jobs, run "make install" before creating the package. So that extra RPAH info will be removed
2019-11-08 09:45:52 -08:00
Changming Sun
138a7f194e Add cleanup step 2019-10-30 08:13:09 -07:00
Pranav Sharma
69970d1f2a
Include the new Privacy.md file in all release packages. (#2200) 2019-10-20 07:58:36 -07:00
Changming Sun
021073b5e5
Update python packaging pipelines (#2167) 2019-10-19 07:42:54 -07:00
Adrian Tsai
4090d0d0de
Add DirectML Execution Provider (#2057)
This change adds a new execution provider powered by [DirectML](https://aka.ms/DirectML).

DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning on Windows. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported hardware and drivers.

The DirectML execution provider is capable of greatly improving evaluation time of models using commodity GPU hardware, without sacrificing broad hardware support or requiring vendor-specific extensions to be installed.

**Note** that the DML EP code was moved verbatim from the existing WindowsAI project, which is why it doesn't yet conform to the onnxruntime coding style. This is something that can be fixed later; we would like to keep formatting/whitespace changes to a minimum for the time being to make it easier to port fixes from WindowsAI to ORT during this transition.

Summary of changes:
* Initial commit of DML EP files under onnxruntime/core/providers/dml
* Add cmake entries for building the DML EP and for pulling down the DirectML redist using nuget
* Add a submodule dependency on the Windows Implementation Library (WIL)
* Add docs under docs/execution_providers/DirectML-ExecutionProvider.md
* Add support for DML EP to provider tests and perf tests
* Add support for DML EP to fns_candy_style_transfer sample
* Add entries to the C ABI for instantiating the DML EP
2019-10-15 06:13:07 -07:00
shahasad
c3ffd1f47d
added continue on error for the linux cleanup step, to mitigate the build failure. root cause unknown (#1936) 2019-09-26 15:57:24 -07:00
Changming Sun
09cdbe9d76
Update test data (#1932) 2019-09-26 10:13:53 -07:00
Changming Sun
d46e023ee4
Remove toolset=14.11 for CUDA build (#1921) 2019-09-25 15:19:37 -07:00
Hector Li
582a27f546
remove sudo from the cleanup step for Linux so that we don't need the sudo access for vstsagent build user
1. remove sudo from the cleanup step for Linux so that we don't need the sudo access for vstsagent build user
2. a minor fix in the install_ubuntu.sh to make the image smaller for openvino
2019-09-18 11:22:37 -07:00
Hector Li
2b8677b210
Enable Openvino nightly build on edge device (#1684)
1. Add openvino GPU nightly build pipeline, this test is running on Intel Up square Edge device. The device are host locally not from Azure VM. We persist a smaller model test data on Edge device.

2. Update the build condition for openvino GPU so it works for GPU_FP32, GPU_FP16

3. add option to install_ubuntu.sh to exclude the package used for nuphar, so that we can save some disk space as the Edge device usually have limited disk space.
2019-09-11 16:36:12 -07:00
KeDengMS
58fe5a6bf1
Enable Nuphar docker build, and reinstate Nuphar tests (#1757)
Enable Nuphar EP docker build
Revert back to LLVM 6.0.1
Reinstate disabled Softmax tests caused by LLVM 8.0.1
Reinstate Nuphar Python test due to stale sympy version
Increase build timeout of Linux CI
2019-09-05 08:50:48 -07:00
shahasad
6f70a78e1f
Fix a few errors in the NuGet pipeline (still broken) (#1656) 2019-08-21 15:42:23 -07:00
shahasad
c9eb13a638
Copy System.Numerics.Tensors sources from dotnet/corefx into onnxruntime (#1605)
Copy System.Numerics.Tensors sources from dotnet/corefx into onnxruntime
2019-08-15 17:28:47 -07:00
Changming Sun
aeb0bcb4a3 parallel build 2019-08-07 08:38:26 -07:00
Changming Sun
7ee8aca1bf
Avoid downloading test data into C:\ (#1562) 2019-08-05 19:53:15 -07:00
Changming Sun
3045a5f88b
Update test data (#1512)
* Update test data
2019-08-01 10:42:08 -07:00
shahasad
a86486ab7f
Post binary sizes to dashboard database (#1517)
Python script and necessary changes in the azure-pipelines yaml file to post the binary size data from NuGet package build. Currently only posted from CPU pipeline. GPU and other pipelines may be added as necessary.
2019-07-30 08:59:43 -07:00
shahasad
258ff06e42 Revert "publish nuget package to azure blob (#1309)" (#1485)
This reverts commit 1601650161.
2019-07-24 18:07:33 -07:00
shahasad
1601650161 publish nuget package to azure blob (#1309) 2019-07-23 11:07:35 -07:00
jignparm
b41f6eef52 Jignparm/copy cuda extensions (#1462)
* Add CUDA extensions for v 10.0

* Add CUDA extensions for v 10.0

* update path

* change 'vsts' to 'github'
2019-07-22 23:27:48 -07:00
shahasad
768ced703c
Expose provider factory C API, especially for CUDA users (#1461)
Exposed provider factory C API, for cpu and cuda providers, into the published packages.
2019-07-22 19:03:06 -07:00
jignparm
e580b76305
Fix ARM64 build + Add NuGet pipeline including ARM binaries (#1335)
* Add arm64 nocontribops pipeline

* minor fix

* Added new template for arm build -- disable all tests

* fix build command

* add arm64 flag for msbuild

* add arm leg as upstream dependency

* update platform to arm64 for msbuild

* remove test task from arm build

* remove ESRP signing of C# dlls in arm build

* Updated to work for both --arm and --arm64

* Make the cross compiling cmake flags symmetric

* Add dynamic check for /Wno-error flag, instead of extra build option

* remove extra full-stop
2019-07-11 11:49:17 -07:00
Changming Sun
58d6ff3f13 Remove AgentPool setting in CI yaml 2019-07-08 15:40:54 -07:00