Commit graph

39 commits

Author SHA1 Message Date
Brian Martin
09ca58044e Merge branch 'layer_dev' into windowsai 2019-12-06 16:23:05 -08:00
Ashwini Khade
e32eff826c
enable nuget package testing on centos7 (#2527)
* add centos tests to linux cpu ci pipeline

* Disable failing test

* use centos6 instead of centos7

* change back to centos7

* add dotnet runtime dependency

* fix dotnet runtime dependencies

* install dotnet sdk instead of runtimes

* add more dotnet dependencies

* temporary skip failing test

* ix lib path

* reenable failing test
2019-12-03 10:16:45 -08:00
Ori Levari
2cfee5744b
Layer dev release pipeline (#2488)
Adds winml binaries to existing cpu nuget package, and creates new gpu dml nuget package with winml binaries and DML EP.
2019-11-27 11:36:20 -08:00
shahasad
882f28a74b
Fix NuGet end to end tests for custom op dll (#2472) 2019-11-25 15:26:09 -08:00
Ashwini Khade
4caf5c9c13
add additional test data set for nuget pipeline (#2448)
* add SAS token to download internal test data for nuget pipeline

* update azure endpoint

* fix keyvault download step

* fix variable declaration for secret group

* fix indentation

* fix yaml syntax for variables

* fix setting secrets for script

* fix env synctax

* Fix macos pipeline

* attempt to add secrets to windows download data

* fix mac and win data download

* fix windows data download

* update test data set url and location
2019-11-25 13:08:03 -08:00
Changming Sun
e7131f6d12 Pull the latest image before running docker build 2019-11-22 13:48:37 -08:00
Changming Sun
080a0a3186
Nuget pipeline changes (#2305)
1. refactor the pipeline, remove some duplicated code
2. Move Windows_py_GPU_Wheels job to Win-GPU-CUDA10. We'll deprecated the "Win-GPU" pool
3. Delete cpu-nocontribops-esrp-pipeline.yml and cpu-nocontribops-pipeline.yml
4. In Linux nuget jobs, run "make install" before creating the package. So that extra RPAH info will be removed
2019-11-08 09:45:52 -08:00
Changming Sun
138a7f194e Add cleanup step 2019-10-30 08:13:09 -07:00
Changming Sun
ce14b07b1c
Fix the GPU nuget pipeline failure (#2255) 2019-10-25 13:55:38 -07:00
Changming Sun
aef055ebe8
Update nuget pipeline to use CentOS6 (#2211) 2019-10-21 17:55:36 -07:00
shahasad
fcf50ca081 Fix nuget mklml pipeline (#2204)
* some fixes on nuget CPU pipeline

* revert d738c89536

* fix for MKLML package

* fix if else
2019-10-21 08:46:28 -07:00
shahasad
7efc9bdcc7 Some condition fixes on nuget pipeline, to get it green (#2195) 2019-10-19 18:28:12 -07:00
shahasad
35dae992f1
Fix nuget gpu ci test error (#2164)
* fix nuget version extraction script for Gpu packages
* fix cuda version in gpu end-to-end test
2019-10-18 23:01:26 -07:00
shahasad
8803f6fff4
C# end to end test fix, and make end to end tests mandatory (#2079) 2019-10-10 19:23:43 -07:00
stevenlix
544e53e24e Update TensorRT to version 6.0.1.5 (#1966)
* remove onnx-tensorrt submodule

* add new onnx-tensorrt submodule (experiment) for trt6

* update engine build for trt6

* update compile and compute for tensorrt6.0

* Update tensorrt_execution_provider.cc

* Update tensorrt_execution_provider.cc

* Update tensorrt_execution_provider.cc

* Update tensorrt_execution_provider.cc

* switch to onnx-tensorrt master for TensorRT6'

* Update tensorrt_execution_provider.cc

* Handle dynamic batch size and add memcpy in TensorRT EP

* update test cases

* Update tensorrt_execution_provider.cc

* update onnx-tensorrt submodule

* Update Dockerfile.ubuntu_tensorrt

* Update Dockerfile.ubuntu_tensorrt

* Update run_dockerbuild.sh

* Update run_dockerbuild.sh

* Update install_ubuntu.sh

* Update concat_op_test.cc

* Update tensorrt_execution_provider.cc

* Upgrade TensorRT to version 6.0.1.5

* Update onnxruntime_providers.cmake

* Update CMakeLists.txt

* Update reduction_ops_test.cc

* Update install_ubuntu.sh

* Update Dockerfile.ubuntu_tensorrt

* Update Dockerfile.tensorrt

* Update BUILD.md

* Update run_dockerbuild.sh

* Update install_ubuntu.sh

* Update onnxruntime_providers.cmake

* Update install_ubuntu.sh

* Update install_ubuntu.sh

* Update gemm_test.cc

* Update gather_op_test.cc

* Update CMakeLists.txt

* Removed submodule

* update onnx-tensorrt submodule

* Add Ubuntu18.04 build option

* Add Ubuntu18.04 build option

* Add Ubuntu18.04 build option

* Add Ubuntu18.04 build option

* Remove redundency

* Fix issue that it does not add memcopy node correctly if some nodes fall back to CUDA EP.
e.g. after partition, there's TRT_Node -> Cuda_node (with CPU memory expected), we still need to add memcpy node between them.

* update for Trt Windows build

* Update onnxruntime_providers.cmake

* Disable opset11 tests on TensorRT

* Update pad_test.cc

* Update build.py

* update scripts for ubuntu18.04

* Disable warning for Windows build
2019-10-06 10:40:53 -07:00
shahasad
b355193841
Add Date-time stamp in NuGet package versioning for appropriate ordering of the packages (#1951) 2019-09-30 16:24:16 -07:00
Changming Sun
d46e023ee4
Remove toolset=14.11 for CUDA build (#1921) 2019-09-25 15:19:37 -07:00
Pranav Sharma
1f4190de3c
Disable Linux x86 builds as they're not required any more. (#1887)
* Mention OrtCreateSessionFromArray in C API doc

* Disable Linux x86 builds as they're not required any more.

* more ...
2019-09-23 00:26:23 -07:00
shahasad
aac6021549
Add NuGet feed publish to nuget pipeline (#1833) 2019-09-13 15:27:35 -07:00
shahasad
833e18345d
Publish perf tool with nightly build (#1728) 2019-08-30 11:25:55 -07:00
shahasad
f25847bccd
More fixes on the NuGet CPU CI pipeline (#1688)
- Fix the Windows end-to-end test in NuGet CI
- Skip the TestModelSerialization, because it is failing on Linux. Must be fixed before API is released for use. Owner is notified.
2019-08-23 18:13:13 -07:00
stevenlix
1c5b15c2b8
Remove memory copy between TensorRT and CUDA (#1561)
* remove memory copy between CUDA and TRT

* add info to RegisterExecutionProvider input

* use new IDeviceAllocator for trt allocator

* remove SetDefaultInputsMemoryType from TRT EP

* remove onnx-tensorrt 5.0

* add submodule onnx-tensorrt branch 5.1

* remove redundancy

* Update transformer_memcpy.cc

* Update tensorrt_execution_provider.cc

* switch to TensorRT 5.1.5.0

* update python binding

* disable failed test case on TensorRT

* Update activation_op_test.cc

* upgrade to TensorRT container 19.06

* update according to feedback

* add comments

* remove tensorrt allocator and use cuda(gpu) allocator

* update onnx-tensorrt submodule

* change ci build cuda directory name
2019-08-08 19:31:39 -07:00
Changming Sun
7ee8aca1bf
Avoid downloading test data into C:\ (#1562) 2019-08-05 19:53:15 -07:00
Hariharan Seshadri
624411bb69
Upload correct ESRP signed package (#1531) (#1534) 2019-08-01 10:56:18 -07:00
Hariharan Seshadri
4d768b3a0f
Fix inclusion of ARM binary in the release pkg (#1513) (#1521)
* Fix inclusion of ARM binary in the release pkg

* Add lib and pdb as well
2019-07-31 15:57:03 -07:00
shahasad
fb5d0fc538
Publish nuget package to azure blob store (#1525)
Publish daily build NuGet package to Azure blob store for sharing among internal partners
2019-07-31 14:17:54 -07:00
shahasad
a86486ab7f
Post binary sizes to dashboard database (#1517)
Python script and necessary changes in the azure-pipelines yaml file to post the binary size data from NuGet package build. Currently only posted from CPU pipeline. GPU and other pipelines may be added as necessary.
2019-07-30 08:59:43 -07:00
Hariharan Seshadri
6df4bc2ebe
Update scripts to access pipeline variables correctly (#1499)
* Update scripts to access IsReleaseBuild pipeline variable correctly

* Correct access of PACKAGENAME pipeline variable

* Fix Linux CUDA 10 package tests

* Enable C# GPU test

* Update
2019-07-25 15:30:32 -07:00
shahasad
258ff06e42 Revert "publish nuget package to azure blob (#1309)" (#1485)
This reverts commit 1601650161.
2019-07-24 18:07:33 -07:00
jignparm
a8e9e1878e
Reduce artifacts size (#1477)
* Update wildcard pattern to match only relevant archives

* Update TensorRT build to add CUDA VS extensions
2019-07-23 22:23:51 -07:00
shahasad
1601650161 publish nuget package to azure blob (#1309) 2019-07-23 11:07:35 -07:00
Yufeng Li
6be93f11e5
build mklml/ngraph without openmp (#1460)
cleanup the option to build mklml/ngraph without openmp
2019-07-22 16:59:32 -07:00
jignparm
2c05291908
Jignparm/patch 0001 (#1419)
* remove extra $ from 8592

* fix
2019-07-20 17:07:56 -07:00
jignparm
9e4ac8c66a
remove mkldnn from gpu nuget package (#1443) 2019-07-20 11:51:00 -07:00
jignparm
e580b76305
Fix ARM64 build + Add NuGet pipeline including ARM binaries (#1335)
* Add arm64 nocontribops pipeline

* minor fix

* Added new template for arm build -- disable all tests

* fix build command

* add arm64 flag for msbuild

* add arm leg as upstream dependency

* update platform to arm64 for msbuild

* remove test task from arm build

* remove ESRP signing of C# dlls in arm build

* Updated to work for both --arm and --arm64

* Make the cross compiling cmake flags symmetric

* Add dynamic check for /Wno-error flag, instead of extra build option

* remove extra full-stop
2019-07-11 11:49:17 -07:00
jignparm
57225cd4ee
Add C++ API test for NuGet package (#1364) 2019-07-09 13:51:51 -07:00
jignparm
a56b294428
Activate compliance tasks for private builds, and also set a daily scheduler (#1280)
* add compliance and build schedules

* cpu-esrp-pipeline.yml

* update schedule time for testing

* add schedules to all pipelines
2019-06-25 11:13:55 -07:00
jignparm
d3e5474c1d
Refactor CI pipelines - add GPU NuGet pipelines and ESRP code signing steps (#1247)
* Simplify linux gpu pipeline

* Refactor win-gpu-ci-pipeline.yml

* Set cuda environment variables for testing and version

* Remove variables from starter script

* minor fix

* Add GPU Nuget pipeline

* Set DisableContribOps environment variable for Linux package tests

* Add ESRP tasks

* Add ESRP signing templates

* Test out hardcode value of ERSP

* Test out hardcode value of ERSP

* Test out hardcode value of ERSP

* Test out hardcode value of ERSP

* test variable expansion

* test variable expansion

* test variable expansion

* test variable expansion

* test variable expansion

* test variable expansion

* test out variable expansion

* test variable expansion

* test variable expansion

* test variable expansion

* test variable expansion

* test variable expansion

* test variable expansion

* test variable expansion

* test variable expansion

* update cpu pipeline to conditionally esrp sign

* Set C# GPU tests to run only if env var is set

* Refactor for easy parameter passing

* refactored esrp templates

* remove variables from template

* Add packaging variables back to pipelines

* update C# for cuda 10

* Merge vars ana parameters for gpu pipeline

* remove vars from mklml pipeline

* display envvars on terminal

* Clean up C# cuda tests, and upgrade to Cuda10

* Introduce CUDNN_PATH pipeline varaible

* YAML variable are always uppercased (not true with classic)

* Update C# GPU test to be more meaningful

* remove macos from gpu tests

* remove debugging info for DisableContribOps option

* Remove DisableContrib ops parameters -- use variables only

* Fix typo from = to -

* remove debug steps

* fix typo

* remove unused variable TESTONGPU from some templates

* clean up CUDA env setup scripts

* Remove CUDNN_PATH from setup_env_cuda.bat
2019-06-20 19:41:30 -07:00
jignparm
08731589c9
Refactor CI pipelines, and add YAML NuGet package generation pipelines ( for CPU, MKLML, NoContribOps) (#1223)
* Initial check in

* Add win x86

* minor update to x86

* update win-ci

* update win-ci

* update win-x86ci

* add linux and mac templates

* add nuget pipelines and test templates

* remove buildConfig

* add compliance template

* fix minor typos

* update pool for macos

* update mac agent pool

* update macos pool

* update agent pools for tests

* turn off debug build for testing

* some modifications to packaging scripts

* change ordering of compliance tasks

* Add mklml pipeline

* Add packagename variable to mklml pipeline

* remove unrequired dependent jobs from mklml pipeline

* Update build command for macOS legs in mklml and cpu pipeline

* Set vcvars to true

* Add no contrib ops pipeline

* Add no-contrib-ops pipeline

* set vcvars to true for package tests

* remove repetition in nuget templates

* get buildarch correct

* get name of test template correct

* remove steps from test_all_os.yml

* add parameters to test_all_os.yml

* Need jobs, not steps

* set envars for disablecontrib ops

* add cleanup tasks and CG to package tests

* fix path to cleanup script for macos

* remove buildDirectory -- not needed

* remove fp16tiny_yolov2 model from nocontribops tests

* remove debugging info

* fix individual linux pipelines to use correct template

* remove unneeded bak_latest2

* increase timeout to 120 to allow for variance

* turn off code coverage report
2019-06-14 14:51:03 -07:00