Commit graph

353 commits

Author SHA1 Message Date
Edward Chen
8d09cefafc Merge remote-tracking branch 'origin/ort_training' into edgchen1/merge_from_master 2020-04-22 16:56:15 +00:00
edgchen1
5492d02c4e
Remove Windows CUDA 9 build definition and helper scripts. (#3615) 2020-04-21 15:22:27 -07:00
Edward Chen
47f1758fdc Add --skip_onnx_tests to orttraining Windows builds. 2020-04-21 21:50:35 +00:00
Edward Chen
297ab43b0c Add --enable_onnx_tests to Windows builds to allow set up of test data directory. 2020-04-21 20:34:55 +00:00
Edward Chen
daa14b64e3 Merge remote-tracking branch 'origin/master' into edgchen1/merge_from_master 2020-04-21 03:31:32 +00:00
Changming Sun
911d125323 Remove openmp from gpu build 2020-04-20 17:13:54 -07:00
liqunfu
781e1c36be
Add front-end MNIST test (#3231)
* add frontend minst test

* to use torch nightly with torchvision

* remove incorrect comment per reviewer's comment

* experiment torchvision import failure

* experiment install_deps.sh

* more experiment install_deps.sh

* experiment install_deps.sh with --upgrade

* Experiment with install_deps.sh.

* Experiment with install_ubuntu.sh.

* Use Ubuntu 18.04 and Python 3.6 for CI.

* Update cmake version for CI.

* Install MPI on Ubuntu 18.04 for CI.

* Increase tolerance for MNIST test.

* Go back to Ubuntu 16.04 for CI, fix installing from deadsnakes ppa.

* Clean-up.

* Update ort_trainer.py from ort_training.

* Get default Ubuntu Python ver back to 3.5.

* Add underscore to opset_version parameter name in ORTTrainer constructor.

* Move loss/model wrap before the call for sample output.

* Update expected values for MNIST test.

Co-authored-by: liqun <liqun@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
Co-authored-by: Sergii Dymchenko <sedymche@microsoft.com>
2020-04-20 11:19:31 -07:00
Changming Sun
d68245853e
Disable downloading test data on Linux (#3581) 2020-04-18 15:54:58 -07:00
Hector Li
5acd8dbe7d
remove option --enable_lto (#3515) 2020-04-17 14:18:56 -07:00
Sheil Kumar
2717c178cc
Fork the WinML APIs into the Microsoft namespace (#3503)
* Migrate winml to Microsoft Namespace (packaging changes are pending)

* add ns_prefix toggle

* fix packaging

* Users/sheilk/add missing raw header (#3484)

* add dualapipartition

* wrong variable for repo root

Co-authored-by: Sheil Kumar <sheilk@microsoft.com>

* remove existence check to force failures

* extra paren

* dualapipartition needs to be referenced from the source

* add microsoft.ai.machinelearning.dll to the output dir

* rename the idl file so that assembly info is correctly added into the winmd

* fix namespaces

* update namespaces

* default to microsoft, and add namespace override as build argument

* update cmakesetings.json as well

* remove from cmakelists.txt

Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
Co-authored-by: Changming Sun <chasun@microsoft.com>
2020-04-17 06:18:54 -07:00
Changming Sun
1a222b3f6e
Disable downloading test data on Windows (#3551)
* Disable downloading test data on Windows
2020-04-16 22:15:20 -07:00
harshitha
80e0c64e2e merged with master 2020-04-16 17:13:36 +00:00
Changming Sun
7c89f38a34
Fix static analysis warnings found by VC++ (#3530)
1. Fix static analysis warnings found by VC++
2. Add a new pipeline for static analysis
3. Merge all the windows CI build into one single yaml file.(Easier to queue them all).
4. Make DNNL build faster by disabling building the tests and examples.
5. Enable custom op unitest.
2020-04-16 01:46:47 -07:00
David Brownell
72cd61baae
Removed use of parameters in python wheel build scripts (#3524) 2020-04-15 10:31:14 -07:00
Changming Sun
a2feb29b0d
Fix build break (#3528)
Ignore some known test failures
Install ONNX package before running Windows CI builds
2020-04-14 18:07:56 -07:00
David Brownell
006c5be1b1
Optionally produce a python wheel that includes featurizers (#3491) 2020-04-14 09:00:13 -07:00
M. Zeeshan Siddiqui
5d99f179b9
Merge pull request #3486 from microsoft/sedymche/merge_master_ort_training
Merge from master into ort_training
2020-04-13 10:55:36 -07:00
Sergii Dymchenko
bf3df41424 Put back SubmoduleCheckoutMode parameter into mac-ci.yml. 2020-04-12 21:49:38 -07:00
George Wu
7f6e407e09
fix python packaging manylinux1 build break. (#3482) 2020-04-11 06:58:22 +08:00
edgchen1
cffdff6702
Publish unit test results from Linux and Mac builds (#3480)
* Added publish test results step to Linux and Mac builds.

* Fix test result file pattern.
2020-04-10 14:51:56 -07:00
liqunfu
e7297e6c9d
create pipeline for ci frontend tests (#3422)
create pipeline for nightly python front-end e2e tests
2020-04-09 15:31:22 -07:00
Sergii Dymchenko
6ba7c99e50 Merge branch 'master' into ort_training 2020-04-09 12:42:04 -07:00
Changming Sun
33006f48c0
Update onnx submodule to 1.7.0 release candidate (#3405)
Update onnx submodule to 1.7.0 release candidate.  This isn't a release tag,  but it will be released soon, in 1-2 weeks.
2020-04-04 16:23:42 -07:00
Changming Sun
a5fea26cb4 Disable model tests for Mac OS X builds 2020-04-02 15:14:32 -07:00
Thiago Crepaldi
759818f2c1 Merge remote-tracking branch 'origin/master' into thiagofc/ort_training_merge_from_master 2020-03-31 10:53:22 -07:00
stevenlix
2332a93db0
Update onnx-tensorrt parser (#3369)
* sync onnx-tensorrt parser and update TensorRT doc

* remove --msvc_toolset 14.16 in tensorrt ci pipeline
2020-03-30 20:31:59 -07:00
Xueyun Zhu
ccc3535e72 resolve conflict 2020-03-20 20:20:35 +00:00
Tiago Koji Castro Shibata
3bdb0b620a
Fix WCOS/Win32 linking bugs (#3126)
* Fix WCOS/Win32 linking bugs

* Remove unused NODEFAULTLIB flags

* Avoid plain target_link_libraries signature

* Avoid plain target_link_libraries signature

* Fix library list escaping

* Use library list instead of string

* Remove duplicate link to windowsapp.lib

* Remove Win32 build workarounds

* Specify CMake policies before initializing language

* Expose Win32 header definitions during build

* Force set API family

* Enable Win32 APIs in featurizer

* Use MT dynamic CRT

* Expose Win32 specific functions

* Disable app container globally

* Disable default wide functions in featurizers

* Add featurizers to test include path

* Workaround https://gitlab.kitware.com/cmake/cmake/issues/19428

* Revert pipeline debugging hacks

* Skip /FI in CUDA sources

* Default to Win32 builds

* Enable WCOS when using WinML

* Use generator expression to apply CMAKE_MSVC_RUNTIME_LIBRARY to C++ only
2020-03-19 08:52:40 -07:00
Changming Sun
0fceb33288
Fix onnxruntime server docker file build failure (#3219)
1. Fix onnxruntime server docker file build failure. Tested with the notebook in ONNX tutorial, it works well.
2. Delete the docker files for the other EPs, because currently they don't work and I don't have enough time to update them.
2020-03-15 14:46:46 -07:00
Tracy Sharpe
fe0b2b2abd
QLinearConv speed up (#3196)
For x86/x64 builds, change the QLinearConv op to use MLAS for the u8u8=s32 GEMM, then requantize the intermediate buffer to u8.
2020-03-13 16:54:55 -07:00
Zeeshan Siddiqui
2cad08bd60 Merged PR 5688: Upgrade ONNX submodule to the latest from github ONNX master.
We want to implement SoftmaxCrossentropy and NegativeLossLikelihoodLoss forward training ops for opset-12 but that requires ONNX submodule to point to the latest commit to have the latest and greatest ONNX spec!

- Reverse integrate changes from *.in.proto files in github ONNX repo.
- Regenerate csharp/test/Microsoft.ML.OnnxRuntime.Tests/OnnxMl.cs
- Disable ONNX tests that don't have op implementation for the latest opset.
2020-03-12 16:51:45 -07:00
edgchen1
fa4dd51e3b
Add back orttraining-linux-gpu-inference-only-ci-pipeline.yml. (#3182) 2020-03-11 18:03:58 -07:00
Edward Chen
80dd62a240 Enable CI for training. 2020-03-11 14:41:32 -07:00
Edward Chen
e542cfd0e0 Introduce training changes. 2020-03-11 14:39:03 -07:00
Hariharan Seshadri
3464801c3e
Explicitly specify NugetPackage parameter while validating nuget in some release pipelines (#3139) 2020-03-10 15:14:09 -07:00
Dmitri Smirnov
f87b6913cd
Add package download step before pushing to feeds (#3162)
Add package download step before publishing.
2020-03-09 14:32:18 -07:00
Changming Sun
6ed5d7c332
Update post_binary_sizes_to_dashboard.py (#3161)
Discussed with Faith, because the data size is very small and changes are gradual, there is no need to delete the old data. We want to keep all the history.
2020-03-09 13:21:58 -07:00
Tiago Koji Castro Shibata
a59243090a
Publish release symbols (#3152)
* Publish release symbols

* Publish symbols if IsReleaseBuild
2020-03-05 22:32:18 -08:00
Dmitri Smirnov
e2894c5ffb
Fix package name overrides (#3150)
Add env var with the package name.
2020-03-05 17:10:55 -08:00
Dmitri Smirnov
2c446a7f2f
Add push to ORT-NIGHTLY. (#3146) 2020-03-05 11:38:22 -08:00
Dmitri Smirnov
ef8768a53f
Override native package name. Preserve managed package name the same. (#3133)
Override native package name. Preserve managed package name the same.
  Specify pckage name for validation purposes.
 Fix up validation package name parameter.
2020-03-04 10:12:55 -08:00
Changming Sun
12605f05d1
Fix CUDA PATH (#3131)
Previously, we put the "bin" folder of all the CUDA verions in the system PATH. And 10.2 is in the front. It's a mess.
So I've removed all of them from the system PATH env. But I need to add one of them back through build scripts.

(The problem only affect the C# test, not the C/C++ tests that forked from build.py).
2020-03-03 14:34:19 -08:00
smk2007
6cdd2b4934
Enable DML Nuget Package for x64 or x86 architectures (#3120)
* add dml gpu pipelines

* add x86 to the gpu dml dev build pipeline

* Enable DML x86 builds

* Fix uint64_t -> size_t warning

* fix warnings

* enable dml on x86 ci builds

* operatorHelper 773 error uint32_t vs uint64_t

* operatorHelper 773 error uint32_t vs uint64_t

* make x86 pipeline use the gpu pool

* more warnings

* fix x86 directml path

* make dml nuget package

* disable tf_pnasnet_large

* disable zfnet512

* make validation use wildcards

* disable x86 dml gpu tests

* add args.

* update gpu.yml

* change nupkg wildcard

* add debug statements

* package x86 dml nupkg

* dont drop managed nuget again from dml pipeline build

* Add DML EULA

* directml license should be renamed to not clobber the existing license

* casing on dml package....

* {} to ()

* fix license name

* disable dml from x86 ci

* typo and cr feedback

* remove featurizers

* ship the dml pdb as well
2020-03-02 20:18:46 -08:00
Dmitri Smirnov
e45326b5df
Create NuGet packaging pipeline for ORT Featurizers (#3125)
Create a new pipeline to publish ORT with Featurizers
  Update pipeline for two separate packages.
  Change package names.
2020-03-02 17:00:56 -08:00
Hariharan Seshadri
86b755774f
Create a separate Nuget hosting just managed assemblies (#3020)
* Initial commit

* More changes

* More changes

* More changes 3

* More changes 4

* More changes 5

* More changes 5

* More changes 6

* More changes 7

* More changes 8

* Remove C# ifdefs

* More changes 10

* More changes 11

* YAML changes for other release pipelines

* Add release notes metadata

* Props and Targets change

* Add CSHarp proj

* More changes 12

* More changes

* Minor fix

* Minor fix

* Fix yaml

* Some missing logic for winml

* Minor update

* Fix casing for winmd file

* Fix casing

* Add targets and props for managed section into native nuget

* revert file

* a
2020-02-27 18:00:17 -08:00
daquexian
37a905f557
Make Java API available on Android (#3030) 2020-02-27 08:23:50 -08:00
Changming Sun
d7500b26bd
Remove Publish Build Symbols from pre-checkin CI build (#3088) 2020-02-25 08:02:36 -08:00
stevenlix
f4a5d17294
Upgrade to CUDA10.2 for TensorRT (#3084)
* Switch to CUDA10.2

* Update win-gpu-tensorrt-ci-pipeline.yml

* Update win-gpu-tensorrt-ci-pipeline.yml

* remove dynamic_shape

* update onnx-tensorrt submodule

* check if input shape is specified for TensorRT subgraph input and enable some TensorRT unit tests

* fix format issue

* add shape inference instruction for TensorRT

* update according to the reviews

* Update win-gpu-tensorrt-ci-pipeline.yml
2020-02-25 05:36:01 -08:00
Hariharan Seshadri
d7f2cdcc7e
Fix target platform of managed OnnxRuntime dll and enable x86 .NET testing (#3056)
* WIP: Re-enable x86 .NET testing in Release pipelines

Enabling x86 testing will make sure that ORT packages doesn’t break x86 projects of customers

* Remove setting some env variables

* Comment out a test failing on x86 builds

* More changes

* Minor fix

* More changes

* More changes

* s

* s

* s

* Revert minor change

* More changes

* More changes

* More changes 2

* explicitly set platform target

* Delete bin and obj folders

* Clean output dirs

* Add back TargetFramwork

* Disable x86 .net framework tests

* Skip x86 tests in MKLML pipeline
2020-02-24 23:02:59 -08:00
Dmitri Smirnov
dae9a31719
Introduce new Featurizers packaging pipeline. (#3068)
Introduce new Featruizers packaging pipeline.
2020-02-24 13:57:38 -08:00