Commit graph

415 commits

Author SHA1 Message Date
Changming Sun
911d125323 Remove openmp from gpu build 2020-04-20 17:13:54 -07:00
Prabhat
381fee47ab
Added support to build onnxruntime with ACL (#3586)
* Added support to build onnxruntime with ACL

* Added ACL build instructions
2020-04-20 13:35:28 +05:30
Prabhat
ea62b3435a
Clean up build.py code (#3466) 2020-04-18 20:48:30 -07:00
Changming Sun
d68245853e
Disable downloading test data on Linux (#3581) 2020-04-18 15:54:58 -07:00
Hariharan Seshadri
b4457ecb7a
Fix gen_doc build option and refresh documentation (#3545)
* Support listing keys in custom metadata map via C/C++ API

* nit

* PR feedback

* Nit

* Initial commit

* More changes

* Support listing keys in custom metadata map via C/C++ API

* nit

* PR feedback

* Nit

* Initial commit

* More changes

* Add md files

* Doc changes

* Update

* revert cmake changes

* Update

* Doc change

* Update

* Update
2020-04-17 14:41:04 -07:00
Hector Li
5acd8dbe7d
remove option --enable_lto (#3515) 2020-04-17 14:18:56 -07:00
Sheil Kumar
2717c178cc
Fork the WinML APIs into the Microsoft namespace (#3503)
* Migrate winml to Microsoft Namespace (packaging changes are pending)

* add ns_prefix toggle

* fix packaging

* Users/sheilk/add missing raw header (#3484)

* add dualapipartition

* wrong variable for repo root

Co-authored-by: Sheil Kumar <sheilk@microsoft.com>

* remove existence check to force failures

* extra paren

* dualapipartition needs to be referenced from the source

* add microsoft.ai.machinelearning.dll to the output dir

* rename the idl file so that assembly info is correctly added into the winmd

* fix namespaces

* update namespaces

* default to microsoft, and add namespace override as build argument

* update cmakesetings.json as well

* remove from cmakelists.txt

Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
Co-authored-by: Changming Sun <chasun@microsoft.com>
2020-04-17 06:18:54 -07:00
Sheil Kumar
951484ba53
Dualapipartitionattibute.h header is missing in nuget package (#3350)
* add dualapipartition

* wrong variable for repo root

Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
2020-04-16 22:21:57 -07:00
Changming Sun
1a222b3f6e
Disable downloading test data on Windows (#3551)
* Disable downloading test data on Windows
2020-04-16 22:15:20 -07:00
Changming Sun
7c89f38a34
Fix static analysis warnings found by VC++ (#3530)
1. Fix static analysis warnings found by VC++
2. Add a new pipeline for static analysis
3. Merge all the windows CI build into one single yaml file.(Easier to queue them all).
4. Make DNNL build faster by disabling building the tests and examples.
5. Enable custom op unitest.
2020-04-16 01:46:47 -07:00
David Brownell
72cd61baae
Removed use of parameters in python wheel build scripts (#3524) 2020-04-15 10:31:14 -07:00
Changming Sun
a2feb29b0d
Fix build break (#3528)
Ignore some known test failures
Install ONNX package before running Windows CI builds
2020-04-14 18:07:56 -07:00
Yufeng Li
baa86f181f
Handle the case that initializers are in graph input (#3449)
warn that initializers are in graph input
provide a tool to move initializer out of graph input
Motivation and Context
ONNX model from IR_VERSION 4 only treats initializers that appear in graph input as non-constant. This may fail some of the graph optimizations, like const folding, operator fusion and etc. Warn the case and provide a tool.
2020-04-14 09:06:04 -07:00
David Brownell
006c5be1b1
Optionally produce a python wheel that includes featurizers (#3491) 2020-04-14 09:00:13 -07:00
George Wu
7f6e407e09
fix python packaging manylinux1 build break. (#3482) 2020-04-11 06:58:22 +08:00
Changming Sun
33006f48c0
Update onnx submodule to 1.7.0 release candidate (#3405)
Update onnx submodule to 1.7.0 release candidate.  This isn't a release tag,  but it will be released soon, in 1-2 weeks.
2020-04-04 16:23:42 -07:00
Changming Sun
a5fea26cb4 Disable model tests for Mac OS X builds 2020-04-02 15:14:32 -07:00
Tiago Koji Castro Shibata
1c334ed0f1
Add Ninja generator to build.py (#3331) 2020-04-01 14:19:22 -07:00
Tiago Koji Castro Shibata
a61400de01
Fix ARM cross compilation (related to #3378, #3298) (#3385) 2020-03-31 17:10:48 -07:00
stevenlix
2332a93db0
Update onnx-tensorrt parser (#3369)
* sync onnx-tensorrt parser and update TensorRT doc

* remove --msvc_toolset 14.16 in tensorrt ci pipeline
2020-03-30 20:31:59 -07:00
Jan Scholz
ce9acf0c21
iOS crosscompilation under linux (#3298)
* added support for ios crosscompilation under linux

* reverted cmake generator change

* if --ios is added protoc can be compiled for host system

* accidently reverted change to compile protoc for host system for ios if protoc exe is not set

* wdata is now used

* accidentally pasted CMAKE_OSX_ARCHITECTURES into CmakeLists.txt, also made bad merge on build.py previously

* removed print

* fixed typeo, deleted commented statements for earlier debugging

* reverted accidental delete

* added asmmacro.h for aarch64 asm
now MlasSgemmKernel**** gets underscore added if needed
no need anymote to differentiate between iOS arm64 and normal amr64 build
onnxruntime.cmake: added check if iOSCross is set to properly set RPATH

* removed 2 spaces

* fix: logcial error fixed, now protoc gets compiled if not supplied with --path_to_protoc_exe

* removed unecessarily added spaces

* removed some more spaces
2020-03-30 19:39:17 -07:00
Tiago Koji Castro Shibata
3bdb0b620a
Fix WCOS/Win32 linking bugs (#3126)
* Fix WCOS/Win32 linking bugs

* Remove unused NODEFAULTLIB flags

* Avoid plain target_link_libraries signature

* Avoid plain target_link_libraries signature

* Fix library list escaping

* Use library list instead of string

* Remove duplicate link to windowsapp.lib

* Remove Win32 build workarounds

* Specify CMake policies before initializing language

* Expose Win32 header definitions during build

* Force set API family

* Enable Win32 APIs in featurizer

* Use MT dynamic CRT

* Expose Win32 specific functions

* Disable app container globally

* Disable default wide functions in featurizers

* Add featurizers to test include path

* Workaround https://gitlab.kitware.com/cmake/cmake/issues/19428

* Revert pipeline debugging hacks

* Skip /FI in CUDA sources

* Default to Win32 builds

* Enable WCOS when using WinML

* Use generator expression to apply CMAKE_MSVC_RUNTIME_LIBRARY to C++ only
2020-03-19 08:52:40 -07:00
Pranav Sharma
435f014d71
Add support for sessions to share a global threadpool. (#3177)
* Add support for sessions to share a global threadpool.

* Fix build issues

* Add tests, fix build issues.

* Added some documentation

* Fix centos issue when threadpools become nullptr due to 1 core.

* Fix mac and x86 build issues

* Address some PR comments

* Disabled test for android, added few more tests and addressed more PR comments.

* const_cast
2020-03-18 15:42:46 -07:00
Changming Sun
0fceb33288
Fix onnxruntime server docker file build failure (#3219)
1. Fix onnxruntime server docker file build failure. Tested with the notebook in ONNX tutorial, it works well.
2. Delete the docker files for the other EPs, because currently they don't work and I don't have enough time to update them.
2020-03-15 14:46:46 -07:00
Tracy Sharpe
fe0b2b2abd
QLinearConv speed up (#3196)
For x86/x64 builds, change the QLinearConv op to use MLAS for the u8u8=s32 GEMM, then requantize the intermediate buffer to u8.
2020-03-13 16:54:55 -07:00
Ori Levari
93569bf0f4
fix regex to populate dll version information correctly 2020-03-13 11:35:49 -07:00
Hariharan Seshadri
3464801c3e
Explicitly specify NugetPackage parameter while validating nuget in some release pipelines (#3139) 2020-03-10 15:14:09 -07:00
Dmitri Smirnov
f87b6913cd
Add package download step before pushing to feeds (#3162)
Add package download step before publishing.
2020-03-09 14:32:18 -07:00
Changming Sun
6ed5d7c332
Update post_binary_sizes_to_dashboard.py (#3161)
Discussed with Faith, because the data size is very small and changes are gradual, there is no need to delete the old data. We want to keep all the history.
2020-03-09 13:21:58 -07:00
Tiago Koji Castro Shibata
a59243090a
Publish release symbols (#3152)
* Publish release symbols

* Publish symbols if IsReleaseBuild
2020-03-05 22:32:18 -08:00
Dmitri Smirnov
e2894c5ffb
Fix package name overrides (#3150)
Add env var with the package name.
2020-03-05 17:10:55 -08:00
Dmitri Smirnov
2c446a7f2f
Add push to ORT-NIGHTLY. (#3146) 2020-03-05 11:38:22 -08:00
Dmitri Smirnov
ef8768a53f
Override native package name. Preserve managed package name the same. (#3133)
Override native package name. Preserve managed package name the same.
  Specify pckage name for validation purposes.
 Fix up validation package name parameter.
2020-03-04 10:12:55 -08:00
Changming Sun
12605f05d1
Fix CUDA PATH (#3131)
Previously, we put the "bin" folder of all the CUDA verions in the system PATH. And 10.2 is in the front. It's a mess.
So I've removed all of them from the system PATH env. But I need to add one of them back through build scripts.

(The problem only affect the C# test, not the C/C++ tests that forked from build.py).
2020-03-03 14:34:19 -08:00
smk2007
6cdd2b4934
Enable DML Nuget Package for x64 or x86 architectures (#3120)
* add dml gpu pipelines

* add x86 to the gpu dml dev build pipeline

* Enable DML x86 builds

* Fix uint64_t -> size_t warning

* fix warnings

* enable dml on x86 ci builds

* operatorHelper 773 error uint32_t vs uint64_t

* operatorHelper 773 error uint32_t vs uint64_t

* make x86 pipeline use the gpu pool

* more warnings

* fix x86 directml path

* make dml nuget package

* disable tf_pnasnet_large

* disable zfnet512

* make validation use wildcards

* disable x86 dml gpu tests

* add args.

* update gpu.yml

* change nupkg wildcard

* add debug statements

* package x86 dml nupkg

* dont drop managed nuget again from dml pipeline build

* Add DML EULA

* directml license should be renamed to not clobber the existing license

* casing on dml package....

* {} to ()

* fix license name

* disable dml from x86 ci

* typo and cr feedback

* remove featurizers

* ship the dml pdb as well
2020-03-02 20:18:46 -08:00
Dmitri Smirnov
e45326b5df
Create NuGet packaging pipeline for ORT Featurizers (#3125)
Create a new pipeline to publish ORT with Featurizers
  Update pipeline for two separate packages.
  Change package names.
2020-03-02 17:00:56 -08:00
Hariharan Seshadri
86b755774f
Create a separate Nuget hosting just managed assemblies (#3020)
* Initial commit

* More changes

* More changes

* More changes 3

* More changes 4

* More changes 5

* More changes 5

* More changes 6

* More changes 7

* More changes 8

* Remove C# ifdefs

* More changes 10

* More changes 11

* YAML changes for other release pipelines

* Add release notes metadata

* Props and Targets change

* Add CSHarp proj

* More changes 12

* More changes

* Minor fix

* Minor fix

* Fix yaml

* Some missing logic for winml

* Minor update

* Fix casing for winmd file

* Fix casing

* Add targets and props for managed section into native nuget

* revert file

* a
2020-02-27 18:00:17 -08:00
daquexian
37a905f557
Make Java API available on Android (#3030) 2020-02-27 08:23:50 -08:00
Ori Levari
5e0f7412cd
Properly handle downlevel and WCOS scenarios (#3075) 2020-02-25 17:47:02 -08:00
Changming Sun
d7500b26bd
Remove Publish Build Symbols from pre-checkin CI build (#3088) 2020-02-25 08:02:36 -08:00
stevenlix
f4a5d17294
Upgrade to CUDA10.2 for TensorRT (#3084)
* Switch to CUDA10.2

* Update win-gpu-tensorrt-ci-pipeline.yml

* Update win-gpu-tensorrt-ci-pipeline.yml

* remove dynamic_shape

* update onnx-tensorrt submodule

* check if input shape is specified for TensorRT subgraph input and enable some TensorRT unit tests

* fix format issue

* add shape inference instruction for TensorRT

* update according to the reviews

* Update win-gpu-tensorrt-ci-pipeline.yml
2020-02-25 05:36:01 -08:00
Hariharan Seshadri
d7f2cdcc7e
Fix target platform of managed OnnxRuntime dll and enable x86 .NET testing (#3056)
* WIP: Re-enable x86 .NET testing in Release pipelines

Enabling x86 testing will make sure that ORT packages doesn’t break x86 projects of customers

* Remove setting some env variables

* Comment out a test failing on x86 builds

* More changes

* Minor fix

* More changes

* More changes

* s

* s

* s

* Revert minor change

* More changes

* More changes

* More changes 2

* explicitly set platform target

* Delete bin and obj folders

* Clean output dirs

* Add back TargetFramwork

* Disable x86 .net framework tests

* Skip x86 tests in MKLML pipeline
2020-02-24 23:02:59 -08:00
Dmitri Smirnov
dae9a31719
Introduce new Featurizers packaging pipeline. (#3068)
Introduce new Featruizers packaging pipeline.
2020-02-24 13:57:38 -08:00
kile0
f367fd921c
Use a custom allocator for temporary buffers in reduction_ops.cc (#2775)
* port the mimalloc allocator

* hook mimalloc opt into common.h and reduction ops

* repurpose USE_MIMALLOC to only denote subbing in of default allocator with mimalloc and some refactoring

* fix unintended cherry pick diffs

* polish alloctor_mimalloc

* explicitly disable mimalloc where it already had been disabled

* update mimalloc to pull in stl allocator

* switch mimalloc stl allocator to use mimalloc library version

* turn mimalloc on by default (only the stl changes are enabled, the python interacting ones are off already and shall remain so)

* move FastAllocVector into cpu specific code

* separate out defines into arena and stl changes

* the rest of the define renames

* bfc arena allocator

* some typos and rename the bfc arena allocator to fit existing class naming conventions

* adjustments in response to comments

* different template instantiations are friends
2020-02-23 16:04:30 +10:00
Changming Sun
61ae134469
Fix binary size report (#3080) 2020-02-22 21:01:06 -08:00
Changming Sun
fb871978b5
Adjust build flags for the release pipelines (#3066)
1. Add LTCG back. It was set to default OFF in my previous PR to speed up Windows build. It is only needed in release pipelines.
2. Remove --use_featurizers from all the packaging pipelines
3. Make sure all the packages have openmp
2020-02-21 16:45:42 -08:00
Changming Sun
179603775f
Use CUDA 10.1 for Linux build (#3057)
Use CUDA 10.1 for Linux build
(Windows change is already in)

Please note, cublas 10.2.1.243 is for CUDA SDK 10.1.243, not CUDA 10.2.x. CUDA 10.2.89 need cublas 10.2.2.89. They match on the last part of the digits.

libcublas10-10.1.0.105 won't work!!!

The cuda docker image by viswamy is already using 10.1, no need to change.
2020-02-21 11:55:32 -08:00
Ori Levari
be12fb3143
include winml x86 binaries in the drop-signed-nuget artifact (#3058) 2020-02-21 11:17:23 -08:00
William Tambellini
c033cfee92
Check python version (#356)
Check python version in build.py
2020-02-20 23:26:25 -08:00
Changming Sun
ef2bba316b
CUDA 10.1 for Windows(#3049) 2020-02-19 23:26:47 -08:00