* Add notebook for bert squad model exported by python 1.4
* update bert performance test tool:
(1) set OpenMP environment variable before importing onnxruntime.
(2) launch new process for each test.
* Add notebook
Reduce combinations in perf test
* update readme
* fix quote
* Allow test multiple batch_size
* Add latency percentile
* Add warm up run
Reset logger for notebook
* refine default settings to test for cpu/gpu
* Add script to dump machine info
* Add notebooks for PyTorch SQuAD model GPU and CPU inference
* Update machineinfo.py: add license header; format by yapf
* Do not reset log handler. Skip adding handler if existed.
* Add comments about GPU result diff.
Filter rows of batch set to keep only one setting.
* update according to review feedback
* Download script from master branch
* Add notebook for bert model exported by keras2onnx
* format columns in result table
* re-run and update notebook
* Fix WCOS/Win32 linking bugs
* Remove unused NODEFAULTLIB flags
* Avoid plain target_link_libraries signature
* Avoid plain target_link_libraries signature
* Fix library list escaping
* Use library list instead of string
* Remove duplicate link to windowsapp.lib
* Remove Win32 build workarounds
* Specify CMake policies before initializing language
* Expose Win32 header definitions during build
* Force set API family
* Enable Win32 APIs in featurizer
* Use MT dynamic CRT
* Expose Win32 specific functions
* Disable app container globally
* Disable default wide functions in featurizers
* Add featurizers to test include path
* Workaround https://gitlab.kitware.com/cmake/cmake/issues/19428
* Revert pipeline debugging hacks
* Skip /FI in CUDA sources
* Default to Win32 builds
* Enable WCOS when using WinML
* Use generator expression to apply CMAKE_MSVC_RUNTIME_LIBRARY to C++ only
* Add support for sessions to share a global threadpool.
* Fix build issues
* Add tests, fix build issues.
* Added some documentation
* Fix centos issue when threadpools become nullptr due to 1 core.
* Fix mac and x86 build issues
* Address some PR comments
* Disabled test for android, added few more tests and addressed more PR comments.
* const_cast
Moved path_lib.h/cc from onnxruntime/core/framework to onnxruntime/core/platform and from the onnxruntime_framework to the onnxruntime_common libraries.
* Add notebooks for GPU and CPU inference of PyTorch BERT SQuAD model
* update bert_optimization.py: Do not add duplicated logger handler
* Add machineinfo.py to show machine configuration for notebook.
* Update bert performance test tool:
(1) Set OpenMP environment variable before importing onnxruntime.
(2) Use sub-process for each test
(3) Allow test multiple batch_size
(4) Add latency percentile
(5) Add warmup
1. Fix onnxruntime server docker file build failure. Tested with the notebook in ONNX tutorial, it works well.
2. Delete the docker files for the other EPs, because currently they don't work and I don't have enough time to update them.
Speed up TfIdf.
Build Trie like structure to quickly exclude dead-ends.
Use ParallelFor() for each of the rows processing.
Make it non-template, batch it.
Check for short tail within the inner loop.
Fixes a bug in TopK cuda implementation when input size is between GridDim::maxThreadsPerBlock and GridDim::maxThreadsPerBlock * 2. In this case, the BitonicTopK will generate all-zero outputs.
* Simplify Normalizer as the spec only requires support for 2D input.
Tried using eigen (LpNorm<1>(), and norm()) on each row but that was much slower.
* Remove unused variable
* Use MlasComputeLogistic instead of manually computing values.
* Update test script to allow the tolerance to be specified when checking float output from logreg_iris.onnx.
We want to implement SoftmaxCrossentropy and NegativeLossLikelihoodLoss forward training ops for opset-12 but that requires ONNX submodule to point to the latest commit to have the latest and greatest ONNX spec!
- Reverse integrate changes from *.in.proto files in github ONNX repo.
- Regenerate csharp/test/Microsoft.ML.OnnxRuntime.Tests/OnnxMl.cs
- Disable ONNX tests that don't have op implementation for the latest opset.
1. misaligned address in atomic_add()
2. GatherNDGradKernel to use atomic_add
3. enable/add UTs for GatherNDGrad and reduction_ops using half
- __CUDA_ARCH__ won't take effect on .cc code, leverage HasCudaEnvironment() instead
4. verified convergence graph and perf test
- p100 is much slower than v100 on fp16
- fp16/128 need to reduce batch size from 66 to 64 to avoid OOM issue
5. verify convergence test on Dev3/v100
TBD - broken UTs related to MatmulIntegerOpTest (works on v100/windows, though)
This is a draft of graph cut and wait/record to demonstrate cut and Wait/Record design. You may find sub models and profiling json under onnxruntime/test if you run "onnxruntime_test_all --gtest_filter=GradientGraphBuilderTest.TrainingSession_WithPipeline"
1. Do not reuse the main thread.
2. Do not plus one when mlas calculate the number of tasks to schedule. (It was me put the plus one there)
This is the second try of #1839
It's known that this change has negative performance impact on some of the models.
* Avoid use of vectors for tracking reader/writer offsets as it adds too much overhead if there are a lot of readers or writers.
Tracy found improvements in resnet34-ssd1200 and BERT Squad with this approach.
