mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-29 20:14:01 +00:00
Second round of cherry-pick (#6083)
* Fix PR #5550 reverted in #5911 (performance improvment for operator Transpose) (#5916) * Improves implementation of transpose operator * Fix issue mentioned in #5911 * adding unit test for function DoTransposeImpl * Make operator TreeEnsemble 5x faster for batches of size 100.000 (#5965) * improves processing time by 10 * extend coverage unit test coverage * better implementation for the multi regression case * better comment, keep parallelization by trees when not enough trees * Initialize a structure in operator ReduceSum (#6005) * fix initialisation issue * Fuse MatMulIntegerToFloat only when scales are scalar (#6008) MatMulIntegerToFloat fusion fuses per-row and per-column MatMulInteger, which is not supported by the MatMulIntegerToFloat kernel now. Limit the fusion to per-matrix only before we supporting the per-channel fully. * Disable Python 3.9 for training Python packaging build. (#6012) Disable Python 3.9 for training Python packaging build. Python 3.9 is not supported by the PyTorch dependency. * Fix bugs for 1: Calibrator should check model inputs; 2: (#6017) quantize_inupts forgot to use parameter initializer_use_weight_qtyp. * Bump highlight.js from 10.2.1 to 10.4.1 in /nodejs Bumps [highlight.js](https://github.com/highlightjs/highlight.js) from 10.2.1 to 10.4.1. - [Release notes](https://github.com/highlightjs/highlight.js/releases) - [Changelog](https://github.com/highlightjs/highlight.js/blob/master/CHANGES.md) - [Commits](https://github.com/highlightjs/highlight.js/compare/10.2.1...10.4.1) Signed-off-by: dependabot[bot] <support@github.com> * work around of the build break in mac (#6069) * Fix the build break in macos release * revert android change * Bump up API version for 1.6 release (#6076) * Update version to 1.6.0 (#6041) * Update version to 1.6.0 * Add v 1.5.3 info * Updating WindowsAI and ONNX version Co-authored-by: Du Li <duli@OrtTrainingDev0.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net> * Rsevert "Fuse MatMulIntegerToFloat only when scales are scalar (#6008)" This reverts commit beb950eb66308eeaa8c60e4db9a006948e2ba7bb. Co-authored-by: Xavier Dupré <xadupre@users.noreply.github.com> Co-authored-by: Yufeng Li <liyufeng1987@gmail.com> Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com> Co-authored-by: Zhang Lei <zhang.huanning@hotmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pranav Sharma <prs@microsoft.com> Co-authored-by: Du Li <duli@OrtTrainingDev0.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
This commit is contained in:
Du Li
GitHub
parent
c38f762821
commit
718ca7f920
20 changed files with 672 additions and 217 deletions
|
|
@ -1 +1 @@
|
|||
1.5.2
|
||||
1.6.0
|
||||
|
|
|
|||
|
|
@ -26,11 +26,13 @@ For more details on ONNX Release versions, see [this page](https://github.com/on
|
|||
|
||||
| ONNX Runtime release version | ONNX release version | ONNX opset version | ONNX ML opset version | Supported ONNX IR version | [Windows ML Availability](https://docs.microsoft.com/en-us/windows/ai/windows-ml/release-notes/)|
|
||||
|------------------------------|--------------------|--------------------|----------------------|------------------|------------------|
|
||||
| 1.5.2 | **1.7** down to 1.2 | 12 | 2 | 6 | Windows AI 1.5+ |
|
||||
| 1.5.1 | **1.7** down to 1.2 | 12 | 2 | 6 | Windows AI 1.5+ |
|
||||
| 1.4.0 | **1.7** down to 1.2 | 12 | 2 | 6 | Windows AI 1.4+ |
|
||||
| 1.3.1 | **1.7** down to 1.2 | 12 | 2 | 6 | Windows AI 1.4+ |
|
||||
| 1.3.0 | **1.7** down to 1.2 | 12 | 2 | 6 | Windows AI 1.3+ |
|
||||
| 1.6.0 | **1.8** down to 1.2 | 13 | 2 | 7 | Windows AI 1.6+ |
|
||||
| 1.5.3 | **1.7** down to 1.2 | 12 | 2 | 7 | Windows AI 1.5+ |
|
||||
| 1.5.2 | **1.7** down to 1.2 | 12 | 2 | 7 | Windows AI 1.5+ |
|
||||
| 1.5.1 | **1.7** down to 1.2 | 12 | 2 | 7 | Windows AI 1.5+ |
|
||||
| 1.4.0 | **1.7** down to 1.2 | 12 | 2 | 7 | Windows AI 1.4+ |
|
||||
| 1.3.1 | **1.7** down to 1.2 | 12 | 2 | 7 | Windows AI 1.4+ |
|
||||
| 1.3.0 | **1.7** down to 1.2 | 12 | 2 | 7 | Windows AI 1.3+ |
|
||||
| 1.2.0<br>1.1.2<br>1.1.1<br>1.1.0 | **1.6** down to 1.2 | 11 | 2 | 6 | Windows AI 1.3+ |
|
||||
| 1.0.0 | **1.6** down to 1.2 | 11 | 2 | 6 | Windows AI 1.3+ |
|
||||
| 0.5.0 | **1.5** down to 1.2 | 10 | 1 | 5 | Windows AI 1.3+ |
|
||||
|
|
|
|||
|
|
@ -8,6 +8,16 @@ For more information on ONNX Runtime, please see `aka.ms/onnxruntime <https://ak
|
|||
Changes
|
||||
-------
|
||||
|
||||
1.6.0
|
||||
^^^^^
|
||||
|
||||
Release Notes : https://github.com/Microsoft/onnxruntime/releases/tag/v1.6.0
|
||||
|
||||
1.5.3
|
||||
^^^^^
|
||||
|
||||
Release Notes : https://github.com/Microsoft/onnxruntime/releases/tag/v1.5.3
|
||||
|
||||
1.5.2
|
||||
^^^^^
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
#include <string.h>
|
||||
|
||||
// This value is used in structures passed to ORT so that a newer version of ORT will still work with them
|
||||
#define ORT_API_VERSION 5
|
||||
#define ORT_API_VERSION 6
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
|
|
|
|||
10
nodejs/package-lock.json
generated
10
nodejs/package-lock.json
generated
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "onnxruntime",
|
||||
"version": "1.5.2",
|
||||
"version": "1.6.0",
|
||||
"lockfileVersion": 1,
|
||||
"requires": true,
|
||||
"dependencies": {
|
||||
|
|
@ -1779,9 +1779,9 @@
|
|||
"dev": true
|
||||
},
|
||||
"highlight.js": {
|
||||
"version": "10.2.1",
|
||||
"resolved": "https://registry.npmjs.org/highlight.js/-/highlight.js-10.2.1.tgz",
|
||||
"integrity": "sha1-CXhP4ulWEqu+/VEJSJRdT+b6lmg=",
|
||||
"version": "10.4.1",
|
||||
"resolved": "https://registry.npmjs.org/highlight.js/-/highlight.js-10.4.1.tgz",
|
||||
"integrity": "sha512-yR5lWvNz7c85OhVAEAeFhVCc/GV4C30Fjzc/rCP0aCWzc1UUOPUk55dK/qdwTZHBvMZo+eZ2jpk62ndX/xMFlg==",
|
||||
"dev": true
|
||||
},
|
||||
"hosted-git-info": {
|
||||
|
|
@ -4258,4 +4258,4 @@
|
|||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
{
|
||||
"name": "onnxruntime",
|
||||
"description": "Node.js binding of ONNXRuntime",
|
||||
"version": "1.5.2",
|
||||
"version": "1.6.0",
|
||||
"main": "./lib/index.js",
|
||||
"types": "./types/lib/index.d.ts",
|
||||
"scripts": {
|
||||
|
|
@ -69,4 +69,4 @@
|
|||
"dependencies": {
|
||||
"prebuild-install": "^5.3.5"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -7,7 +7,7 @@ ONNX Runtime is a performance-focused scoring engine for Open Neural Network Exc
|
|||
For more information on ONNX Runtime, please see `aka.ms/onnxruntime <https://aka.ms/onnxruntime/>`_
|
||||
or the `Github project <https://github.com/microsoft/onnxruntime/>`_.
|
||||
"""
|
||||
__version__ = "1.5.2"
|
||||
__version__ = "1.6.0"
|
||||
__author__ = "Microsoft"
|
||||
|
||||
from onnxruntime.capi._pybind_state import get_all_providers, get_available_providers, get_device, set_seed, \
|
||||
|
|
|
|||
|
|
@ -139,7 +139,7 @@ template <typename T>
|
|||
TreeEnsembleClassifier<T>::TreeEnsembleClassifier(const OpKernelInfo& info)
|
||||
: OpKernel(info),
|
||||
tree_ensemble_(
|
||||
100,
|
||||
80,
|
||||
50,
|
||||
info.GetAttrOrDefault<std::string>("aggregate_function", "SUM"),
|
||||
info.GetAttrsOrDefault<float>("base_values"),
|
||||
|
|
|
|||
|
|
@ -262,126 +262,180 @@ void TreeEnsembleCommon<ITYPE, OTYPE>::ComputeAgg(concurrency::ThreadPool* ttp,
|
|||
const ITYPE* x_data = X->template Data<ITYPE>();
|
||||
OTYPE* z_data = Z->template MutableData<OTYPE>();
|
||||
int64_t* label_data = label == nullptr ? nullptr : label->template MutableData<int64_t>();
|
||||
auto max_num_threads = concurrency::ThreadPool::DegreeOfParallelism(ttp);
|
||||
|
||||
if (n_targets_or_classes_ == 1) {
|
||||
if (N == 1) {
|
||||
ScoreValue<OTYPE> score = {0, 0};
|
||||
if (n_trees_ <= parallel_tree_) {
|
||||
if (n_trees_ <= parallel_tree_) { /* section A: 1 output, 1 row and not enough trees to parallelize */
|
||||
for (int64_t j = 0; j < n_trees_; ++j) {
|
||||
agg.ProcessTreeNodePrediction1(score, *ProcessTreeNodeLeave(roots_[j], x_data));
|
||||
}
|
||||
} else {
|
||||
std::vector<ScoreValue<OTYPE>> scores_t(n_trees_, {0, 0});
|
||||
} else { /* section B: 1 output, 1 row and enough trees to parallelize */
|
||||
std::vector<ScoreValue<OTYPE>> scores(n_trees_, {0, 0});
|
||||
concurrency::ThreadPool::TryBatchParallelFor(
|
||||
ttp,
|
||||
SafeInt<int32_t>(n_trees_),
|
||||
[this, &scores_t, &agg, x_data](ptrdiff_t j) {
|
||||
agg.ProcessTreeNodePrediction1(scores_t[j], *ProcessTreeNodeLeave(roots_[j], x_data));
|
||||
[this, &scores, &agg, x_data](ptrdiff_t j) {
|
||||
agg.ProcessTreeNodePrediction1(scores[j], *ProcessTreeNodeLeave(roots_[j], x_data));
|
||||
},
|
||||
0);
|
||||
|
||||
for (auto it = scores_t.cbegin(); it != scores_t.cend(); ++it) {
|
||||
for (auto it = scores.cbegin(); it != scores.cend(); ++it) {
|
||||
agg.MergePrediction1(score, *it);
|
||||
}
|
||||
}
|
||||
|
||||
agg.FinalizeScores1(z_data, score, label_data);
|
||||
} else {
|
||||
if (N <= parallel_N_) {
|
||||
ScoreValue<OTYPE> score;
|
||||
size_t j;
|
||||
} else if (N <= parallel_N_) { /* section C: 1 output, 2+ rows but not enough rows to parallelize */
|
||||
ScoreValue<OTYPE> score;
|
||||
size_t j;
|
||||
|
||||
for (int64_t i = 0; i < N; ++i) {
|
||||
score = {0, 0};
|
||||
for (j = 0; j < static_cast<size_t>(n_trees_); ++j) {
|
||||
agg.ProcessTreeNodePrediction1(score, *ProcessTreeNodeLeave(roots_[j], x_data + i * stride));
|
||||
}
|
||||
|
||||
agg.FinalizeScores1(z_data + i * n_targets_or_classes_, score,
|
||||
label_data == nullptr ? nullptr : (label_data + i));
|
||||
for (int64_t i = 0; i < N; ++i) {
|
||||
score = {0, 0};
|
||||
for (j = 0; j < static_cast<size_t>(n_trees_); ++j) {
|
||||
agg.ProcessTreeNodePrediction1(score, *ProcessTreeNodeLeave(roots_[j], x_data + i * stride));
|
||||
}
|
||||
} else {
|
||||
concurrency::ThreadPool::TryBatchParallelFor(
|
||||
ttp,
|
||||
SafeInt<int32_t>(N),
|
||||
[this, &agg, x_data, z_data, stride, label_data](ptrdiff_t i) {
|
||||
ScoreValue<OTYPE> score = {0, 0};
|
||||
for (size_t j = 0; j < static_cast<size_t>(n_trees_); ++j) {
|
||||
agg.ProcessTreeNodePrediction1(score, *ProcessTreeNodeLeave(roots_[j], x_data + i * stride));
|
||||
}
|
||||
|
||||
agg.FinalizeScores1(z_data + i * n_targets_or_classes_, score,
|
||||
label_data == nullptr ? nullptr : (label_data + i));
|
||||
},
|
||||
0);
|
||||
agg.FinalizeScores1(z_data + i, score,
|
||||
label_data == nullptr ? nullptr : (label_data + i));
|
||||
}
|
||||
} else if (n_trees_ > max_num_threads) { /* section D: 1 output, 2+ rows and enough trees to parallelize */
|
||||
auto num_threads = std::min<int32_t>(max_num_threads, SafeInt<int32_t>(n_trees_));
|
||||
std::vector<ScoreValue<OTYPE>> scores(num_threads * N);
|
||||
concurrency::ThreadPool::TrySimpleParallelFor(
|
||||
ttp,
|
||||
num_threads,
|
||||
[this, &agg, &scores, num_threads, x_data, N, stride](ptrdiff_t batch_num) {
|
||||
auto work = concurrency::ThreadPool::PartitionWork(batch_num, num_threads, this->n_trees_);
|
||||
for (int64_t i = 0; i < N; ++i) {
|
||||
scores[batch_num * N + i] = {0, 0};
|
||||
}
|
||||
for (auto j = work.start; j < work.end; ++j) {
|
||||
for (int64_t i = 0; i < N; ++i) {
|
||||
agg.ProcessTreeNodePrediction1(scores[batch_num * N + i], *ProcessTreeNodeLeave(roots_[j], x_data + i * stride));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
concurrency::ThreadPool::TrySimpleParallelFor(
|
||||
ttp,
|
||||
num_threads,
|
||||
[&agg, &scores, num_threads, label_data, z_data, N](ptrdiff_t batch_num) {
|
||||
auto work = concurrency::ThreadPool::PartitionWork(batch_num, num_threads, N);
|
||||
for (auto i = work.start; i < work.end; ++i) {
|
||||
for (int64_t j = 1; j < num_threads; ++j) {
|
||||
agg.MergePrediction1(scores[i], scores[j * N + i]);
|
||||
}
|
||||
agg.FinalizeScores1(z_data + i, scores[i],
|
||||
label_data == nullptr ? nullptr : (label_data + i));
|
||||
}
|
||||
});
|
||||
} else { /* section E: 1 output, 2+ rows, parallelization by rows */
|
||||
concurrency::ThreadPool::TryBatchParallelFor(
|
||||
ttp,
|
||||
SafeInt<int32_t>(N),
|
||||
[this, &agg, x_data, z_data, stride, label_data](ptrdiff_t i) {
|
||||
ScoreValue<OTYPE> score = {0, 0};
|
||||
for (size_t j = 0; j < static_cast<size_t>(n_trees_); ++j) {
|
||||
agg.ProcessTreeNodePrediction1(score, *ProcessTreeNodeLeave(roots_[j], x_data + i * stride));
|
||||
}
|
||||
|
||||
agg.FinalizeScores1(z_data + i, score,
|
||||
label_data == nullptr ? nullptr : (label_data + i));
|
||||
},
|
||||
0);
|
||||
}
|
||||
} else {
|
||||
if (N == 1) {
|
||||
std::vector<ScoreValue<OTYPE>> scores(n_targets_or_classes_, {0, 0});
|
||||
if (n_trees_ <= parallel_tree_) {
|
||||
if (N == 1) { /* section A2: 2+ outputs, 1 row, not enough trees to parallelize */
|
||||
if (n_trees_ <= parallel_tree_) { /* section A2 */
|
||||
std::vector<ScoreValue<OTYPE>> scores(n_targets_or_classes_, {0, 0});
|
||||
for (int64_t j = 0; j < n_trees_; ++j) {
|
||||
agg.ProcessTreeNodePrediction(scores, *ProcessTreeNodeLeave(roots_[j], x_data));
|
||||
}
|
||||
} else {
|
||||
// split the work into one block per thread so we can re-use the 'private_scores' vector as much as possible
|
||||
// TODO: Refine the number of threads used
|
||||
auto num_threads = std::min<int32_t>(concurrency::ThreadPool::DegreeOfParallelism(ttp), SafeInt<int32_t>(n_trees_));
|
||||
OrtMutex merge_mutex;
|
||||
agg.FinalizeScores(scores, z_data, -1, label_data);
|
||||
} else { /* section B2: 2+ outputs, 1 row, enough trees to parallelize */
|
||||
auto num_threads = std::min<int32_t>(max_num_threads, SafeInt<int32_t>(n_trees_));
|
||||
std::vector<std::vector<ScoreValue<OTYPE>>> scores(num_threads);
|
||||
concurrency::ThreadPool::TrySimpleParallelFor(
|
||||
ttp,
|
||||
num_threads,
|
||||
[this, &agg, &scores, &merge_mutex, num_threads, x_data](ptrdiff_t batch_num) {
|
||||
std::vector<ScoreValue<OTYPE>> private_scores(n_targets_or_classes_, {0, 0});
|
||||
[this, &agg, &scores, num_threads, x_data](ptrdiff_t batch_num) {
|
||||
scores[batch_num].resize(n_targets_or_classes_, {0, 0});
|
||||
auto work = concurrency::ThreadPool::PartitionWork(batch_num, num_threads, n_trees_);
|
||||
for (auto j = work.start; j < work.end; ++j) {
|
||||
agg.ProcessTreeNodePrediction(private_scores, *ProcessTreeNodeLeave(roots_[j], x_data));
|
||||
agg.ProcessTreeNodePrediction(scores[batch_num], *ProcessTreeNodeLeave(roots_[j], x_data));
|
||||
}
|
||||
|
||||
std::lock_guard<OrtMutex> lock(merge_mutex);
|
||||
agg.MergePrediction(scores, private_scores);
|
||||
});
|
||||
}
|
||||
|
||||
agg.FinalizeScores(scores, z_data, -1, label_data);
|
||||
} else {
|
||||
if (N <= parallel_N_) {
|
||||
std::vector<ScoreValue<OTYPE>> scores(n_targets_or_classes_);
|
||||
size_t j;
|
||||
|
||||
for (int64_t i = 0; i < N; ++i) {
|
||||
std::fill(scores.begin(), scores.end(), ScoreValue<OTYPE>({0, 0}));
|
||||
for (j = 0; j < roots_.size(); ++j) {
|
||||
agg.ProcessTreeNodePrediction(scores, *ProcessTreeNodeLeave(roots_[j], x_data + i * stride));
|
||||
}
|
||||
|
||||
agg.FinalizeScores(scores, z_data + i * n_targets_or_classes_, -1,
|
||||
label_data == nullptr ? nullptr : (label_data + i));
|
||||
for (size_t i = 1; i < scores.size(); ++i) {
|
||||
agg.MergePrediction(scores[0], scores[i]);
|
||||
}
|
||||
} else {
|
||||
// split the work into one block per thread so we can re-use the 'scores' vector as much as possible
|
||||
// TODO: Refine the number of threads used.
|
||||
auto num_threads = std::min<int32_t>(concurrency::ThreadPool::DegreeOfParallelism(ttp), SafeInt<int32_t>(N));
|
||||
concurrency::ThreadPool::TrySimpleParallelFor(
|
||||
ttp,
|
||||
num_threads,
|
||||
[this, &agg, num_threads, x_data, z_data, label_data, N, stride](ptrdiff_t batch_num) {
|
||||
size_t j;
|
||||
std::vector<ScoreValue<OTYPE>> scores(n_targets_or_classes_);
|
||||
auto work = concurrency::ThreadPool::PartitionWork(batch_num, num_threads, N);
|
||||
|
||||
for (auto i = work.start; i < work.end; ++i) {
|
||||
std::fill(scores.begin(), scores.end(), ScoreValue<OTYPE>({0, 0}));
|
||||
for (j = 0; j < roots_.size(); ++j) {
|
||||
agg.ProcessTreeNodePrediction(scores, *ProcessTreeNodeLeave(roots_[j], x_data + i * stride));
|
||||
}
|
||||
|
||||
agg.FinalizeScores(scores,
|
||||
z_data + i * n_targets_or_classes_, -1,
|
||||
label_data == nullptr ? nullptr : (label_data + i));
|
||||
}
|
||||
});
|
||||
agg.FinalizeScores(scores[0], z_data, -1, label_data);
|
||||
}
|
||||
} else if (N <= parallel_N_) { /* section C2: 2+ outputs, 2+ rows, not enough rows to parallelize */
|
||||
std::vector<ScoreValue<OTYPE>> scores(n_targets_or_classes_);
|
||||
size_t j;
|
||||
|
||||
for (int64_t i = 0; i < N; ++i) {
|
||||
std::fill(scores.begin(), scores.end(), ScoreValue<OTYPE>({0, 0}));
|
||||
for (j = 0; j < roots_.size(); ++j) {
|
||||
agg.ProcessTreeNodePrediction(scores, *ProcessTreeNodeLeave(roots_[j], x_data + i * stride));
|
||||
}
|
||||
|
||||
agg.FinalizeScores(scores, z_data + i * n_targets_or_classes_, -1,
|
||||
label_data == nullptr ? nullptr : (label_data + i));
|
||||
}
|
||||
} else if (n_trees_ >= max_num_threads) { /* section: D2: 2+ outputs, 2+ rows, enough trees to parallelize*/
|
||||
auto num_threads = std::min<int32_t>(max_num_threads, SafeInt<int32_t>(n_trees_));
|
||||
std::vector<std::vector<ScoreValue<OTYPE>>> scores(num_threads * N);
|
||||
concurrency::ThreadPool::TrySimpleParallelFor(
|
||||
ttp,
|
||||
num_threads,
|
||||
[this, &agg, &scores, num_threads, x_data, N, stride](ptrdiff_t batch_num) {
|
||||
auto work = concurrency::ThreadPool::PartitionWork(batch_num, num_threads, this->n_trees_);
|
||||
for (int64_t i = 0; i < N; ++i) {
|
||||
scores[batch_num * N + i].resize(n_targets_or_classes_, {0, 0});
|
||||
}
|
||||
for (auto j = work.start; j < work.end; ++j) {
|
||||
for (int64_t i = 0; i < N; ++i) {
|
||||
agg.ProcessTreeNodePrediction(scores[batch_num * N + i], *ProcessTreeNodeLeave(roots_[j], x_data + i * stride));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
concurrency::ThreadPool::TrySimpleParallelFor(
|
||||
ttp,
|
||||
num_threads,
|
||||
[this, &agg, &scores, num_threads, label_data, z_data, N](ptrdiff_t batch_num) {
|
||||
auto work = concurrency::ThreadPool::PartitionWork(batch_num, num_threads, N);
|
||||
for (auto i = work.start; i < work.end; ++i) {
|
||||
for (int64_t j = 1; j < num_threads; ++j) {
|
||||
agg.MergePrediction(scores[i], scores[j * N + i]);
|
||||
}
|
||||
agg.FinalizeScores(scores[i], z_data + i * this->n_targets_or_classes_, -1,
|
||||
label_data == nullptr ? nullptr : (label_data + i));
|
||||
}
|
||||
});
|
||||
} else { /* section E2: 2+ outputs, 2+ rows, parallelization by rows */
|
||||
auto num_threads = std::min<int32_t>(max_num_threads, SafeInt<int32_t>(N));
|
||||
concurrency::ThreadPool::TrySimpleParallelFor(
|
||||
ttp,
|
||||
num_threads,
|
||||
[this, &agg, num_threads, x_data, z_data, label_data, N, stride](ptrdiff_t batch_num) {
|
||||
size_t j;
|
||||
std::vector<ScoreValue<OTYPE>> scores(n_targets_or_classes_);
|
||||
auto work = concurrency::ThreadPool::PartitionWork(batch_num, num_threads, N);
|
||||
|
||||
for (auto i = work.start; i < work.end; ++i) {
|
||||
std::fill(scores.begin(), scores.end(), ScoreValue<OTYPE>({0, 0}));
|
||||
for (j = 0; j < roots_.size(); ++j) {
|
||||
agg.ProcessTreeNodePrediction(scores, *ProcessTreeNodeLeave(roots_[j], x_data + i * stride));
|
||||
}
|
||||
|
||||
agg.FinalizeScores(scores,
|
||||
z_data + i * n_targets_or_classes_, -1,
|
||||
label_data == nullptr ? nullptr : (label_data + i));
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
} // namespace detail
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ template <typename T>
|
|||
TreeEnsembleRegressor<T>::TreeEnsembleRegressor(const OpKernelInfo& info)
|
||||
: OpKernel(info),
|
||||
tree_ensemble_(
|
||||
100,
|
||||
80,
|
||||
50,
|
||||
info.GetAttrOrDefault<std::string>("aggregate_function", "SUM"),
|
||||
info.GetAttrsOrDefault<float>("base_values"),
|
||||
|
|
|
|||
|
|
@ -74,10 +74,10 @@ namespace onnxruntime {
|
|||
x<int64_t>);
|
||||
|
||||
#define REGISTER_UNARY_ELEMENTWISE_VERSIONED_KERNEL_INT8_ONLY(x, startVer, endVer) \
|
||||
ONNX_CPU_OPERATOR_VERSIONED_TYPED_KERNEL( \
|
||||
x, \
|
||||
startVer, \
|
||||
endVer, \
|
||||
ONNX_CPU_OPERATOR_VERSIONED_TYPED_KERNEL( \
|
||||
x, \
|
||||
startVer, \
|
||||
endVer, \
|
||||
int8_t, \
|
||||
KernelDefBuilder().TypeConstraint("T", DataTypeImpl::GetTensorType<int8_t>()), \
|
||||
x<int8_t>);
|
||||
|
|
@ -138,7 +138,6 @@ REGISTER_UNARY_ELEMENTWISE_KERNEL_INT64_ONLY(ReduceMax, 13);
|
|||
REGISTER_UNARY_ELEMENTWISE_KERNEL_INT8_ONLY(ReduceMax, 13);
|
||||
REGISTER_UNARY_ELEMENTWISE_KERNEL_UINT8_ONLY(ReduceMax, 13);
|
||||
|
||||
|
||||
REGISTER_UNARY_ELEMENTWISE_VERSIONED_KERNEL(ReduceMean, 1, 10);
|
||||
REGISTER_UNARY_ELEMENTWISE_VERSIONED_KERNEL(ReduceMean, 11, 12);
|
||||
REGISTER_UNARY_ELEMENTWISE_KERNEL(ReduceMean, 13);
|
||||
|
|
@ -361,6 +360,9 @@ void NoTransposeReduce(Tensor* output, const TensorShape& new_input_shape, const
|
|||
if (last_results.last_loop_red_size == 0 || last_results.last_loop_size == 0)
|
||||
return;
|
||||
}
|
||||
ORT_ENFORCE(last_results.last_loop_red_size > 0);
|
||||
ORT_ENFORCE(last_results.last_loop_size > 0);
|
||||
ORT_ENFORCE(last_results.projected_index.size() > 0);
|
||||
int64_t denominator = last_results.last_loop_red_size * last_results.projected_index.size();
|
||||
|
||||
if (AGG::two_loops()) {
|
||||
|
|
|
|||
|
|
@ -24,6 +24,14 @@ class ResultsNoTransposePrepareForReduce {
|
|||
std::vector<int64_t> unprojected_index;
|
||||
int64_t last_loop_size;
|
||||
int64_t last_loop_inc;
|
||||
|
||||
ResultsNoTransposePrepareForReduce() : input_shape(), reduced_axes(), projected_index(), unprojected_index() {
|
||||
last_loop_red_size = 0;
|
||||
last_loop_red_inc = 0;
|
||||
last_loop_size = 0;
|
||||
last_loop_inc = 0;
|
||||
}
|
||||
|
||||
bool equal(const std::vector<int64_t>& local_input_shape, const std::vector<int64_t>& local_reduced_axes) {
|
||||
if (input_shape.size() != local_input_shape.size())
|
||||
return false;
|
||||
|
|
|
|||
|
|
@ -15,23 +15,85 @@ namespace onnxruntime {
|
|||
etc.
|
||||
*/
|
||||
|
||||
// ComputeOffset: compute offset into a tensor. This is essentially the dot-product of
|
||||
// index and stride, restricted to the specified number of axes.
|
||||
static inline size_t ComputeOffset(const std::vector<int64_t>& index, const std::vector<size_t>& stride, int64_t num_axes) {
|
||||
size_t offset = 0;
|
||||
for (int64_t j = 0; j < num_axes; ++j) {
|
||||
offset += index[j] * stride[j];
|
||||
struct MultiIndex {
|
||||
size_t n_axes;
|
||||
std::vector<size_t> index;
|
||||
std::vector<size_t> upper_bound;
|
||||
std::vector<int64_t> stride;
|
||||
|
||||
/* There is one MultiIndex instance per axis in the tensor.
|
||||
* The array keeps track of the position of a pointer walking through the data.
|
||||
* Any function using it creates an array of MultiIndex
|
||||
* then calls function IncrementIndexAndComputeOffsetSetup
|
||||
* to initialize the array. This constructor does not initialize
|
||||
* anything because it would be overwritten by function
|
||||
* IncrementIndexAndComputeOffsetSetup. This one calls method Init.
|
||||
* Function IncrementIndexAndComputeOffset is called to increment
|
||||
* the array of MultiIndex to move to the next data in the tensor.
|
||||
*/
|
||||
MultiIndex() : index(), upper_bound(), stride() { n_axes = 0; }
|
||||
|
||||
void Init(size_t num_axes) {
|
||||
index.resize(num_axes);
|
||||
upper_bound.resize(num_axes);
|
||||
stride.resize(num_axes);
|
||||
n_axes = num_axes;
|
||||
}
|
||||
return offset;
|
||||
|
||||
void InitAxis(size_t n_axis, size_t i, size_t n, int64_t s) {
|
||||
index[n_axis] = i;
|
||||
upper_bound[n_axis] = n;
|
||||
stride[n_axis] = s;
|
||||
}
|
||||
};
|
||||
|
||||
/* This function initializes an array of MultiIndex of size num_axes (one instance per axis).
|
||||
* target_dims is the shape of the transposed tensor, stride is linked to the tensor to
|
||||
* be transposed, if source_dims is the shape, stride[i] = source_dims[i+1] * source_dims[i+2] * ... * 1.
|
||||
* element_size is the size of the tensor element (sizeof(float), sizeof(double)).
|
||||
*/
|
||||
static void IncrementIndexAndComputeOffsetSetup(MultiIndex& mindex, size_t num_axes, const std::vector<int64_t>& target_dims,
|
||||
const std::vector<size_t>& stride, size_t element_size) {
|
||||
mindex.Init(num_axes);
|
||||
size_t naxes = 0;
|
||||
for (size_t i = 0; i < num_axes; ++i) {
|
||||
if (target_dims[i] == 1)
|
||||
continue;
|
||||
mindex.InitAxis(naxes, 0, static_cast<size_t>(target_dims[i]), stride[i] * element_size);
|
||||
++naxes;
|
||||
}
|
||||
ORT_ENFORCE(naxes > 0, "Method IncrementIndexAndComputeOffset assumes this value is strictly positive.");
|
||||
mindex.n_axes = naxes;
|
||||
}
|
||||
|
||||
// IncrementIndex: Increment an index into a tensor (in lexicographic ordering), wrapping
|
||||
// around the specified upper_bound.
|
||||
static inline void IncrementIndex(std::vector<int64_t>& index, const std::vector<int64_t>& upper_bound, int64_t num_axes) {
|
||||
for (int64_t k = num_axes - 1; k >= 0; --k) {
|
||||
index[k]++;
|
||||
if (index[k] < upper_bound[k]) break;
|
||||
index[k] = 0;
|
||||
/* This function increments an array of MultiIndex initialized by function IncrementIndexAndComputeOffsetSetup.
|
||||
* It increments the last dimension, checks if it stays within boundary. If it stays in, it returns,
|
||||
* otherwise, it reset the dimension to zero and increments the previous one.
|
||||
* While doing that, every modification brought to the array of indices is applied on the
|
||||
* pointer local_source. It avoids computing again local_source from the source tensor.
|
||||
* At every time, the following condition is verified:
|
||||
* local_source = source + (sum_i mindex[i].index * mindex[i].stride
|
||||
*/
|
||||
template <typename T>
|
||||
static inline void IncrementIndexAndComputeOffset(MultiIndex& mindex, const T*& local_source) {
|
||||
// Increment the last dimension.
|
||||
int pos = static_cast<int>(mindex.n_axes) - 1;
|
||||
local_source += mindex.stride[pos];
|
||||
// Checks it stays within boundaries.
|
||||
if (++mindex.index[pos] < mindex.upper_bound[pos])
|
||||
return;
|
||||
// If not, loops on other indices.
|
||||
// The first test is outside the loop to be faster.
|
||||
// As it is the most common case.
|
||||
local_source -= mindex.stride[pos] * mindex.index[pos];
|
||||
mindex.index[pos] = 0;
|
||||
--pos;
|
||||
for (; pos >= 0; --pos) {
|
||||
local_source += mindex.stride[pos];
|
||||
if (++mindex.index[pos] < mindex.upper_bound[pos])
|
||||
break;
|
||||
local_source -= mindex.stride[pos] * mindex.index[pos];
|
||||
mindex.index[pos] = 0;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -55,17 +117,14 @@ static void DoTransposeImpl(int64_t num_axes, const std::vector<int64_t>& target
|
|||
size_t num_blocks, size_t num_elts_in_block, const std::vector<size_t>& stride,
|
||||
const uint8_t* source, uint8_t* target, size_t element_size) {
|
||||
size_t blocksize = num_elts_in_block * element_size;
|
||||
// index used to iterate over target iteration-space
|
||||
std::vector<int64_t> target_index(num_axes, 0);
|
||||
MultiIndex mindex;
|
||||
IncrementIndexAndComputeOffsetSetup(mindex, num_axes, target_dims, stride, element_size);
|
||||
|
||||
const uint8_t* local_source = source;
|
||||
for (size_t i = 0; i < num_blocks; ++i) {
|
||||
// convert target_index into an offset in source data
|
||||
size_t source_offset = ComputeOffset(target_index, stride, num_axes);
|
||||
|
||||
// copy
|
||||
memcpy(target, source + source_offset * element_size, blocksize);
|
||||
|
||||
// increment target_index:
|
||||
IncrementIndex(target_index, target_dims, num_axes);
|
||||
ORT_ENFORCE((local_source >= source) && (local_source < source + num_blocks * blocksize));
|
||||
memcpy(target, local_source, blocksize);
|
||||
IncrementIndexAndComputeOffset(mindex, local_source);
|
||||
target += blocksize;
|
||||
}
|
||||
}
|
||||
|
|
@ -73,17 +132,15 @@ static void DoTransposeImpl(int64_t num_axes, const std::vector<int64_t>& target
|
|||
static void DoTransposeImpl(int64_t num_axes, const std::vector<int64_t>& target_dims,
|
||||
size_t num_blocks, size_t num_elts_in_block, const std::vector<size_t>& stride,
|
||||
const std::string* source, std::string* target) {
|
||||
// index used to iterate over target iteration-space
|
||||
std::vector<int64_t> target_index(num_axes, 0);
|
||||
ORT_ENFORCE(num_axes > 0, "Transpose not implemented for empty tensors.");
|
||||
MultiIndex mindex;
|
||||
IncrementIndexAndComputeOffsetSetup(mindex, num_axes, target_dims, stride, 1);
|
||||
|
||||
const std::string* local_source = source;
|
||||
for (size_t i = 0; i < num_blocks; ++i) {
|
||||
// convert target_index into an offset in source data
|
||||
size_t source_offset = ComputeOffset(target_index, stride, num_axes);
|
||||
|
||||
// copy
|
||||
DoTransposeSingleBlock(num_elts_in_block, source + source_offset, target);
|
||||
|
||||
// increment target_index:
|
||||
IncrementIndex(target_index, target_dims, num_axes);
|
||||
ORT_ENFORCE((local_source >= source) && (local_source < source + num_blocks * num_elts_in_block));
|
||||
DoTransposeSingleBlock(num_elts_in_block, local_source, target);
|
||||
IncrementIndexAndComputeOffset(mindex, local_source);
|
||||
target += num_elts_in_block;
|
||||
}
|
||||
}
|
||||
|
|
@ -93,67 +150,40 @@ inline void CopyPrim(uint8_t* target, const uint8_t* source) {
|
|||
*reinterpret_cast<T*>(target) = *reinterpret_cast<const T*>(source);
|
||||
}
|
||||
|
||||
// The function does not check num_axes > 0 but this is expected.
|
||||
template <class T>
|
||||
static void TypedDoTransposeEltWise(int64_t num_axes, const std::vector<int64_t>& target_dims, size_t num_blocks,
|
||||
const std::vector<size_t>& stride, const uint8_t* source, uint8_t* target) {
|
||||
MultiIndex mindex;
|
||||
IncrementIndexAndComputeOffsetSetup(mindex, num_axes, target_dims, stride, sizeof(T));
|
||||
|
||||
const uint8_t* local_source = source;
|
||||
uint8_t* target_end = target + sizeof(T) * num_blocks;
|
||||
for (; target != target_end; target += sizeof(T)) {
|
||||
ORT_ENFORCE((local_source >= source) && (local_source < source + sizeof(T) * num_blocks));
|
||||
CopyPrim<T>(target, local_source);
|
||||
IncrementIndexAndComputeOffset(mindex, local_source);
|
||||
}
|
||||
}
|
||||
|
||||
// DoTransposeEltWise: specialization of DoTranspose for the num_elts_in_block=1 case.
|
||||
// copies source tensor to target, transposing elements.
|
||||
// The stride vector indicates the transposition.
|
||||
static void DoTransposeEltWise(int64_t num_axes, const std::vector<int64_t>& target_dims, size_t num_blocks,
|
||||
const std::vector<size_t>& stride, const uint8_t* source, uint8_t* target,
|
||||
size_t element_size) {
|
||||
// index used to iterate over target iteration-space
|
||||
std::vector<int64_t> target_index(num_axes, 0);
|
||||
|
||||
void DoTransposeEltWise(int64_t num_axes, const std::vector<int64_t>& target_dims, size_t num_blocks,
|
||||
const std::vector<size_t>& stride, const uint8_t* source, uint8_t* target,
|
||||
size_t element_size) {
|
||||
switch (element_size) {
|
||||
case sizeof(uint64_t):
|
||||
for (size_t i = 0; i < num_blocks; ++i) {
|
||||
// convert target_index into an offset in source data
|
||||
size_t source_offset = ComputeOffset(target_index, stride, num_axes);
|
||||
|
||||
// copy
|
||||
CopyPrim<uint64_t>(target, source + (source_offset * element_size));
|
||||
|
||||
// increment target_index:
|
||||
IncrementIndex(target_index, target_dims, num_axes);
|
||||
target += element_size;
|
||||
}
|
||||
TypedDoTransposeEltWise<uint64_t>(num_axes, target_dims, num_blocks, stride, source, target);
|
||||
break;
|
||||
case sizeof(uint32_t):
|
||||
for (size_t i = 0; i < num_blocks; ++i) {
|
||||
// convert target_index into an offset in source data
|
||||
size_t source_offset = ComputeOffset(target_index, stride, num_axes);
|
||||
|
||||
// copy
|
||||
CopyPrim<uint32_t>(target, source + (source_offset * element_size));
|
||||
|
||||
// increment target_index:
|
||||
IncrementIndex(target_index, target_dims, num_axes);
|
||||
target += element_size;
|
||||
}
|
||||
TypedDoTransposeEltWise<uint32_t>(num_axes, target_dims, num_blocks, stride, source, target);
|
||||
break;
|
||||
case sizeof(uint16_t):
|
||||
for (size_t i = 0; i < num_blocks; ++i) {
|
||||
// convert target_index into an offset in source data
|
||||
size_t source_offset = ComputeOffset(target_index, stride, num_axes);
|
||||
|
||||
// copy
|
||||
CopyPrim<uint16_t>(target, source + (source_offset * element_size));
|
||||
|
||||
// increment target_index:
|
||||
IncrementIndex(target_index, target_dims, num_axes);
|
||||
target += element_size;
|
||||
}
|
||||
TypedDoTransposeEltWise<uint16_t>(num_axes, target_dims, num_blocks, stride, source, target);
|
||||
break;
|
||||
case sizeof(uint8_t):
|
||||
for (size_t i = 0; i < num_blocks; ++i) {
|
||||
// convert target_index into an offset in source data
|
||||
size_t source_offset = ComputeOffset(target_index, stride, num_axes);
|
||||
|
||||
// copy
|
||||
*target = *(source + (source_offset * element_size));
|
||||
|
||||
// increment target_index:
|
||||
IncrementIndex(target_index, target_dims, num_axes);
|
||||
target += element_size;
|
||||
}
|
||||
TypedDoTransposeEltWise<uint8_t>(num_axes, target_dims, num_blocks, stride, source, target);
|
||||
break;
|
||||
default:
|
||||
assert(false);
|
||||
|
|
@ -162,17 +192,16 @@ static void DoTransposeEltWise(int64_t num_axes, const std::vector<int64_t>& tar
|
|||
|
||||
static void DoTransposeEltWise(int64_t num_axes, const std::vector<int64_t>& target_dims, size_t num_blocks,
|
||||
const std::vector<size_t>& stride, const std::string* source, std::string* target) {
|
||||
ORT_ENFORCE(num_axes > 0, "Transpose not implemented for empty tensors.");
|
||||
MultiIndex mindex;
|
||||
IncrementIndexAndComputeOffsetSetup(mindex, num_axes, target_dims, stride, 1);
|
||||
|
||||
// index used to iterate over target iteration-space
|
||||
std::vector<int64_t> target_index(num_axes, 0);
|
||||
const std::string* local_source = source;
|
||||
for (size_t i = 0; i < num_blocks; ++i) {
|
||||
// convert target_index into an offset in source data
|
||||
size_t source_offset = ComputeOffset(target_index, stride, num_axes);
|
||||
|
||||
// copy
|
||||
*target = *(source + source_offset);
|
||||
|
||||
// increment target_index:
|
||||
IncrementIndex(target_index, target_dims, num_axes);
|
||||
ORT_ENFORCE((local_source >= source) && (local_source < source + num_blocks));
|
||||
*target = *local_source;
|
||||
IncrementIndexAndComputeOffset(mindex, local_source);
|
||||
target++;
|
||||
}
|
||||
}
|
||||
|
|
@ -274,13 +303,15 @@ template <typename T>
|
|||
static void SimpleTransposeSingleAxisOutwards(const T* input_data, T* output_data,
|
||||
int64_t num_loops, int64_t num_writers,
|
||||
int64_t writes_per_loop, int64_t writes_per_writer_per_loop) {
|
||||
const T* end;
|
||||
for (int64_t l = 0; l < num_loops; ++l) {
|
||||
T* output_for_first_writer = output_data;
|
||||
|
||||
for (auto wwpl = 0; wwpl < writes_per_writer_per_loop; ++wwpl) {
|
||||
T* output_for_current_writer = output_for_first_writer;
|
||||
|
||||
for (int64_t w = 0; w < num_writers; ++w) {
|
||||
end = input_data + num_writers;
|
||||
for (; input_data != end;) {
|
||||
*output_for_current_writer = *input_data++;
|
||||
|
||||
// skip to output position for next writer
|
||||
|
|
@ -379,13 +410,15 @@ template <typename T>
|
|||
static void SimpleTransposeSingleAxisInwards(const T* input_data, T* output_data,
|
||||
int64_t num_loops, int64_t num_readers,
|
||||
int64_t reads_per_loop, int64_t reads_per_reader_per_loop) {
|
||||
T* end;
|
||||
for (int64_t l = 0; l < num_loops; ++l) {
|
||||
const T* input_for_first_reader = input_data;
|
||||
|
||||
for (auto rrpl = 0; rrpl < reads_per_reader_per_loop; ++rrpl) {
|
||||
const T* input_for_current_reader = input_for_first_reader;
|
||||
|
||||
for (int64_t r = 0; r < num_readers; ++r) {
|
||||
end = output_data + num_readers;
|
||||
for (; output_data != end;) {
|
||||
*output_data++ = *input_for_current_reader;
|
||||
// skip to input position for next reader
|
||||
input_for_current_reader += reads_per_reader_per_loop;
|
||||
|
|
@ -560,6 +593,20 @@ static bool IsMovingSingleAxis(const std::vector<size_t>& permutations, size_t&
|
|||
return single_axis_moved;
|
||||
}
|
||||
|
||||
bool IsTransposeReshape(const std::vector<size_t>& perm, const std::vector<int64_t>& input_dims) {
|
||||
// As long as the dims with values > 1 stay in the same order, it's a reshape.
|
||||
// Example: Shape=(1,1,1024,4096) -> perm=(2,0,3,1).
|
||||
size_t last_permuted_axis = 0;
|
||||
for (size_t i = 0; i < perm.size(); ++i) {
|
||||
if (input_dims[perm[i]] == 1)
|
||||
continue;
|
||||
if (perm[i] < last_permuted_axis)
|
||||
return false;
|
||||
last_permuted_axis = perm[i];
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
//`input_shape_override` overrides the shape of `input` for compute purposes.
|
||||
Status TransposeBase::DoTranspose(const std::vector<size_t>& permutations, const Tensor& input, Tensor& output,
|
||||
const TensorShape* input_shape_override) {
|
||||
|
|
@ -572,6 +619,14 @@ Status TransposeBase::DoTranspose(const std::vector<size_t>& permutations, const
|
|||
status = ORT_MAKE_STATUS(ONNXRUNTIME, FAIL, "Mismatched data types between input and output Tensors. ",
|
||||
input_type, " != ", output_type);
|
||||
} else {
|
||||
TensorShape shape = input_shape_override ? *input_shape_override : input.Shape();
|
||||
if (IsTransposeReshape(permutations, shape.GetDims())) {
|
||||
// As long as the dims with values > 1 stay in the same order, it's a reshape.
|
||||
// Example: Shape=(1,1,1024,4096) -> perm=(2,0,3,1).
|
||||
CopyCpuTensor(&input, &output);
|
||||
return Status::OK();
|
||||
}
|
||||
|
||||
size_t from = 0, to = 0;
|
||||
bool moving_single_axis = IsMovingSingleAxis(permutations, from, to);
|
||||
|
||||
|
|
@ -607,6 +662,13 @@ Status Transpose::Compute(OpKernelContext* ctx) const {
|
|||
if (output_shape.Size() == 0)
|
||||
return Status::OK();
|
||||
|
||||
if (IsTransposeReshape(*p_perm, input_dims)) {
|
||||
// As long as the dims with values > 1 stay in the same order, it's a reshape.
|
||||
// Example: Shape=(1,1,1024,4096) -> perm=(2,0,3,1).
|
||||
CopyCpuTensor(&X, &Y);
|
||||
return Status::OK();
|
||||
}
|
||||
|
||||
size_t from = 0, to = 0;
|
||||
bool moving_single_axis = IsMovingSingleAxis(*p_perm, from, to);
|
||||
|
||||
|
|
|
|||
|
|
@ -10,6 +10,16 @@
|
|||
|
||||
namespace onnxruntime {
|
||||
|
||||
/** Tells if the transpose is equivalent to a reshape:
|
||||
empty dimensions can change place, not empty dimensions must be in
|
||||
the same order in the permuted tenosr.
|
||||
*/
|
||||
bool IsTransposeReshape(const std::vector<size_t>& perm, const std::vector<int64_t>& input_dims);
|
||||
|
||||
void DoTransposeEltWise(int64_t num_axes, const std::vector<int64_t>& target_dims, size_t num_blocks,
|
||||
const std::vector<size_t>& stride, const uint8_t* source, uint8_t* target,
|
||||
size_t element_size);
|
||||
|
||||
class TransposeBase {
|
||||
public:
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -2063,7 +2063,6 @@ static constexpr OrtApi ort_api_1_to_6 = {
|
|||
&OrtApis::SetGlobalSpinControl,
|
||||
// End of Version 5 - DO NOT MODIFY ABOVE (see above text for more information)
|
||||
|
||||
// Version 6 - In development, feel free to add/remove/rearrange here
|
||||
&OrtApis::AddInitializer,
|
||||
&OrtApis::CreateEnvWithCustomLoggerAndGlobalThreadPools,
|
||||
&OrtApis::SessionOptionsAppendExecutionProvider_CUDA,
|
||||
|
|
@ -2071,6 +2070,9 @@ static constexpr OrtApi ort_api_1_to_6 = {
|
|||
&OrtApis::SetGlobalDenormalAsZero,
|
||||
&OrtApis::CreateArenaCfg,
|
||||
&OrtApis::ReleaseArenaCfg,
|
||||
// End of Version 6 - DO NOT MODIFY ABOVE (see above text for more information)
|
||||
|
||||
// Version 7 - In development, feel free to add/remove/rearrange here
|
||||
};
|
||||
|
||||
// Assert to do a limited check to ensure Version 1 of OrtApi never changes (will detect an addition or deletion but not if they cancel out each other)
|
||||
|
|
|
|||
|
|
@ -58,6 +58,7 @@ class ONNXCalibrater:
|
|||
model = onnx.shape_inference.infer_shapes(model)
|
||||
value_infos = {vi.name: vi for vi in model.graph.value_info}
|
||||
value_infos.update({ot.name: ot for ot in model.graph.output})
|
||||
value_infos.update({it.name: it for it in model.graph.input})
|
||||
|
||||
added_nodes = []
|
||||
added_outputs = []
|
||||
|
|
@ -264,4 +265,4 @@ def calibrate(model_path,
|
|||
quantization_params_dict = calibrater.calculate_quantization_params(dict_for_quantization)
|
||||
|
||||
print("Calibrated,quantized parameters calculated and returned.")
|
||||
return quantization_params_dict
|
||||
return quantization_params_dict
|
||||
|
|
|
|||
|
|
@ -806,7 +806,7 @@ class ONNXQuantizer:
|
|||
# Quantize the input
|
||||
initializer = find_by_name(node_input, self.model.initializer())
|
||||
if initializer is not None:
|
||||
weight = self._get_quantized_weight(initializer, self.weight_qType)
|
||||
weight = self._get_quantized_weight(initializer, self.weight_qType if initializer_use_weight_qType else self.input_qType)
|
||||
|
||||
# Update graph
|
||||
self._update_weight(weight)
|
||||
|
|
|
|||
|
|
@ -8,7 +8,31 @@ namespace onnxruntime {
|
|||
namespace test {
|
||||
|
||||
template <typename T>
|
||||
void GenTreeAndRunTest(const std::vector<T>& X, const std::vector<float>& base_values, const std::vector<float>& results, const std::string& aggFunction, bool one_obs = false) {
|
||||
void _multiply_update_array(std::vector<T>& data, int n, T inc = 0) {
|
||||
std::vector<T> copy = data;
|
||||
data.resize(copy.size() * n);
|
||||
T cst = 0;
|
||||
for (int i = 0; i < n; ++i) {
|
||||
for (size_t j = 0; j < copy.size(); ++j) {
|
||||
data[j + i * copy.size()] = copy[j] + cst;
|
||||
}
|
||||
cst += inc;
|
||||
}
|
||||
}
|
||||
|
||||
void _multiply_update_array_string(std::vector<std::string>& data, int n) {
|
||||
std::vector<std::string> copy = data;
|
||||
data.resize(copy.size() * n);
|
||||
for (int i = 0; i < n; ++i) {
|
||||
for (size_t j = 0; j < copy.size(); ++j) {
|
||||
data[j + i * copy.size()] = copy[j];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
void GenTreeAndRunTest(const std::vector<T>& X, const std::vector<float>& base_values, const std::vector<float>& results, const std::string& aggFunction,
|
||||
bool one_obs = false, int64_t n_obs = 8, int n_trees = 1) {
|
||||
OpTester test("TreeEnsembleRegressor", 1, onnxruntime::kMLDomain);
|
||||
|
||||
//tree
|
||||
|
|
@ -26,6 +50,21 @@ void GenTreeAndRunTest(const std::vector<T>& X, const std::vector<float>& base_v
|
|||
std::vector<float> target_weights = {1.5f, 27.5f, 2.25f, 20.75f, 2.f, 23.f, 3.f, 14.f, 0.f, 41.f, 1.83333333f, 24.5f, 0.f, 41.f, 2.75f, 16.25f, 2.f, 23.f, 3.f, 14.f, 2.66666667f, 17.f, 2.f, 23.f, 3.f, 14.f};
|
||||
std::vector<int64_t> classes = {0, 1};
|
||||
|
||||
if (n_trees > 1) {
|
||||
// Multiplies the number of trees to test the parallelization by trees.
|
||||
_multiply_update_array(lefts, n_trees);
|
||||
_multiply_update_array(rights, n_trees);
|
||||
_multiply_update_array(treeids, n_trees, (int64_t)3);
|
||||
_multiply_update_array(nodeids, n_trees);
|
||||
_multiply_update_array(featureids, n_trees);
|
||||
_multiply_update_array(thresholds, n_trees);
|
||||
_multiply_update_array_string(modes, n_trees);
|
||||
_multiply_update_array(target_treeids, n_trees, (int64_t)3);
|
||||
_multiply_update_array(target_nodeids, n_trees);
|
||||
_multiply_update_array(target_classids, n_trees);
|
||||
_multiply_update_array(target_weights, n_trees);
|
||||
}
|
||||
|
||||
//add attributes
|
||||
test.AddAttribute("nodes_truenodeids", lefts);
|
||||
test.AddAttribute("nodes_falsenodeids", rights);
|
||||
|
|
@ -51,6 +90,8 @@ void GenTreeAndRunTest(const std::vector<T>& X, const std::vector<float>& base_v
|
|||
} // default function is SUM
|
||||
|
||||
//fill input data
|
||||
std::vector<T> xn;
|
||||
std::vector<float> yn;
|
||||
if (one_obs) {
|
||||
auto X1 = X;
|
||||
auto results1 = results;
|
||||
|
|
@ -58,13 +99,69 @@ void GenTreeAndRunTest(const std::vector<T>& X, const std::vector<float>& base_v
|
|||
results1.resize(2);
|
||||
test.AddInput<T>("X", {1, 3}, X1);
|
||||
test.AddOutput<float>("Y", {1, 2}, results1);
|
||||
} else {
|
||||
} else if (n_obs == 8) {
|
||||
test.AddInput<T>("X", {8, 3}, X);
|
||||
test.AddOutput<float>("Y", {8, 2}, results);
|
||||
} else {
|
||||
int64_t i;
|
||||
size_t k;
|
||||
ASSERT_TRUE(n_obs % 8 == 0);
|
||||
xn.resize(n_obs * 3);
|
||||
yn.resize(n_obs * 2);
|
||||
for (i = 0; i < n_obs; i += 8) {
|
||||
for (k = 0; k < 24; ++k) {
|
||||
xn[i * 3 + k] = X[k];
|
||||
}
|
||||
for (k = 0; k < 16; ++k) {
|
||||
yn[i * 2 + k] = results[k];
|
||||
}
|
||||
}
|
||||
ASSERT_TRUE(i == n_obs);
|
||||
test.AddInput<T>("X", {n_obs, 3}, xn);
|
||||
test.AddOutput<float>("Y", {n_obs, 2}, yn);
|
||||
}
|
||||
|
||||
test.Run();
|
||||
} // namespace test
|
||||
|
||||
TEST(MLOpTest, TreeRegressorMultiTargetBatchTreeA2) {
|
||||
// TreeEnsemble implements different paths depending on n_trees or N.
|
||||
// This test and the next ones go through all sections for multi-targets.
|
||||
std::vector<float> X = {1.f, 0.0f, 0.4f, 3.0f, 44.0f, -3.f, 12.0f, 12.9f, -312.f, 23.0f, 11.3f, -222.f, 23.0f, 11.3f, -222.f, 23.0f, 3311.3f, -222.f, 23.0f, 11.3f, -222.f, 43.0f, 413.3f, -114.f};
|
||||
std::vector<float> results = {1.33333333f, 29.f, 3.f, 14.f, 2.f, 23.f, 2.f, 23.f, 2.f, 23.f, 2.66666667f, 17.f, 2.f, 23.f, 3.f, 14.f};
|
||||
std::vector<float> base_values{0.f, 0.f};
|
||||
GenTreeAndRunTest(X, base_values, results, "AVERAGE", true, 8, 1); // section A2
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorMultiTargetBatchTreeB2) {
|
||||
std::vector<float> X = {1.f, 0.0f, 0.4f, 3.0f, 44.0f, -3.f, 12.0f, 12.9f, -312.f, 23.0f, 11.3f, -222.f, 23.0f, 11.3f, -222.f, 23.0f, 3311.3f, -222.f, 23.0f, 11.3f, -222.f, 43.0f, 413.3f, -114.f};
|
||||
std::vector<float> results = {1.33333333f, 29.f, 3.f, 14.f, 2.f, 23.f, 2.f, 23.f, 2.f, 23.f, 2.66666667f, 17.f, 2.f, 23.f, 3.f, 14.f};
|
||||
std::vector<float> base_values{0.f, 0.f};
|
||||
GenTreeAndRunTest(X, base_values, results, "AVERAGE", true, 8, 130); // section B2
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorMultiTargetBatchTreeC2) {
|
||||
std::vector<float> X = {1.f, 0.0f, 0.4f, 3.0f, 44.0f, -3.f, 12.0f, 12.9f, -312.f, 23.0f, 11.3f, -222.f, 23.0f, 11.3f, -222.f, 23.0f, 3311.3f, -222.f, 23.0f, 11.3f, -222.f, 43.0f, 413.3f, -114.f};
|
||||
std::vector<float> results = {1.33333333f, 29.f, 3.f, 14.f, 2.f, 23.f, 2.f, 23.f, 2.f, 23.f, 2.66666667f, 17.f, 2.f, 23.f, 3.f, 14.f};
|
||||
std::vector<float> base_values{0.f, 0.f};
|
||||
GenTreeAndRunTest(X, base_values, results, "AVERAGE", false, 200, 130); // section C2
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorMultiTargetBatchTreeD2) {
|
||||
std::vector<float> X = {1.f, 0.0f, 0.4f, 3.0f, 44.0f, -3.f, 12.0f, 12.9f, -312.f, 23.0f, 11.3f, -222.f, 23.0f, 11.3f, -222.f, 23.0f, 3311.3f, -222.f, 23.0f, 11.3f, -222.f, 43.0f, 413.3f, -114.f};
|
||||
std::vector<float> results = {1.33333333f, 29.f, 3.f, 14.f, 2.f, 23.f, 2.f, 23.f, 2.f, 23.f, 2.66666667f, 17.f, 2.f, 23.f, 3.f, 14.f};
|
||||
std::vector<float> base_values{0.f, 0.f};
|
||||
GenTreeAndRunTest(X, base_values, results, "AVERAGE", true, 8, 30); // section D2
|
||||
GenTreeAndRunTest(X, base_values, results, "AVERAGE", false, 200, 30); // section D2
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorMultiTargetBatchTreeE2) {
|
||||
std::vector<float> X = {1.f, 0.0f, 0.4f, 3.0f, 44.0f, -3.f, 12.0f, 12.9f, -312.f, 23.0f, 11.3f, -222.f, 23.0f, 11.3f, -222.f, 23.0f, 3311.3f, -222.f, 23.0f, 11.3f, -222.f, 43.0f, 413.3f, -114.f};
|
||||
std::vector<float> results = {1.33333333f, 29.f, 3.f, 14.f, 2.f, 23.f, 2.f, 23.f, 2.f, 23.f, 2.66666667f, 17.f, 2.f, 23.f, 3.f, 14.f};
|
||||
std::vector<float> base_values{0.f, 0.f};
|
||||
GenTreeAndRunTest(X, base_values, results, "AVERAGE", false, 200, 1); // section E2
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorMultiTargetAverage) {
|
||||
std::vector<float> X = {1.f, 0.0f, 0.4f, 3.0f, 44.0f, -3.f, 12.0f, 12.9f, -312.f, 23.0f, 11.3f, -222.f, 23.0f, 11.3f, -222.f, 23.0f, 3311.3f, -222.f, 23.0f, 11.3f, -222.f, 43.0f, 413.3f, -114.f};
|
||||
std::vector<float> results = {1.33333333f, 29.f, 3.f, 14.f, 2.f, 23.f, 2.f, 23.f, 2.f, 23.f, 2.66666667f, 17.f, 2.f, 23.f, 3.f, 14.f};
|
||||
|
|
@ -97,7 +194,7 @@ TEST(MLOpTest, TreeRegressorMultiTargetMaxDouble) {
|
|||
GenTreeAndRunTest<double>(X, base_values, results, "MAX", true);
|
||||
}
|
||||
|
||||
void GenTreeAndRunTest1(const std::string& aggFunction, bool one_obs) {
|
||||
void GenTreeAndRunTest1(const std::string& aggFunction, bool one_obs, int64_t n_obs = 3, int n_trees = 1) {
|
||||
OpTester test("TreeEnsembleRegressor", 1, onnxruntime::kMLDomain);
|
||||
|
||||
//tree
|
||||
|
|
@ -115,6 +212,21 @@ void GenTreeAndRunTest1(const std::string& aggFunction, bool one_obs) {
|
|||
std::vector<float> target_weights = {33.33333f, 16.66666f, 33.33333f, -3.33333f, 16.66666f, -3.333333f};
|
||||
std::vector<int64_t> classes = {0, 1};
|
||||
|
||||
if (n_trees > 1) {
|
||||
// Multiplies the number of trees to test the parallelization by trees.
|
||||
_multiply_update_array(lefts, n_trees);
|
||||
_multiply_update_array(rights, n_trees);
|
||||
_multiply_update_array(treeids, n_trees, (int64_t)3);
|
||||
_multiply_update_array(nodeids, n_trees);
|
||||
_multiply_update_array(featureids, n_trees);
|
||||
_multiply_update_array(thresholds, n_trees);
|
||||
_multiply_update_array_string(modes, n_trees);
|
||||
_multiply_update_array(target_treeids, n_trees, (int64_t)3);
|
||||
_multiply_update_array(target_nodeids, n_trees);
|
||||
_multiply_update_array(target_classids, n_trees);
|
||||
_multiply_update_array(target_weights, n_trees);
|
||||
}
|
||||
|
||||
std::vector<float> results;
|
||||
if (aggFunction == "AVERAGE") {
|
||||
test.AddAttribute("aggregate_function", "AVERAGE");
|
||||
|
|
@ -149,16 +261,32 @@ void GenTreeAndRunTest1(const std::string& aggFunction, bool one_obs) {
|
|||
// SUM aggregation by default -- no need to add explicitly
|
||||
|
||||
//fill input data
|
||||
std::vector<float> xn, yn;
|
||||
if (one_obs) {
|
||||
ASSERT_TRUE(n_obs == 3);
|
||||
auto X1 = X;
|
||||
auto results1 = results;
|
||||
X1.resize(2);
|
||||
results1.resize(1);
|
||||
test.AddInput<float>("X", {1, 2}, X1);
|
||||
test.AddOutput<float>("Y", {1, 1}, results1);
|
||||
} else {
|
||||
} else if (n_obs == 3) {
|
||||
test.AddInput<float>("X", {3, 2}, X);
|
||||
test.AddOutput<float>("Y", {3, 1}, results);
|
||||
} else {
|
||||
ASSERT_TRUE(n_obs % 3 == 0);
|
||||
xn.resize(n_obs * 2);
|
||||
yn.resize(n_obs);
|
||||
for (int64_t i = 0; i < n_obs; i += 3) {
|
||||
for (size_t k = 0; k < 6; ++k) {
|
||||
xn[i * 2 + k] = X[k];
|
||||
}
|
||||
for (size_t k = 0; k < 3; ++k) {
|
||||
yn[i + k] = results[k];
|
||||
}
|
||||
}
|
||||
test.AddInput<float>("X", {n_obs, 2}, xn);
|
||||
test.AddOutput<float>("Y", {n_obs, 1}, yn);
|
||||
}
|
||||
test.Run();
|
||||
}
|
||||
|
|
@ -168,6 +296,34 @@ TEST(MLOpTest, TreeRegressorSingleTargetSum) {
|
|||
GenTreeAndRunTest1("SUM", true);
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorSingleTargetSumBatch) {
|
||||
GenTreeAndRunTest1("SUM", false, 201);
|
||||
GenTreeAndRunTest1("SUM", false, 40002);
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorSingleTargetBatchTreeA) {
|
||||
// TreeEnsemble implements different paths depending on n_trees or N.
|
||||
// This test and the next ones goe through all sections for one target.
|
||||
GenTreeAndRunTest1("SUM", true, 3, 1); // section A
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorSingleTargetBatchTreeB) {
|
||||
GenTreeAndRunTest1("AVERAGE", true, 3, 30); // section B
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorSingleTargetBatchTreeC) {
|
||||
GenTreeAndRunTest1("AVERAGE", false, 3, 1); // section C
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorSingleTargetBatchTreeD) {
|
||||
GenTreeAndRunTest1("AVERAGE", false, 201, 30); // section D
|
||||
GenTreeAndRunTest1("AVERAGE", false, 201, 130); // section D
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorSingleTargetBatchTreeE) {
|
||||
GenTreeAndRunTest1("AVERAGE", false, 201, 1); // section E
|
||||
}
|
||||
|
||||
TEST(MLOpTest, TreeRegressorSingleTargetAverage) {
|
||||
GenTreeAndRunTest1("AVERAGE", false);
|
||||
GenTreeAndRunTest1("AVERAGE", true);
|
||||
|
|
|
|||
|
|
@ -4,10 +4,23 @@
|
|||
#include "gtest/gtest.h"
|
||||
#include "test/providers/provider_test_utils.h"
|
||||
#include "test/providers/compare_provider_test_utils.h"
|
||||
#include "core/providers/cpu/tensor/transpose.h"
|
||||
|
||||
namespace onnxruntime {
|
||||
namespace test {
|
||||
|
||||
TEST(TransposeOpTest, IsTransposeReshapeTest) {
|
||||
std::vector<int64_t> input_dims{1, 2, 3, 4, 1};
|
||||
std::vector<size_t> perm{0, 1, 2, 3, 4};
|
||||
ASSERT_TRUE(IsTransposeReshape(perm, input_dims));
|
||||
perm = std::vector<size_t>{1, 2, 3, 0, 4};
|
||||
ASSERT_TRUE(IsTransposeReshape(perm, input_dims));
|
||||
perm = std::vector<size_t>{4, 1, 0, 2, 3};
|
||||
ASSERT_TRUE(IsTransposeReshape(perm, input_dims));
|
||||
perm = std::vector<size_t>{4, 1, 0, 3, 2};
|
||||
ASSERT_FALSE(IsTransposeReshape(perm, input_dims));
|
||||
}
|
||||
|
||||
// Some of the tests can't run on TensorrtExecutionProvider because of errors.
|
||||
// Those tests will fallback to other EPs.
|
||||
|
||||
|
|
@ -124,11 +137,11 @@ TEST(TransposeOpTest, TwoDim_int16) {
|
|||
2, 5,
|
||||
3, 6};
|
||||
|
||||
#if defined(OPENVINO_CONFIG_MYRIAD) || defined(OPENVINO_CONFIG_VAD_M)
|
||||
TransposeTest(input_shape, input_vals, &perm, expected_shape, expected_vals, true, false);
|
||||
#else
|
||||
TransposeTest(input_shape, input_vals, &perm, expected_shape, expected_vals);
|
||||
#endif
|
||||
#if defined(OPENVINO_CONFIG_MYRIAD) || defined(OPENVINO_CONFIG_VAD_M)
|
||||
TransposeTest(input_shape, input_vals, &perm, expected_shape, expected_vals, true, false);
|
||||
#else
|
||||
TransposeTest(input_shape, input_vals, &perm, expected_shape, expected_vals);
|
||||
#endif
|
||||
}
|
||||
|
||||
TEST(TransposeOpTest, TwoDim_mlfloat16) {
|
||||
|
|
@ -246,6 +259,39 @@ TEST(TransposeOpTest, ThreeDimSuffix) {
|
|||
TransposeTest(input_shape, input_vals, &perm, expected_shape, expected_vals, false); //TensorRT: illegal error
|
||||
}
|
||||
|
||||
TEST(TransposeOpTest, TransposeReshape) {
|
||||
std::vector<int64_t> input_shape({1, 4, 2, 1, 3});
|
||||
std::vector<float> input_vals = {
|
||||
1.0f, 2.0f, 3.0f,
|
||||
4.0f, 5.0f, 6.0f,
|
||||
|
||||
1.1f, 2.1f, 3.1f,
|
||||
4.1f, 5.1f, 6.1f,
|
||||
|
||||
1.2f, 2.2f, 3.2f,
|
||||
4.2f, 5.2f, 6.2f,
|
||||
|
||||
1.3f, 2.3f, 3.3f,
|
||||
4.3f, 5.3f, 6.3f};
|
||||
|
||||
std::vector<int64_t> perm = {1, 3, 2, 4, 0};
|
||||
std::vector<int64_t> expected_shape({4, 1, 2, 3, 1});
|
||||
auto expected_vals = {
|
||||
1.0f, 2.0f, 3.0f,
|
||||
4.0f, 5.0f, 6.0f,
|
||||
|
||||
1.1f, 2.1f, 3.1f,
|
||||
4.1f, 5.1f, 6.1f,
|
||||
|
||||
1.2f, 2.2f, 3.2f,
|
||||
4.2f, 5.2f, 6.2f,
|
||||
|
||||
1.3f, 2.3f, 3.3f,
|
||||
4.3f, 5.3f, 6.3f};
|
||||
|
||||
TransposeTest(input_shape, input_vals, &perm, expected_shape, expected_vals, false); //TensorRT: illegal error
|
||||
}
|
||||
|
||||
TEST(TransposeOpTest, ThreeDimStr) {
|
||||
std::vector<int64_t> input_shape({4, 2, 3});
|
||||
std::vector<std::string> input_vals = {
|
||||
|
|
@ -419,10 +465,112 @@ TEST(TransposeOpTest, SingleAxisMovingInwardsBlockCopy) {
|
|||
TransposeTest(input_shape, input_vals, &perm, expected_shape, expected_vals, false);
|
||||
}
|
||||
|
||||
TEST(TransposeOpTest, NDim) {
|
||||
std::vector<int64_t> input_shape({2, 2, 2, 2});
|
||||
std::vector<float> input_vals = {1.0f, 2.0f, 3.0f, 4.0f,
|
||||
5.0f, 6.0f, 7.0f, 8.0f,
|
||||
9.0f, 10.0f, 11.0f, 12.0f,
|
||||
13.0f, 14.0f, 15.0f, 16.0f};
|
||||
|
||||
std::vector<int64_t> perm = {1, 0, 2, 3};
|
||||
auto expected_vals = {1.0f, 2.0f, 3.0f, 4.0f,
|
||||
9.0f, 10.0f, 11.0f, 12.0f,
|
||||
5.0f, 6.0f, 7.0f, 8.0f,
|
||||
13.0f, 14.0f, 15.0f, 16.0f};
|
||||
TransposeTest(input_shape, input_vals, &perm, input_shape, expected_vals);
|
||||
|
||||
perm = {1, 0, 3, 2};
|
||||
auto expected_vals2 = {1.0f, 3.0f, 2.0f, 4.0f,
|
||||
9.0f, 11.0f, 10.0f, 12.0f,
|
||||
5.0f, 7.0f, 6.0f, 8.0f,
|
||||
13.0f, 15.0f, 14.0f, 16.0f};
|
||||
TransposeTest(input_shape, input_vals, &perm, input_shape, expected_vals2);
|
||||
}
|
||||
|
||||
TEST(TransposeOpTest, DoTransposeImpl) {
|
||||
std::vector<int64_t> input_shape({5, 2, 1, 3});
|
||||
std::vector<float> input_vals(30);
|
||||
for (auto it = input_vals.begin(); it != input_vals.end(); ++it) {
|
||||
*it = static_cast<float>(std::distance(input_vals.begin(), it));
|
||||
}
|
||||
std::vector<int64_t> perm = {2, 1, 0, 3};
|
||||
std::vector<int64_t> expected_shape({1, 2, 5, 3});
|
||||
auto expected_vals = {0.0f, 1.0f, 2.0f, 6.0f, 7.0f, 8.0f,
|
||||
12.0f, 13.0f, 14.0f, 18.0f, 19.0f, 20.0f,
|
||||
24.0f, 25.0f, 26.0f, 3.0f, 4.0f, 5.0f,
|
||||
9.0f, 10.0f, 11.0f, 15.0f, 16.0f, 17.0f,
|
||||
21.0f, 22.0f, 23.0f, 27.0f, 28.0f, 29.0f};
|
||||
TransposeTest(input_shape, input_vals, &perm, expected_shape, expected_vals);
|
||||
}
|
||||
|
||||
TEST(TransposeOpTest, DoTransposeImplString) {
|
||||
std::vector<int64_t> input_shape({5, 2, 1, 3});
|
||||
std::vector<std::string> input_vals(30);
|
||||
for (auto it = input_vals.begin(); it != input_vals.end(); ++it) {
|
||||
*it = std::string("n") + std::to_string(static_cast<int>(std::distance(input_vals.begin(), it)));
|
||||
}
|
||||
std::vector<int64_t> perm = {2, 1, 0, 3};
|
||||
std::vector<int64_t> expected_shape({1, 2, 5, 3});
|
||||
std::initializer_list<std::string> expected_vals = {"n0", "n1", "n2", "n6", "n7", "n8",
|
||||
"n12", "n13", "n14", "n18", "n19", "n20",
|
||||
"n24", "n25", "n26", "n3", "n4", "n5",
|
||||
"n9", "n10", "n11", "n15", "n16", "n17",
|
||||
"n21", "n22", "n23", "n27", "n28", "n29"};
|
||||
TransposeTest(input_shape, input_vals, &perm, expected_shape, expected_vals);
|
||||
}
|
||||
|
||||
TEST(TransposeOpTest, DoTransposeEltWise) {
|
||||
// Configuration where DoTransposeEltWise is called.
|
||||
std::vector<int64_t> input_shape({2, 2, 2, 2});
|
||||
std::vector<float> input_vals = {1.0f, 2.0f, 3.0f, 4.0f,
|
||||
5.0f, 6.0f, 7.0f, 8.0f,
|
||||
9.0f, 10.0f, 11.0f, 12.0f,
|
||||
13.0f, 14.0f, 15.0f, 16.0f};
|
||||
|
||||
std::vector<int64_t> perm = {1, 0, 3, 2};
|
||||
auto expected_vals2 = {1.0f, 3.0f, 2.0f, 4.0f,
|
||||
9.0f, 11.0f, 10.0f, 12.0f,
|
||||
5.0f, 7.0f, 6.0f, 8.0f,
|
||||
13.0f, 15.0f, 14.0f, 16.0f};
|
||||
TransposeTest(input_shape, input_vals, &perm, input_shape, expected_vals2);
|
||||
|
||||
// Specific test which tests that function DoTransposeEltWise does not
|
||||
// copy values outside the target buffer.
|
||||
TensorShape tensor_shape(input_shape);
|
||||
std::vector<size_t> stride(input_shape.size());
|
||||
for (size_t i = 0; i < input_shape.size(); i++) {
|
||||
size_t inpdim = perm[i];
|
||||
if (inpdim + 1 < input_shape.size())
|
||||
stride[i] = tensor_shape.SizeFromDimension(inpdim + 1);
|
||||
else
|
||||
stride[i] = 1;
|
||||
}
|
||||
|
||||
std::vector<float> input_vals_end = {1.0f, 2.0f, 3.0f, 4.0f,
|
||||
5.0f, 6.0f, 7.0f, 8.0f,
|
||||
9.0f, 10.0f, 11.0f, 12.0f,
|
||||
13.0f, 14.0f, 15.0f, 16.0f,
|
||||
-1.0f, -1.0f};
|
||||
std::vector<float> target(input_vals_end.size(), 17.0f);
|
||||
|
||||
std::vector<float> expected_vals3 = {1.0f, 3.0f, 2.0f, 4.0f,
|
||||
9.0f, 11.0f, 10.0f, 12.0f,
|
||||
5.0f, 7.0f, 6.0f, 8.0f,
|
||||
13.0f, 15.0f, 14.0f, 16.0f,
|
||||
17.0f, 17.0f};
|
||||
|
||||
DoTransposeEltWise(input_shape.size(), input_shape, 16,
|
||||
stride, (uint8_t*)input_vals_end.data(), (uint8_t*)target.data(),
|
||||
sizeof(float));
|
||||
for (size_t i = 0; i < input_vals_end.size(); ++i) {
|
||||
ASSERT_TRUE(target[i] == expected_vals3[i]);
|
||||
}
|
||||
}
|
||||
|
||||
#if USE_CUDA
|
||||
constexpr const char* kGpuExecutionProvider = kCudaExecutionProvider;
|
||||
constexpr const char* kGpuExecutionProvider = kCudaExecutionProvider;
|
||||
#elif USE_ROCM
|
||||
constexpr const char* kGpuExecutionProvider = kRocmExecutionProvider;
|
||||
constexpr const char* kGpuExecutionProvider = kRocmExecutionProvider;
|
||||
#endif
|
||||
|
||||
#if defined(USE_CUDA) || defined(USE_ROCM)
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
Name: onnxruntime
|
||||
Version: 1.5.2
|
||||
Version: 1.6.0
|
||||
Release: 1%{?dist}
|
||||
Summary: onnxruntime
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue