mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-28 20:11:22 +00:00
a8e776b78b
### Description The Java API currently only supports fp16 output tensors which it automatically casts to floats on the way out. This PR adds support for creating fp16 and bf16 tensors (from `java.nio.Buffer` objects or as the output of models, creation from Java short arrays is not supported), along with efficient methods for casting `FloatBuffer` into `ShortBuffer` filled with fp16 or bf16 values and vice versa. The fp16 conversions use a trick to pull in the efficient conversion methods added to Java 20, falling back to ports of the MLAS methods otherwise. The Java 20 methods can be special cased by the C2 JIT compiler to emit the single instruction on x86 and ARM which converts fp32<->fp16, or the vectorized versions thereof, so they should be quite a bit faster than the MLAS ported one. ### Motivation and Context fp16 and bf16 are increasingly popular formats and we've had several requests for this functionality. Fixes #7003. cc @yuslepukhin @cassiebreviu --------- Co-authored-by: Scott McKay <Scott.McKay@microsoft.com> |
||
|---|---|---|
| .. | ||
| ai_onnxruntime_OnnxMap.c | ||
| ai_onnxruntime_OnnxRuntime.c | ||
| ai_onnxruntime_OnnxSequence.c | ||
| ai_onnxruntime_OnnxSparseTensor.c | ||
| ai_onnxruntime_OnnxTensor.c | ||
| ai_onnxruntime_OrtAllocator.c | ||
| ai_onnxruntime_OrtEnvironment.c | ||
| ai_onnxruntime_OrtEnvironment_ThreadingOptions.c | ||
| ai_onnxruntime_OrtSession.c | ||
| ai_onnxruntime_OrtSession_RunOptions.c | ||
| ai_onnxruntime_OrtSession_SessionOptions.c | ||
| ai_onnxruntime_OrtTrainingSession.c | ||
| ai_onnxruntime_OrtTrainingSession_OrtCheckpointState.c | ||
| ai_onnxruntime_providers_OrtCUDAProviderOptions.c | ||
| ai_onnxruntime_providers_OrtTensorRTProviderOptions.c | ||
| OrtJniUtil.c | ||
| OrtJniUtil.h | ||