Commit graph

3 commits

Author SHA1 Message Date
Adam Pocock
aed43f429a
[java] Enable output pinning in OrtSession and OrtTrainingSession (#16835) 2023-09-26 01:49:13 -07:00
Adam Pocock
a8e776b78b
[java] Adds support for fp16 and bf16 tensors (#16703)
### Description
The Java API currently only supports fp16 output tensors which it
automatically casts to floats on the way out. This PR adds support for
creating fp16 and bf16 tensors (from `java.nio.Buffer` objects or as the
output of models, creation from Java short arrays is not supported),
along with efficient methods for casting `FloatBuffer` into
`ShortBuffer` filled with fp16 or bf16 values and vice versa.

The fp16 conversions use a trick to pull in the efficient conversion
methods added to Java 20, falling back to ports of the MLAS methods
otherwise. The Java 20 methods can be special cased by the C2 JIT
compiler to emit the single instruction on x86 and ARM which converts
fp32<->fp16, or the vectorized versions thereof, so they should be quite
a bit faster than the MLAS ported one.

### Motivation and Context
fp16 and bf16 are increasingly popular formats and we've had several
requests for this functionality. Fixes #7003.

cc @yuslepukhin  @cassiebreviu

---------

Co-authored-by: Scott McKay <Scott.McKay@microsoft.com>
2023-07-21 21:14:41 +10:00
Adam Pocock
ba91457183
[java] Adding addExternalInitializers and addInitializer to OrtSession.SessionOptions (#16198)
### Description
Adds support for adding external initializers or overriding initializers
to a session options from Java.

### Motivation and Context
We want to instantiate large models from Java without filesystem access.

cc @yuslepukhin
2023-07-05 12:51:59 -07:00