saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-07 17:15:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update ABI.md (#299)

Browse source 
* Update ABI.md
This commit is contained in:
Changming Sun 2019-01-10 16:34:42 -08:00 committed by GitHub
parent 02852a0881
commit 02962ce9d8
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 47 additions and 46 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

43
docs/ABI.md
View file

@ -1,43 +0,0 @@
# ONNX Runtime ABI
We release ONNX Runtime as both static library and shared library on Windows, Linux and Mac OS X. [ABI (Application Binary Interface)](https://en.wikipedia.org/wiki/Application_binary_interface) is only for the shared library. It allows you upgrade ONNX Runtime to a newer version without recompiling.
The ABI contains:
1. A set of C functions for creating an inference session and load/run a model. No global variables would be directly exported. All these functions can be directly used in .net and .net core through PInvoke. All these public symbols are C symbols, no C++.
2. (TODO) A C++ API for authoring custom ops and put them into a separated dll.
# Functionality
[C API](C_API.md)
# Integration
Q: Should I statically link to ONNX Runtime or dynamically?
A: On Windows, Any custom op DLL must dynamically link to ONNX Runtime.
Dynamical linking also helps on solving diamond dependency problem. For example, if part of your program depends on ONNX 1.2 but ONNX Runtime depends on ONNX 1.3, then dynamically linking to them would be better.
Q: Any requirement on CUDA version? My program depends on CUDA 9.0, but the ONNX Runtime binary was built with CUDA 9.1. Is it ok to put them together?
A: Yes. Because ONNX Runtime statically linked to CUDA.
# Dev Notes
## Global Variables
Global variables may get constructed or destructed inside "DllMain". There are significant limits on what you can safely do in a DLL entry point. See ['DLL General Best Practices'](https://docs.microsoft.com/en-us/windows/desktop/dlls/dynamic-link-library-best-practices). For example, you can't put a ONNX Runtime InferenceSession into a global variable.
## Undefined symbols in a shared library
On Windows, you can't build a DLL with undefined symbols. Every symbol must be get resolved at link time. On Linux, you can.
In this project, we setup a rule: when building a shared library, every symbol must get resolved at link time, unless it's a custom op.
For custom op, on Linux, don't pass any libraries(except libc, pthreads) to linker. So that, even the application is statically linked to ONNX Runtime, they can still use the same custom op binary.
## Default visibility
On POSIX systems, please always specify "-fvisibility=hidden" and "-fPIC" when compiling any code in ONNX Runtime shared library.
See [pybind11 FAQ](https://github.com/pybind/pybind11/blob/master/docs/faq.rst#someclass-declared-with-greater-visibility-than-the-type-of-its-field-someclassmember--wattributes)
## RTLD_LOCAL vs RTLD_GLOBAL
RTLD_LOCAL and RTLD_GLOBAL are two flags of [dlopen(3)](http://pubs.opengroup.org/onlinepubs/9699919799/functions/dlopen.html) function on Linux. By default, it's RTLD_LOCAL. And basically you can say, there no corresponding things like RTLD_GLOBAL on Windows.
If your application is a shared library, which statically linked to ONNX Runtime, and your application needs to dynamically load a custom op, then your application must be loaded with RTLD_GLOBAL. In all other cases, you should use RTLD_LOCAL. ONNX Runtime python binding is a good example of why sometimes RTLD_GLOBAL is needed.

47
docs/ABI_Dev_Notes.md Normal file
View file

@ -0,0 +1,47 @@
## Global Variables
Global variables may get constructed or destructed inside "DllMain". There are significant limits on what you can safely do in a DLL entry point. See ['DLL General Best Practices'](https://docs.microsoft.com/en-us/windows/desktop/dlls/dynamic-link-library-best-practices). For example, you can't put a ONNX Runtime InferenceSession into a global variable.
## Thread Local variables
Thread Local variables must be function local, that on Windows they will be initialized as the first time of use. Otherwise, it may not work.
Also, you must destroy these thread Local variables before onnxruntime.dll is unloaded, if the variable has a non-trivial destructor. That means, only onnxruntime internal threads can access these variables. It is, the thread must be created by onnxruntime and destroyed by onnxruntime.
## No undefined symbols
On Windows, you can't build a DLL with undefined symbols. Every symbol must be get resolved at link time. On Linux, you can.
In order to simplify things, we require every symbol must get resolved at link time. The same rule applies for all the platforms. And this is easier for us to control symbol visibility.
## Default visibility and how to export a symbol
On Linux, by default, at linker's view, every symbol is global. It's easy to use but it's also much easier to cause conflicts and core dumps. We have encountered too many such problems in ONNX python binding. Indeed, if you have a well design, for each shared lib, you only need to export **one** function. ONNX Runtime python binding is a good example. See [pybind11 FAQ](https://github.com/pybind/pybind11/blob/master/docs/faq.rst#someclass-declared-with-greater-visibility-than-the-type-of-its-field-someclassmember--wattributes) for more info.
For controling the visibility, we use linkder version scripts on Linux and def files on Windows. They work similar. That:
1. Only C functions can be exported.
2. All the function names must be explicitly listed in a text file.
3. Don't export any C++ class/struct, or global variable.
Also, on Linux and Mac operating systems, all the code must be compiled with "-fPIC".
On Windows, we don't use dllexport but we still need dllimport.
Therefore, our DLLEXPORT macro is like:
```
#ifdef _WIN32
// Define ORT_DLL_IMPORT if your program is dynamically linked to Ort.
#ifdef ORT_DLL_IMPORT
#define ORT_EXPORT __declspec(dllimport)
#else
#define ORT_EXPORT
#endif
#else
#define ORT_EXPORT
#endif
```
## RTLD_LOCAL vs RTLD_GLOBAL
RTLD_LOCAL and RTLD_GLOBAL are two flags of [dlopen(3)](http://pubs.opengroup.org/onlinepubs/9699919799/functions/dlopen.html) function on POSIX systems. By default, it's RTLD_LOCAL. And basically you can say, there no corresponding things like RTLD_GLOBAL on Windows.
There is one case you need to use RTLD_GLOBAL on POSIX systems:
1. There is a shared lib which is dynamically loaded by some application(like python or dotnet)
2. The shared lib is statically linked to ONNX Runtime
3. The shared lib needs to dynamically load a custom op
Then the shared lib should be loaded with RTLD_GLOBAL, not RTLD_LOCAL. Otherwise in the custom op library, it can not find ONNX Runtime symbols.

3
docs/Versioning.md
View file

@ -5,9 +5,6 @@ ONNX Runtime follows [Semantic Versioning 2.0](https://semver.org/) for its publ
Each release has the form MAJOR.MINOR.PATCH. The meanings of MAJOR, MINOR and PATCH are
same as what is described in the semantic versioning doc linked above.
### ABI
Please refer to the [ABI doc](ABI.md) for more details about binary compatibility.
## Current stable release version
The version number of the current stable release can be found
[here](../VERSION_NUMBER)