Commit graph

28 commits

Author SHA1 Message Date
Rachel Guo
288b80d363
Add MacOS build to ORT C Pod (#18550)
### Description
<!-- Describe your changes. -->

As title.

1. Add macos build as an optionally enabled arch for pod and changes to
exsiting build_ios_framework/assemble_c_pod scripts.
2. Enable macos build arch in ios packaging pipeline (currently for
variants other than Mobile) and check the output artifacts are correct.
3. Write MacOS Test Target scheme in the test app and integrate into ios
packaging CI testing pipeline.
Currently the changes only apply to onnxruntime-c pod. as the original
request was from ORT SPM which consumes the onnxruntime-c pod only as
the binary target. TODO: could look into adding macos platform to objc
pod as well.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Enable macos platform support in cocoapods. and also potentially produce
binary target for enabling macos platform in SPM as well.

Replace https://github.com/microsoft/onnxruntime/pull/18334

---------

Co-authored-by: rachguo <rachguo@rachguos-Mac-mini.local>
Co-authored-by: rachguo <rachguo@rachguos-Mini.attlocal.net>
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
2023-11-28 10:11:53 -08:00
Scott McKay
ae211999dd
Attempt to make the usage of the Android emulator in CIs more robust (#17903)
### Description
<!-- Describe your changes. -->
Android emulator usage updates:
- Change approach to detecting boot has completed
- use `-delay-adb` and a simple command (`ls`) with `wait-for-device` as
the first step
    - this ensures enough startup has occurred for adb to be responsive
- use secondary loop on the python side to check for sys.boot_completed
to be set
- doing the check on the python side provides more feedback and seems to
work well
- make the 'stop' logic more precise by using psutil
- add internal timeout of 20 mins for emulator startup
  - waiting for the CI jobs overall timeout is way too long
- value is hardcoded for now (most CIs startup in under 10 mins) but
could be made configurable if needed

CI updates:
- add template for using the Android emulator
  - update CIs to use template
- reorder React Native CI
- minimize the time the Android emulator or iOS simulator is running by
moving some build steps around
  - don't run both at the same time
- unnecessary and potentially adds significant memory pressure to the
machine
- fix QNN Android emulator CI as much as possible
- now everything works apart from running onnx_test_runner with the QNN
EP

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Fix inconsistent detection of the emulator boot completing.

---------

Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
2023-10-15 08:42:36 +10:00
Changming Sun
a942bbf489
Update nodejs to 18.x (#17657)
1. Upgrade nodejs from 16.x to 18.x for Windows pipelines
2. Avoid using Azure DevOps "NodeTool" on Linux. The tool installs
nodejs from internet or local disk cache. But we already moved all Linux
tests to docker. So we do not need the installer anymore.
3. Remove some other unused code.
2023-09-25 14:12:11 -07:00
Scott McKay
2fd25de360
Use verbose logging in Android emulator in React Native CI (#16528)
### Description
<!-- Describe your changes. -->
Set emulator logging to verbose to see if it helps with intermittent
React Native CI failures when emulator crashes at startup


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
2023-06-30 11:51:20 +10:00
Rachel Guo
892b1b19ea
[js/rn] limit x86_64 arch in detox xcodebuild for react native e2e test (#16460)
### Description
<!-- Describe your changes. -->

As title.




### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Works with local onnxruntime-c pod in js/rn/e2e test.
2023-06-27 09:45:04 -07:00
Yi Zhang
6e9541046e
extend react native ci timeout limit (#16469)
### Description
<!-- Describe your changes. -->

### Motivation and Context
2 consecutive runs in npm pipeline failed due to time out
2023-06-27 08:44:03 +08:00
Rachel Guo
04dbdc96bf
[js/rn] Fix React Native CI pipeline E2E test (#16447)
### Description
<!-- Describe your changes. -->

Based on this kindly provided quick fix:
https://github.com/microsoft/onnxruntime/pull/16411

See more description in the above linked pr about bumping AGP version,
etc.

Also fixed import header file path in detox e2e test.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Good build:

https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=1041757&view=logs&j=de302ec2-2305-57e0-e8c6-cd89c569f2a3&t=9894c870-b8ce-548d-51ff-8f44d21a4117&l=18
2023-06-22 14:33:49 -07:00
Rachel Guo
b4b126ffb0
Set onnxruntime-c local pod path environment variable for react native e2e tests on ci (#16431)
### Description
<!-- Describe your changes. -->
Set onnxruntime-c local pod path environment variable for react native
e2e tests on react-native-ci.yml


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Previously the E2E test project is not properly consuming a local built
onnxruntime-c version pod.


https://github.com/microsoft/onnxruntime/pull/16411#issuecomment-1598512816
2023-06-21 00:27:36 -07:00
Edward Chen
4f23577cb5
[React Native] Publish E2E test logs on build failure too. (#16327)
### Description
<!-- Describe your changes. -->

Publish E2E test logs on build failure too.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Get more information about intermittent test failures.
2023-06-12 17:56:46 -07:00
Edward Chen
1261d0b8ba
Fix some build issues on MacOS with Xcode 14.3. (#15878)
- Fix flatbuffers flatc warning, unused-but-set-variable.
- Address `-Wshorten-64-to-32` warnings (fix in our code, allow in dependencies' code).
- Update CI builds to use Xcode 14.3.
- Update minimum iOS version to 12.0.
- Update Mac hosted agents to MacOS 13 where possible.
2023-06-07 12:07:11 -07:00
Rachel Guo
2cb3fb18b5
Integrate React Native E2E test with detox framework (#15133)
### Description
<!-- Describe your changes. -->

Integrate react native e2e test framework with detox.
https://wix.github.io/Detox/

Good build in CI:

https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=946695&view=results

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Write cross-platform end-to-end tests in JavaScript. 
Resolve flaky e2e tests in react native ci pipelines.

---------

Co-authored-by: rachguo <rachguo@rachguos-Mini.attlocal.net>
Co-authored-by: rachguo <rachguo@rachguos-Mac-mini.local>
2023-04-21 09:46:26 -07:00
Yi Zhang
84746a8efe
Revert "Retry the step of Start Android simulator (#15584)" (#15620)
This reverts commit 64b63921a2.


### Motivation and Context
From
https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=970086&view=logs&s=28fb2bf2-39c5-5feb-1887-4904233f6193&j=de302ec2-2305-57e0-e8c6-cd89c569f2a3
It's useless to rerun the step.
2023-04-21 08:33:18 -07:00
Yi Zhang
64b63921a2
Retry the step of Start Android simulator (#15584)
### Description
Add Retry once There's a failure in `Start Android Simulator`. 

### Motivation and Context
`Start Android Simulator` isn't stable enough and the pipeline would
hang.

We could find many instances in
https://dev.azure.com/onnxruntime/onnxruntime/_pipeline/analytics/stageawareoutcome?definitionId=188&contextType=build
2023-04-20 12:06:35 +08:00
Yi Zhang
feafbc4263
Refactor all Mac build steps (#15440)
### Description


### Motivation and Context
Make the compilation cache steps easy to use and maintain
Reduce cache storage.
2023-04-11 12:12:46 +08:00
Hariharan Seshadri
f77c8f4863
Fix Npm packaging pipeline (#15425)
### Description
It seems like https://github.com/microsoft/onnxruntime/pull/15329
re-worked some jobs in `react-native-ci.yml` into stages. When this
template is used from within `npm-packaging-pipeline.yml`, there is
problem in that there is a stage that contains multiple stages as jobs.
Per my understanding, this is not acceptable to Azure DevOps. So,
re-working some portion of `npm-packaging-pipeline.yml` to accomadate
changes in https://github.com/microsoft/onnxruntime/pull/15329

### Motivation and Context
Fix NPM packaging pipeline
Validating test run with fix:
https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=297391&view=results
2023-04-07 22:13:39 -07:00
Yi Zhang
962d8d2b19
Add compilation cache in react native CI (#15329)
### Description
1. Replacing jobs with stages for better debugging and maintainance
2. Added compilation cache to accelerate the workflow.
3. Splited building protobuf and major code as 2 tasks



### Motivation and Context
Reduced compilation time about one hour.
test run:

https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=943695&view=logs&j=de302ec2-2305-57e0-e8c6-cd89c569f2a3&t=8b360243-7783-51da-8079-2304089d3d1d
2023-04-06 10:39:14 +08:00
Changming Sun
ffcfb1ec98
Remove protobuf submodule (#15190)
### Description
Remove protobuf submodule as a follow-up of #13523

"Android CI Pipeline" and "Zip-Nuget-Java-Nodejs Packaging Pipeline"
need to be tested.


### Motivation and Context
It is related to
[AB#11753](https://aiinfra.visualstudio.com/6a833879-cd9b-44a4-a9de-adc2d818f13c/_workitems/edit/11753)

Fixed
[AB#14027](https://aiinfra.visualstudio.com/6a833879-cd9b-44a4-a9de-adc2d818f13c/_workitems/edit/14027)
2023-03-27 10:35:49 -07:00
Rachel Guo
db4e664f7c
Re-enable react native e2e android unit test for CI and upgrade targetSDK level for test project (#14989)
### Description
<!-- Describe your changes. -->

Re-enable the react native e2e android unit test for react native CI as
recent change of specifying `default` instead of `google-apis` in
android emulator CI tests gives pretty stable result for now.

Upgrade the targetSDKversion for gradle test project in
react-native/android to meet minimum target api level requirement for
Google Play apps.


https://support.google.com/googleplay/android-developer/answer/11926878?hl=en

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

React Native CI issue.
2023-03-14 13:35:38 -07:00
Jian Chen
b4fe98ac2e
Update to MacOS-12 (#14924)
### Description
<!-- Describe your changes. -->


Update to MacOS-12
### Motivation and Context

Fixed
[AB#13233](https://aiinfra.visualstudio.com/6a833879-cd9b-44a4-a9de-adc2d818f13c/_workitems/edit/13233)
2023-03-09 10:18:14 -08:00
Edward Chen
c46c7ccba5
Update Gradle version (#14862)
- Update Gradle version used in most places from 6.8.3 to 8.0.1. Update Android Gradle Plugin version where applicable.
  Not updated in this change: React Native Android projects (under `js/react_native/`). That can be done later along with updating the React Native projects.

- Add Gradle wrapper in `java/` to make it easier to consistently use a specific Gradle version.
2023-03-08 12:22:06 -08:00
Rachel Guo
0700788b6e
Disable e2e android react native CI test temporarily (#14803)
### Description
<!-- Describe your changes. -->

Disable e2e android react native test temporarily to unblock the CI
failure with no easy fix.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Temp solution to unblock CI failure.
2023-02-24 09:32:18 -08:00
Edward Chen
ad78579b66
Update java/build.gradle to not use deprecated features that were removed in gradle 8.0. (#14733)
### Description
<!-- Describe your changes. -->

Update java/build.gradle to not use deprecated features that were
removed in gradle 8.0.
Also move gradle wrapper setup from a script into a step template.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Fix builds which use hosted Mac agents and gradle.

Recently the system version of gradle got upgraded to 8.0. Even though
we use an older gradle wrapper version, java/build.gradle is still
processed with gradle 8.0 in the initial call to `gradle wrapper`.
2023-02-20 11:19:49 +08:00
Edward Chen
2fa18ea77e
[React Native CI] Record more info to debug E2E test (#13329)
Record more info from the React Native CI E2E test. In particular, log the view hierarchy when exiting the test and dump logs from Android emulator to the build output.
2022-10-18 17:21:28 -07:00
Scott McKay
078ceab1db
Use full ORT package for onnxruntime-react-native. (#13037)
**Description**: 
Use full ORT package for onnxruntime-react-native.

Left the params required for the mobile build in comments so they're
easily discovered if we need to create onnxruntime-react-native-mobile
in the future.

**Motivation and Context**
Remove barrier to using ORT with react native as the mobile package that
was being used supports a limited range of opsets/operators/types, and
requires ORT format models. The full package will run any model.
2022-09-23 07:20:03 +10:00
Edward Chen
454f77cd94
Update kernel matching logic: decouple from op schemas and remove kernel def hashes (#12791)
# Motivation
Currently, ORT minimal builds use kernel def hashes to map from nodes to
kernels to execute when loading the model. As the kernel def hashes must
be known ahead of time, this works for statically registered kernels.
This works well for the CPU EP.
For this approach to work, the kernel def hashes must also be known at
ORT format model conversion time, which means the EP with statically
registered kernels must also be enabled then. This is not an issue for
the always-available CPU EP. However, we do not want to require that any
EP which statically registers kernels is always available too.
Consequently, we explore another approach to match nodes to kernels that
does not rely on kernel def hashes. An added benefit of this is the
possibility of moving away from kernel def hashes completely, which
would eliminate the maintenance burden of keeping the hashes stable.

# Approach
In a full build, ORT uses some information from the ONNX op schema to
match a node to a kernel. We want to avoid including the ONNX op schema
in a minimal build to reduce binary size. Essentially, we take the
necessary information from the ONNX op schema and make it available in a
minimal build.
We decouple the ONNX op schema from the kernel matching logic. The
kernel matching logic instead relies on per-op information which can
either be obtained from the ONNX op schema or another source.
This per-op information must be available in a minimal build when there
are no ONNX op schemas. We put it in the ORT format model.
Existing uses of kernel def hashes to look up kernels are replaced
with the updated kernel matching logic. We no longer store
kernel def hashes in the ORT format model’s session state and runtime
optimization representations. We no longer keep the logic to
generate and ensure stability of kernel def hashes.
2022-09-20 14:24:59 -07:00
Yulong Wang
bd973bcf1e
[js/rn] upgrade dependencies for e2e test (#11863)
* [js/rn] upgrade dependencies for e2e test

* use JDK11 only for gradle

* expand variable
2022-06-27 14:56:49 -07:00
Yulong Wang
3437967e63
[js/rn] fix CI packaging for react native E2E test (#11463)
* [js/rn] fix ORTRN packaging in CI

* fix env var setting
2022-05-09 18:09:52 -07:00
Sunghoon
740870a285
[js] Create npm packaging pipeline (#10886)
* create npm packaging pipeline

* fix indentations

* Update npm-packaging-pipeline.yml for Azure Pipelines

* Update npm-packaging-pipeline.yml for Azure Pipelines

* Update npm-packaging-pipeline.yml for Azure Pipelines

* react-native-ci as a template

* fix typos

* fix template paths

* add a depencendy

* change a stage name

* set different artifact name for each package

* fix typo

* Update npm-packaging-pipeline.yml for Azure Pipelines

Set a build Id for node npm package as a parameter

* Update npm-packaging-pipeline.yml for Azure Pipelines

Set a build Id for node npm package as a parameter

* Update npm-packaging-pipeline.yml for Azure Pipelines
2022-03-17 16:44:00 -07:00