onnxruntime/tools/ci_build/github/azure-pipelines/templates/react-native-ci.yml
Chi Lo b713855a98
Release 1.11.0 cherry pick round 1 (#10915)
* Update to flatbuffers v2.0.0 (#10866)

* Fix Reduced ops pipeline (#10861)

* Fix a couple of issues with the python package tools (#10858)

* Tweaks to the model utils
  * Add handling for a dim_value of -1 when replacing the entire input shape. This occurs in models exported from PaddlePaddle
  * make pytorch helpers accessible in package
  * make QDQ helpers accessible in package

* Fix wrong percentile values returned during calibration (#10847)

* Use numpy.percentile to get the lookup value.

* Use 1.0 as float value rather than integer.

* Add missing cdf parameter for `np.percentile`.

* Use 100. instead of 1.0

* Remove print.

* Update from @yufenglee

* Add support for opset 16 to transpose optimizer. (#10841)

* Add support for opset 16 to transpose optimizer.

Only change required is for GridSample to be added to the layout sensitive ops. The existing handling for layout transpose works with that as the first input and first output are layout sensitive.

Update the optimize to be able to return an error message if it fails.

* Use separate build directories for full and mobile iOS packages. (#10835)

* Address performance issue with abseil flat_hash_table. (#10819)

When returning by value in a cross DLL call, the hash table
even though containing all the entries that are originally there
can not find at least some of them. Reverting to std::unordered_set
pending further investigation.

* Mark end of version 11 C API. (#10803)

* Mark end of version 11 C API

* Add static_assert

* avoid using LocalFree on FormatMessageW buffer (#10796)

* remove local free

* Remove local free from onnxruntime

* don't allocate

* Change to use constexpr to satisfy  CPU build warning

* Integrate C-API tests into Pipelines for release packages (#10794)

* add c-api test for package

* fix bug for running c-api test for package

* refine run application script

* remove redundant code

* include CUDA test

* Remove testing CUDA EP temporarily

* fix bug

* Code refactor

* try to fix YAML bug

* try to fix YAML bug

* try to fix YAML bug

* fix bug for multiple directories in Pipelines

* fix bug

* add comments and fix bug

* Update c-api-noopenmp-packaging-pipelines.yml

* Remove failOnStandardError flag in Pipelines

* Detect runtime CUDA JIT and warn the user (#10781)

* Use cudaMalloc vs cudaDeviceSynchronize and show the total time

* Update convert_onnx_models_to_ort.py to support runtime optimizations. (#10765)

Add runtime optimization support to ONNX -> ORT format conversion script.
Replace `--optimization_level`, `--use_nnapi`, and `--use_coreml` with a new `--optimization_style` option.

* Add multithreading test and put a lock on nvinfer1::createInferRuntime() for TRT EP (#10714)

* Add multithread unit test and put lock on library call

* update code

* remove debug code

* add comment

* add one session multi-threads inference

* Put lock for build engine all the time

* Update naming and comment

* remove unnecessary lock

* Revert "remove unnecessary lock"

This reverts commit 9c2317b1d2273dec0ebdeb52160bc757839e5edc.

* Fix handling of nodes inserted by NHWC transformer. (#10904) (#10925)

* Revert "Upsample support NHWC (#10554)" (#10917)

This reverts commit bd08f11a58.

Co-authored-by: Yufeng Li <liyufeng1987@gmail.com>

* [python API] Change raise import error when `C:\Windows\System32\vcruntime140_1.dll` is not found to warning (#10927)

* remove throw if C:\\Windows\\System32\\vcruntime140_1.dll cannot be found

* Add comments and update warning message

* adding back accidentally removed line

Co-authored-by: gwang0000 <62914304+gwang0000@users.noreply.github.com>

* [js] Create npm packaging pipeline (#10886)

* create npm packaging pipeline

* fix indentations

* Update npm-packaging-pipeline.yml for Azure Pipelines

* Update npm-packaging-pipeline.yml for Azure Pipelines

* Update npm-packaging-pipeline.yml for Azure Pipelines

* react-native-ci as a template

* fix typos

* fix template paths

* add a depencendy

* change a stage name

* set different artifact name for each package

* fix typo

* Update npm-packaging-pipeline.yml for Azure Pipelines

Set a build Id for node npm package as a parameter

* Update npm-packaging-pipeline.yml for Azure Pipelines

Set a build Id for node npm package as a parameter

* Update npm-packaging-pipeline.yml for Azure Pipelines

* Follow up update for python API checking if `vcruntime140_1.dll` is available (#10927) (#10933)

Co-authored-by: Hariharan Seshadri <hasesh@microsoft.com>
Co-authored-by: Scott McKay <skottmckay@gmail.com>
Co-authored-by: Funtowicz Morgan <mfuntowicz@users.noreply.github.com>
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
Co-authored-by: Dmitri Smirnov <yuslepukhin@users.noreply.github.com>
Co-authored-by: Pranav Sharma <prs@microsoft.com>
Co-authored-by: Ryan Lai <rylai@microsoft.com>
Co-authored-by: Ryan Hill <38674843+RyanUnderhill@users.noreply.github.com>
Co-authored-by: Yi-Hong Lyu <yilyu@microsoft.com>
Co-authored-by: Yufeng Li <liyufeng1987@gmail.com>
Co-authored-by: Guoyu Wang <62914304+gwang-msft@users.noreply.github.com>
Co-authored-by: gwang0000 <62914304+gwang0000@users.noreply.github.com>
Co-authored-by: Sunghoon <35605090+hanbitmyths@users.noreply.github.com>
2022-03-18 11:16:30 -07:00

300 lines
11 KiB
YAML

parameters:
- name: NpmPackagingMode
displayName: 'NPM packages publish configuration'
type: string
default: 'dev'
- name: BuildConfig
displayName: 'Build config'
type: string
values:
- 'Release'
- 'MinSizeRel'
- 'Debug'
- 'RelWithDebugInfo'
default: 'Release'
- name: PoolName
displayName: 'Pool name'
type: string
- name: PackageName
displayName: 'Package name'
type: string
default: 'NPM_packages'
jobs:
- template: android-java-api-aar.yml
parameters:
buildConfig: '${{parameters.BuildConfig}}'
buildSettings: '$(Build.SourcesDirectory)/tools/ci_build/github/js/react_native_e2e_mobile_aar_build_settings.json'
includedOpsConfig: '$(Build.SourcesDirectory)/tools/ci_build/github/android/mobile_package.required_operators.config'
artifactName: 'onnxruntime-android-mobile-aar'
job_name_suffix: 'For_React_Native'
pool_name: '${{parameters.PoolName}}'
packageName: 'onnxruntime-mobile'
- job: ReactNative_CI
pool:
vmImage: 'macOS-11'
variables:
runCodesignValidationInjection: false
dependsOn:
- Android_Java_API_AAR_Packaging_For_React_Native
timeoutInMinutes: 120
steps:
# Onnx has no 3.9 python package available yet, need to use python 3.8 to avoid build onnx package
# pythonVersion can be updated in Azure pipeline settings
# https://dev.azure.com/onnxruntime/onnxruntime/_build?definitionId=188
- task: UsePythonVersion@0
displayName: Use Python $(pythonVersion)
inputs:
versionSpec: $(pythonVersion)
- task: NodeTool@0
inputs:
versionSpec: '16.x'
- script:
brew install coreutils ninja npm yarn
displayName: Install coreutils, ninja, npm, and yarn
- script:
/bin/bash $(Build.SourcesDirectory)/tools/ci_build/github/android/setup_gradle_wrapper.sh $(pwd)
displayName: Setup gradle wrapper to use gradle 6.8.3
- script: |
python3 -m pip install -q flatbuffers
workingDirectory: '$(Build.BinariesDirectory)'
displayName: Install python modules
- script: |
python3 $(Build.SourcesDirectory)/tools/ci_build/github/apple/build_ios_framework.py \
--config ${{parameters.BuildConfig}} \
--build_dir $(Build.BinariesDirectory)/ios_framework \
--include_ops_by_config $(Build.SourcesDirectory)/tools/ci_build/github/android/mobile_package.required_operators.config \
$(Build.SourcesDirectory)/tools/ci_build/github/js/react_native_e2e_mobile_ios_framework_build_settings.json
cd $(Build.BinariesDirectory)/ios_framework/framework_out
zip -r onnxruntime-mobile-c.zip .
displayName: Build iOS package
- task: DownloadPipelineArtifact@2
inputs:
buildType: 'current'
artifactName: 'onnxruntime-android-mobile-aar'
targetPath: '$(Build.BinariesDirectory)/android-mobile-aar'
displayName: Download Android Aar artifacts
- task: CopyFiles@2
inputs:
sourceFolder: $(Build.BinariesDirectory)/android-mobile-aar
contents: onnxruntime-mobile-*.aar
targetFolder: $(Build.SourcesDirectory)/js/react_native/android/libs
displayName: Copy Android package to React Native directory
- task: CopyFiles@2
inputs:
sourceFolder: $(Build.BinariesDirectory)/ios_framework/framework_out
contents: onnxruntime-mobile-c.zip
targetFolder: $(Build.SourcesDirectory)/js/react_native/local_pods
displayName: Copy iOS package to React Native directory
- script: |
npm ci
workingDirectory: '$(Build.SourcesDirectory)/js'
displayName: npm ci js
- script: |
npm ci
workingDirectory: '$(Build.SourcesDirectory)/js/common'
displayName: npm ci js/common
- script: |
yarn
workingDirectory: '$(Build.SourcesDirectory)/js/react_native'
displayName: yarn js/react_native
- script: |
python3 tools/python/run_android_emulator.py \
--android-sdk-root $(ANDROID_SDK_ROOT) \
--create-avd --system-image "system-images;android-30;google_apis;x86_64" \
--start --emulator-extra-args="-partition-size 4096" \
--emulator-pid-file $(Build.BinariesDirectory)/emulator.pid
displayName: Start Android Emulator
- script: |
xcrun simctl create iPhoneRNTest com.apple.CoreSimulator.SimDeviceType.iPhone-13
workingDirectory: '$(Build.SourcesDirectory)/js/react_native/e2e/ios'
displayName: Start iOS Simulator
- task: Gradle@3
inputs:
gradleWrapperFile: '$(Build.SourcesDirectory)/js/react_native/android/gradlew'
workingDirectory: '$(Build.SourcesDirectory)/js/react_native/android'
options: '--stacktrace'
tasks: 'connectedDebugAndroidTest'
publishJUnitResults: true
testResultsFiles: '**/TEST-*.xml'
testRunTitle: 'React Native Android Instrumented Test results'
javaHomeOption: 'JDKVersion'
sonarQubeRunAnalysis: false
spotBugsAnalysis: false
displayName: Run React Native Android Instrumented Tests
continueOnError: false
- script: |
pod install
workingDirectory: '$(Build.SourcesDirectory)/js/react_native/ios'
displayName: Pod install for onnxruntime react native ios bridge library
- task: Xcode@5
inputs:
actions: 'test'
configuration: 'Debug'
sdk: 'iphonesimulator'
xcWorkspacePath: '$(Build.SourcesDirectory)/js/react_native/ios/OnnxruntimeModule.xcworkspace'
scheme: 'OnnxruntimeModuleTest'
packageApp: false
destinationPlatformOption: 'iOS'
destinationSimulators: 'iPhone 13,OS=latest'
workingDirectory: '$(Build.SourcesDirectory)/js/react_native/ios'
xcprettyArgs: '--output build/reports/test-results.xml'
publishJUnitResults: true
testRunTitle: 'React Native iOS Instrumented Test Results'
displayName: Run React Native iOS Instrumented Tests
- task: PublishTestResults@2
inputs:
testResultsFiles: '$(Build.SourcesDirectory)/js/react_native/ios/build/reports/test-results.xml'
failTaskOnFailedTests: true
testRunTitle: 'React Native iOS Instrumented Test results'
condition: succeededOrFailed()
displayName: Publish React Native iOS Instrumented Test Results
- script: |
yarn prepack-e2e
workingDirectory: '$(Build.SourcesDirectory)/js/react_native'
displayName: Prepare Android and iOS e2e tests
- task: PowerShell@2
inputs:
filePath: '$(Build.SourcesDirectory)/tools/ci_build/github/js/pack-npm-packages.ps1'
arguments: '"-dev.$(Get-Date -Format yyyyMMdd)-$(git rev-parse --short HEAD)" $(Build.SourcesDirectory) react_native'
workingDirectory: '$(Build.SourcesDirectory)'
errorActionPreference: stop
displayName: Pack NPM packages
- script: |
mv $(Build.SourcesDirectory)/js/common/onnxruntime-common*.tgz onnxruntime-common.tgz
yarn add --no-lockfile file:./onnxruntime-common.tgz
mv $(Build.SourcesDirectory)/js/react_native/onnxruntime-react-native*.tgz onnxruntime-react-native.tgz
yarn add --no-lockfile file:./onnxruntime-react-native.tgz
yarn
workingDirectory: '$(Build.SourcesDirectory)/js/react_native/e2e'
displayName: Bootstrap Android and iOS e2e tests
- script: |
pod install
workingDirectory: '$(Build.SourcesDirectory)/js/react_native/e2e/ios'
displayName: Pod install for onnxruntime react native ios e2e tests
- script: |
keytool -genkey -v -keystore debug.keystore -alias androiddebugkey -storepass android \
-keypass android -keyalg RSA -keysize 2048 -validity 999999 -dname "CN=Android Debug,O=Android,C=US"
workingDirectory: '$(Build.SourcesDirectory)/js/react_native/e2e/android'
displayName: Generate a debug keystore
- task: CopyFiles@2
inputs:
sourceFolder: $(Build.BinariesDirectory)/android-mobile-aar
contents: onnxruntime-mobile-*.aar
targetFolder: $(Build.SourcesDirectory)/js/react_native/e2e/node_modules/onnxruntime-react-native/android/libs
displayName: Copy Android package to React Native e2e directory
- task: Gradle@3
inputs:
gradleWrapperFile: '$(Build.SourcesDirectory)/js/react_native/e2e/android/gradlew'
workingDirectory: '$(Build.SourcesDirectory)/js/react_native/e2e/android'
options: '--stacktrace'
tasks: ':app:connectedDebugAndroidTest'
publishJUnitResults: true
testResultsFiles: '**/TEST-*.xml'
testRunTitle: 'React Native Android e2e Test results'
javaHomeOption: 'JDKVersion'
sonarQubeRunAnalysis: false
spotBugsAnalysis: false
displayName: Run React Native Android e2e Tests
continueOnError: false
- script: |
export FORCE_BUNDLING=1
export RCT_NO_LAUNCH_PACKAGER=1
export ENTRY_FILE=index.tsx
xcrun xcodebuild test -workspace $(Build.SourcesDirectory)/js/react_native/e2e/ios/OnnxruntimeModuleExample.xcworkspace \
-scheme OnnxruntimeModuleExample -destination 'platform=iOS Simulator,OS=latest,name=iPhoneRNTest' \
-derivedDataPath $(Build.BinariesDirectory)/react_native/ios_e2e_test/derived_data | xcpretty -r junit --no-color \
--output $(Build.SourcesDirectory)/js/react_native/e2e/ios/build/reports/test-results.xml
workingDirectory: '$(Build.SourcesDirectory)/js/react_native/e2e'
displayName: Run React Native iOS e2e tests
- task: PublishTestResults@2
inputs:
testResultsFiles: '$(Build.SourcesDirectory)/js/react_native/e2e/ios/build/reports/test-results.xml'
failTaskOnFailedTests: true
testRunTitle: 'React Native iOS e2e Test results'
condition: succeededOrFailed()
displayName: Publish React Native iOS e2e Test Results
- script: |
python3 tools/python/run_android_emulator.py \
--android-sdk-root $(ANDROID_SDK_ROOT) \
--stop \
--emulator-pid-file $(Build.BinariesDirectory)/emulator.pid
displayName: Stop Android Emulator
condition: always()
- script: |
xcrun simctl delete iPhoneRNTest
workingDirectory: '$(Build.SourcesDirectory)/js/react_native/e2e/ios'
displayName: Stop iOS Simulator
condition: always()
- script: |
git restore .
cd react_native
yarn prepack-rel
workingDirectory: '$(Build.SourcesDirectory)/js'
displayName: Restore git changes and prepack for npm publish
- task: PowerShell@2
inputs:
filePath: '$(Build.SourcesDirectory)/tools/ci_build/github/js/pack-npm-packages.ps1'
arguments: '"${{parameters.NpmPackagingMode}}" $(Build.SourcesDirectory) react_native'
workingDirectory: '$(Build.SourcesDirectory)'
errorActionPreference: stop
displayName: Pack NPM packages
- task: CopyFiles@2
inputs:
sourceFolder: $(Build.SourcesDirectory)/js/common
contents: onnxruntime-common*.tgz
targetFolder: $(Build.ArtifactStagingDirectory)
displayName: 'Create Artifacts onnxruntime-common'
- task: CopyFiles@2
inputs:
sourceFolder: $(Build.SourcesDirectory)/js/react_native
contents: onnxruntime-react-native*.tgz
targetFolder: $(Build.ArtifactStagingDirectory)
displayName: Create Artifacts onnxruntime-react-native
- task: PublishPipelineArtifact@0
inputs:
artifactName: '${{parameters.PackageName}}'
targetPath: '$(Build.ArtifactStagingDirectory)'
displayName: Publish Pipeline Artifact
- template: component-governance-component-detection-steps.yml
parameters :
condition : 'succeeded'
- task: mspremier.PostBuildCleanup.PostBuildCleanup-task.PostBuildCleanup@3
displayName: Clean Agent Directories
condition: always()