mirror of
https://github.com/saymrwulf/pytorch.git
synced 2026-05-14 20:57:59 +00:00
4143312e67
Periodically run testsuite for s390x
**Dependencies update**
Package z3-solver is updated from version 4.12.2.0 to version 4.12.6.0. This is a minor version update, so no functional change is expected.
The reason for update is build on s390x. pypi doesn't provide binary build for z3-solver for versions 4.12.2.0 or 4.12.6.0 for s390x. Unfortunately, version 4.12.2.0 fails to build with newer gcc used on s390x builders, but those errors are fixed in version 4.12.6.0. Due to this minor version bump fixes build on s390x.
```
# pip3 install z3-solver==4.12.2.0
...
In file included from /tmp/pip-install-756iytc6/z3-solver_ce6f750b780b4146a9a7c01e52672071/core/src/util/region.cpp:53:
/tmp/pip-install-756iytc6/z3-solver_ce6f750b780b4146a9a7c01e52672071/core/src/util/region.cpp: In member function ‘void* region::allocate(size_t)’:
/tmp/pip-install-756iytc6/z3-solver_ce6f750b780b4146a9a7c01e52672071/core/src/util/tptr.h:29:62: error: ‘uintptr_t’ does not name a type
29 | #define ALIGN(T, PTR) reinterpret_cast<T>(((reinterpret_cast<uintptr_t>(PTR) >> PTR_ALIGNMENT) + \
| ^~~~~~~~~
/tmp/pip-install-756iytc6/z3-solver_ce6f750b780b4146a9a7c01e52672071/core/src/util/region.cpp:82:22: note: in expansion of macro ‘ALIGN’
82 | m_curr_ptr = ALIGN(char *, new_curr_ptr);
| ^~~~~
/tmp/pip-install-756iytc6/z3-solver_ce6f750b780b4146a9a7c01e52672071/core/src/util/region.cpp:57:1: note: ‘uintptr_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
56 | #include "util/page.h"
+++ |+#include <cstdint>
57 |
```
**Python paths update**
On AlmaLinux 8 s390x, old paths:
```
python -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib())'
/usr/lib/python3.12/site-packages
```
Total result is `/usr/lib/python3.12/site-packages/torch;/usr/lib/python3.12/site-packages`
New paths:
```
python -c 'import site; print(";".join([x for x in site.getsitepackages()] + [x + "/torch" for x in site.getsitepackages()]))'
/usr/local/lib64/python3.12/site-packages;/usr/local/lib/python3.12/site-packages;/usr/lib64/python3.12/site-packages;/usr/lib/python3.12/site-packages;/usr/local/lib64/python3.12/site-packages/torch;/usr/local/lib/python3.12/site-packages/torch;/usr/lib64/python3.12/site-packages/torch;/usr/lib/python3.12/site-packages/torch
```
```
# python -c 'import torch ; print(torch)'
<module 'torch' from '/usr/local/lib64/python3.12/site-packages/torch/__init__.py'>
```
`pip3 install dist/*.whl` installs torch into `/usr/local/lib64/python3.12/site-packages`, and later it's not found by cmake with old paths:
```
CMake Error at CMakeLists.txt:9 (find_package):
By not providing "FindTorch.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "Torch", but
CMake did not find one.
```
https://github.com/pytorch/pytorch/actions/runs/10994060107/job/30521868178?pr=125401
**Builders availability**
Build took 60 minutes
Tests took: 150, 110, 65, 55, 115, 85, 50, 70, 105, 110 minutes (split into 10 shards)
60 + 150 + 110 + 65 + 55 + 115 + 85 + 50 + 70 + 105 + 110 = 975 minutes used. Let's double it. It would be 1950 minutes.
We have 20 machines * 24 hours = 20 * 24 * 60 = 20 * 1440 = 28800 minutes
We currently run 5 nightly binaries builds, each on average 90 minutes build, 15 minutes test, 5 minutes upload, 110 minutes total for each, 550 minutes total. Doubling would be 1100 minutes.
That leaves 28800 - 1100 = 27700 minutes total. Periodic tests would use will leave 25750 minutes.
Nightly binaries build + nightly tests = 3050 minutes.
25750 / 3050 = 8.44. So we could do both 8 more times for additional CI runs for any reason. And that is with pretty good safety margin.
**Skip test_tensorexpr**
On s390x, pytorch is built without llvm.
Even if it would be built with llvm, llvm currently doesn't support used features on s390x and test fails with errors like:
```
JIT session error: Unsupported target machine architecture in ELF object pytorch-jitted-objectbuffer
unknown file: Failure
C++ exception with description "valOrErr INTERNAL ASSERT FAILED at "/var/lib/jenkins/workspace/torch/csrc/jit/tensorexpr/llvm_jit.h":34, please report a bug to PyTorch. Unexpected failure in LLVM JIT: Failed to materialize symbols: { (main, { func }) }
```
**Disable cpp/static_runtime_test on s390x**
Quantization is not fully supported on s390x in pytorch yet.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/125401
Approved by: https://github.com/malfet
Co-authored-by: Nikita Shulga <2453524+malfet@users.noreply.github.com>
77 lines
2.9 KiB
YAML
77 lines
2.9 KiB
YAML
name: s390x-periodic
|
|
|
|
on:
|
|
schedule:
|
|
# We have several schedules so jobs can check github.event.schedule to activate only for a fraction of the runs.
|
|
# Also run less frequently on weekends.
|
|
- cron: 29 8 * * * # about 1:29am PDT, for mem leak check and rerun disabled tests
|
|
push:
|
|
tags:
|
|
- ciflow/periodic/*
|
|
- ciflow/s390/*
|
|
branches:
|
|
- release/*
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref_name }}-${{ github.ref_type == 'branch' && github.sha }}-${{ github.event_name == 'workflow_dispatch' }}-${{ github.event_name == 'schedule' }}-${{ github.event.schedule }}
|
|
cancel-in-progress: true
|
|
|
|
permissions: read-all
|
|
|
|
jobs:
|
|
llm-td:
|
|
if: github.repository_owner == 'pytorch'
|
|
name: before-test
|
|
uses: ./.github/workflows/llm_td_retrieval.yml
|
|
permissions:
|
|
id-token: write
|
|
contents: read
|
|
|
|
target-determination:
|
|
name: before-test
|
|
uses: ./.github/workflows/target_determination.yml
|
|
needs: llm-td
|
|
permissions:
|
|
id-token: write
|
|
contents: read
|
|
|
|
linux-manylinux-2_28-py3-cpu-s390x-build:
|
|
if: github.repository_owner == 'pytorch'
|
|
name: linux-manylinux-2_28-py3-cpu-s390x
|
|
uses: ./.github/workflows/_linux-build.yml
|
|
with:
|
|
build-environment: linux-s390x-binary-manywheel
|
|
docker-image-name: pytorch/manylinuxs390x-builder:cpu-s390x-main
|
|
runner: linux.s390x
|
|
test-matrix: |
|
|
{ include: [
|
|
{ config: "default", shard: 1, num_shards: 10, runner: "linux.s390x" },
|
|
{ config: "default", shard: 2, num_shards: 10, runner: "linux.s390x" },
|
|
{ config: "default", shard: 3, num_shards: 10, runner: "linux.s390x" },
|
|
{ config: "default", shard: 4, num_shards: 10, runner: "linux.s390x" },
|
|
{ config: "default", shard: 5, num_shards: 10, runner: "linux.s390x" },
|
|
{ config: "default", shard: 6, num_shards: 10, runner: "linux.s390x" },
|
|
{ config: "default", shard: 7, num_shards: 10, runner: "linux.s390x" },
|
|
{ config: "default", shard: 8, num_shards: 10, runner: "linux.s390x" },
|
|
{ config: "default", shard: 9, num_shards: 10, runner: "linux.s390x" },
|
|
{ config: "default", shard: 10, num_shards: 10, runner: "linux.s390x" },
|
|
]}
|
|
secrets: inherit
|
|
|
|
linux-manylinux-2_28-py3-cpu-s390x-test:
|
|
permissions:
|
|
id-token: write
|
|
contents: read
|
|
name: linux-manylinux-2_28-py3-cpu-s390x
|
|
uses: ./.github/workflows/_linux-test.yml
|
|
needs:
|
|
- linux-manylinux-2_28-py3-cpu-s390x-build
|
|
- target-determination
|
|
with:
|
|
build-environment: linux-s390x-binary-manywheel
|
|
docker-image: pytorch/manylinuxs390x-builder:cpu-s390x-main
|
|
test-matrix: ${{ needs.linux-manylinux-2_28-py3-cpu-s390x-build.outputs.test-matrix }}
|
|
timeout-minutes: 480
|
|
use-gha: "yes"
|
|
secrets: inherit
|