saymrwulf/pytorch
1
0
Fork 0
mirror of https://github.com/saymrwulf/pytorch.git synced 2026-05-14 20:57:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
pytorch/tools/stats
History
Michael Suo 5029a91f7b [ci] delete JOB_BASE_NAME (#80046) 
`JOB_BASE_NAME` was a holdover from jenkins compatibility. Eventually,
it morphed to be always set to the build enviroment + `-test` or
`-build`, and we used it to detect whether we were in a build or test.

That's sort of pointless, so removing and fixing up the few remaining
use cases.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/80046
Approved by: https://github.com/malfet, https://github.com/janeyx99
2022-06-23 21:06:48 +00:00
..
__init__.py
export_slow_tests.py Run Black on all of tools/ 2022-04-20 17:29:41 +00:00
import_test_stats.py [disable bot] Handle empty bodies (#77078) 2022-05-09 14:21:56 +00:00
print_test_stats.py [ci] delete JOB_BASE_NAME (#80046) 2022-06-23 21:06:48 +00:00
README.md [ci/docs] add some documentation about the stats uploading process 2022-06-14 21:35:04 +00:00
s3_stat_parser.py [ci] clean up dead code related to PR test selection 2022-06-13 16:09:51 +00:00
sccache_stats_to_json.py [ci] move sccache stats off RDS 2022-06-13 16:09:53 +00:00
scribe.py Revert "Revert "[ci] remove remaining RDS dependency"" 2022-06-15 10:04:13 -07:00
test_history.py Run Black on all of tools/ 2022-04-20 17:29:41 +00:00
upload_sccache_stats.py [ci] move sccache stats off RDS 2022-06-13 16:09:53 +00:00
upload_stats_lib.py Revert "Revert "[ci] convert empty s3 artifacts from error to warning"" 2022-06-15 10:04:23 -07:00
upload_test_stats.py [ci] Pull out generic upload_test_stats functionality 2022-06-13 16:09:53 +00:00

README.md

PyTorch CI Stats

We track various stats about each CI job.

  1. Jobs upload their artifacts to an intermediate data store (either GitHub Actions artifacts or S3, depending on what permissions the job has). Example: a9f6a35a33/.github/workflows/_linux-build.yml (L144-L151)
  2. When a workflow completes, a workflow_run event triggers upload-test-stats.yml.
  3. upload-test-stats downloads the raw stats from the intermediate data store and uploads them as JSON to Rockset, our metrics backend.
graph LR
    J1[Job with AWS creds<br>e.g. linux, win] --raw stats--> S3[(AWS S3)]
    J2[Job w/o AWS creds<br>e.g. mac] --raw stats--> GHA[(GH artifacts)]

    S3 --> uts[upload-test-stats.yml]
    GHA --> uts

    uts --json--> R[(Rockset)]

Why this weird indirection? Because writing to Rockset requires special permissions which, for security reasons, we do not want to give to pull request CI. Instead, we implemented GitHub's recommended pattern for cases like this.

For more details about what stats we export, check out upload-test-stats.yml