saymrwulf/pytorch
1
0
Fork 0
mirror of https://github.com/saymrwulf/pytorch.git synced 2026-05-14 20:57:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
pytorch/docker.Makefile

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

118 lines
3.7 KiB
Text
Raw Normal View History

ci: Add workflow to build official docker images with multiarch (#83437) Resolves https://github.com/pytorch/pytorch/issues/80764 Signed-off-by: Eli Uriegas <seemethere101@gmail.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/83437 Approved by: https://github.com/ZainRizvi, https://github.com/malfet
2022-08-16 20:07:16 +00:00
DOCKER_REGISTRY ?= docker.io
DOCKER_ORG ?= $(shell docker info 2>/dev/null | sed '/Username:/!d;s/.* //')
DOCKER_IMAGE ?= pytorch
[releng] Docker release Refactor Push nightly tags step. Move cuda and cudnn version to docker tag rather then name (#116097) Follow up after : https://github.com/pytorch/pytorch/pull/116070 This PR does 2 things. 1. Refactor Push nightly tags step, don't need to extract CUDA_VERSION anymore. New tag should be in this format: ``${PYTORCH_VERSION}-cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION)-runtime`` 2. Move cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION) from docker name to tag Pull Request resolved: https://github.com/pytorch/pytorch/pull/116097 Approved by: https://github.com/jeanschmidt
2023-12-19 13:53:08 +00:00
DOCKER_FULL_NAME = $(DOCKER_REGISTRY)/$(DOCKER_ORG)/$(DOCKER_IMAGE)
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
ifeq ("$(DOCKER_ORG)","")
$(warning WARNING: No docker user found using results from whoami)
docker: Add make variable to add docker build args (#48942) Summary: Adds an extra make variable 'EXTRA_DOCKER_BUILD_FLAGS' that allows us to add extra docker build flags to the docker build command. Example: make -f docker.Makefile EXTRA_DOCKER_BUILD_FLAGS=--no-cache devel-image Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/48942 Reviewed By: walterddr Differential Revision: D25376288 Pulled By: seemethere fbshipit-source-id: 9cf2c2a5e01d505fa54447604ecd653dcbdd42e1
2020-12-08 04:14:02 +00:00
DOCKER_ORG = $(shell whoami)
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
endif
[releng] Docker release Refactor Push nightly tags step. Move cuda and cudnn version to docker tag rather then name (#116097) Follow up after : https://github.com/pytorch/pytorch/pull/116070 This PR does 2 things. 1. Refactor Push nightly tags step, don't need to extract CUDA_VERSION anymore. New tag should be in this format: ``${PYTORCH_VERSION}-cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION)-runtime`` 2. Move cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION) from docker name to tag Pull Request resolved: https://github.com/pytorch/pytorch/pull/116097 Approved by: https://github.com/jeanschmidt
2023-12-19 13:53:08 +00:00
CUDA_VERSION_SHORT ?= 12.1
CUDA_VERSION ?= 12.1.1
CUDNN_VERSION ?= 8
Move release docker container builds to ubuntu22.04 (#113032) Move Official Docker builds for the release to : nvidia/cuda:12.1.1-cudnn8-devel-ubuntu22.04 Pull Request resolved: https://github.com/pytorch/pytorch/pull/113032 Approved by: https://github.com/malfet, https://github.com/huydhn
2023-11-06 17:33:40 +00:00
BASE_RUNTIME = ubuntu:22.04
Use nvidia/cuda:CUDA_VERSION-devel-ubuntu22.04 as base for official Docker release (#125770) We don't need to install cudnn system wide. We already install it with pytorch install during this step: ``` [conda-installs 2/3] RUN case linux/amd64 in "linux/arm64") pip install --extra-index-url https://download.pytorch.org/whl/cpu/ torch torchvision torchaudio ;; *) /opt/conda/bin/conda install -c "pytorch-nightly" -c "nvidia" -y "python=3.10" pytorch torchvision torchaudio "pytorch-cuda=$(echo 12.1.1 | cut -d'.' -f 1-2)" ;; esac && /opt/conda/bin/conda clean -ya ``` Ref: https://github.com/pytorch/pytorch/actions/runs/8998055687/job/24717424912 Validate via: https://github.com/pytorch/builder/actions/workflows/validate_docker_images.yml Pull Request resolved: https://github.com/pytorch/pytorch/pull/125770 Approved by: https://github.com/nWEIdia, https://github.com/seemethere
2024-05-08 18:41:30 +00:00
BASE_DEVEL = nvidia/cuda:$(CUDA_VERSION)-devel-ubuntu22.04
Allow cmake vars in docker build (#100867) Fixes #100866 Pull Request resolved: https://github.com/pytorch/pytorch/pull/100867 Approved by: https://github.com/malfet
2023-05-10 17:44:26 +00:00
CMAKE_VARS ?=
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
Use CUDA 11.2 for nightly docker build. (#51990) Summary: Set CUDA_VERSION to 11.2.0 since Nvidia name their docker image on Ubuntu 18.04 to be nvidia/cuda:11.2.0-cudnn8-devel-ubuntu18.04. Note that cudatoolkit 11.2.0 is not yet on [conda](https://repo.anaconda.com/pkgs/main/linux-64/), and we need to wait for that before merging this PR. - https://hub.docker.com/r/nvidia/cuda/ Pull Request resolved: https://github.com/pytorch/pytorch/pull/51990 Reviewed By: samestep Differential Revision: D26371193 Pulled By: xuzhao9 fbshipit-source-id: 76915490dc30ddb03ceeeadb3c45a6c02b60401e
2021-02-10 18:43:22 +00:00
# The conda channel to use to install cudatoolkit
Use the conda channel defined in docker.Makefile to install cudatoolkit (#53316) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/53316 Test Plan: Nightly Docker build CI This is a follow-up PR after docker moved default CUDA => 11.1. Only merge this after https://github.com/pytorch/pytorch/issues/53299 is committed. Reviewed By: albanD Differential Revision: D26996287 Pulled By: xuzhao9 fbshipit-source-id: 0c2e03da41d036d7aada3e07d479a3dede219f58
2021-03-12 19:13:49 +00:00
CUDA_CHANNEL = nvidia
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
# The conda channel to use to install pytorch / torchvision
ci: Add workflow to build official docker images with multiarch (#83437) Resolves https://github.com/pytorch/pytorch/issues/80764 Signed-off-by: Eli Uriegas <seemethere101@gmail.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/83437 Approved by: https://github.com/ZainRizvi, https://github.com/malfet
2022-08-16 20:07:16 +00:00
INSTALL_CHANNEL ?= pytorch
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
Increase Python version for Docker builds (#125782) Fixes https://github.com/pytorch/pytorch/issues/73714 Pull Request resolved: https://github.com/pytorch/pytorch/pull/125782 Approved by: https://github.com/huydhn
2024-05-08 20:32:32 +00:00
PYTHON_VERSION ?= 3.11
docker: Only match tags that start with v* (#120670) To avoid issues where version could be confused with a ciflow tag. Example: ``` ❯ git describe --tags --always ciflow/periodic/c3496d50f0bb437c70f27085f71155209277bfd4-47-g4ca24959d1a ❯ git describe --tags --always --match "v[1-9]*.*" v1.8.0-rc1-36500-g4ca24959d1a ``` Resolves https://github.com/pytorch/pytorch/issues/120392 Signed-off-by: Eli Uriegas <eliuriegas@meta.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/120670 Approved by: https://github.com/kit1980, https://github.com/atalman
2024-02-26 23:49:54 +00:00
# Match versions that start with v followed by a number, to avoid matching with tags like ciflow
PYTORCH_VERSION ?= $(shell git describe --tags --always --match "v[1-9]*.*")
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
# Can be either official / dev
ci: Add workflow to build official docker images with multiarch (#83437) Resolves https://github.com/pytorch/pytorch/issues/80764 Signed-off-by: Eli Uriegas <seemethere101@gmail.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/83437 Approved by: https://github.com/ZainRizvi, https://github.com/malfet
2022-08-16 20:07:16 +00:00
BUILD_TYPE ?= dev
BUILD_PROGRESS ?= auto
Fix Docker image generation (#88741) Pass install channel when building nightly images Pass `TRITON_VERSION` argument to install triton for nightly images Fix `generate_pytorch_version.py` to work with unannotated tags and avoid failures like the following: ``` % git checkout nightly % ./.github/scripts/generate_pytorch_version.py fatal: No annotated tags can describe '93f15b1b54ca5fb4a7ca9c21a813b4b86ebaeafa'. However, there were unannotated tags: try --tags. Traceback (most recent call last): File "/Users/nshulga/git/pytorch/pytorch-release/./.github/scripts/generate_pytorch_version.py", line 120, in <module> main() File "/Users/nshulga/git/pytorch/pytorch-release/./.github/scripts/generate_pytorch_version.py", line 115, in main print(version_obj.get_release_version()) File "/Users/nshulga/git/pytorch/pytorch-release/./.github/scripts/generate_pytorch_version.py", line 75, in get_release_version if not get_tag(): File "/Users/nshulga/git/pytorch/pytorch-release/./.github/scripts/generate_pytorch_version.py", line 37, in get_tag dirty_tag = subprocess.check_output( File "/Users/nshulga/miniforge3/lib/python3.9/subprocess.py", line 424, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/Users/nshulga/miniforge3/lib/python3.9/subprocess.py", line 528, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['git', 'describe']' returned non-zero exit status 128. ``` After the change nightly is reported as(due to autolabelling issue, should be fixed by ttps://github.com/pytorch/test-infra/pull/1047 ): ``` % ./.github/scripts/generate_pytorch_version.py ciflow/inductor/26921+cpu ``` Even for tagged release commits version generation was wrong: ``` % git checkout release/1.13 % ./.github/scripts/generate_pytorch_version.py ciflow/periodic/79617-4848-g7c98e70d44+cpu ``` After the fix, it is as expected: ``` % ./.github/scripts/generate_pytorch_version.py 1.13.0+cpu ``` Pull Request resolved: https://github.com/pytorch/pytorch/pull/88741 Approved by: https://github.com/dagitses, https://github.com/msaroufim
2022-11-10 00:06:31 +00:00
# Intentionally left blank
TRITON_VERSION ?=
docker: Add make variable to add docker build args (#48942) Summary: Adds an extra make variable 'EXTRA_DOCKER_BUILD_FLAGS' that allows us to add extra docker build flags to the docker build command. Example: make -f docker.Makefile EXTRA_DOCKER_BUILD_FLAGS=--no-cache devel-image Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/48942 Reviewed By: walterddr Differential Revision: D25376288 Pulled By: seemethere fbshipit-source-id: 9cf2c2a5e01d505fa54447604ecd653dcbdd42e1
2020-12-08 04:14:02 +00:00
BUILD_ARGS = --build-arg BASE_IMAGE=$(BASE_IMAGE) \
--build-arg PYTHON_VERSION=$(PYTHON_VERSION) \
--build-arg CUDA_VERSION=$(CUDA_VERSION) \
Use CUDA 11.2 for nightly docker build. (#51990) Summary: Set CUDA_VERSION to 11.2.0 since Nvidia name their docker image on Ubuntu 18.04 to be nvidia/cuda:11.2.0-cudnn8-devel-ubuntu18.04. Note that cudatoolkit 11.2.0 is not yet on [conda](https://repo.anaconda.com/pkgs/main/linux-64/), and we need to wait for that before merging this PR. - https://hub.docker.com/r/nvidia/cuda/ Pull Request resolved: https://github.com/pytorch/pytorch/pull/51990 Reviewed By: samestep Differential Revision: D26371193 Pulled By: xuzhao9 fbshipit-source-id: 76915490dc30ddb03ceeeadb3c45a6c02b60401e
2021-02-10 18:43:22 +00:00
--build-arg CUDA_CHANNEL=$(CUDA_CHANNEL) \
docker: add environment variable PYTORCH_VERSION (#50154) Summary: The aim is being able to inspect a container image and determine immediately which version of pytorch it contains. Closes https://github.com/pytorch/pytorch/issues/48324 Signed-off-by: Felix Abecassis <fabecassis@nvidia.com> seemethere PTAL. As you requested in https://github.com/pytorch/pytorch/issues/48324#issuecomment-754237156, I'm submitting the patch. But I could only do limited testing as I'm not sure these Makefile/Dockerfile are used for pushing the Docker Hub images (since the Makefile tags the image with a `v` prefix for the version, as in: `pytorch:v1.7.1`, but Docker Hub images don't have this prefix). Also on the master branch we currently have the following: ``` $ git describe --tags v1.4.0a0-11171-g68a6e46379 ``` So it's a little off, but it behaves as expected on the `release/1.7` branch. Pull Request resolved: https://github.com/pytorch/pytorch/pull/50154 Reviewed By: walterddr Differential Revision: D25828491 Pulled By: seemethere fbshipit-source-id: 500ec96cb5f5da1321610002d5e3678f4b0b94b5
2021-01-07 22:11:01 +00:00
--build-arg PYTORCH_VERSION=$(PYTORCH_VERSION) \
Fix Docker image generation (#88741) Pass install channel when building nightly images Pass `TRITON_VERSION` argument to install triton for nightly images Fix `generate_pytorch_version.py` to work with unannotated tags and avoid failures like the following: ``` % git checkout nightly % ./.github/scripts/generate_pytorch_version.py fatal: No annotated tags can describe '93f15b1b54ca5fb4a7ca9c21a813b4b86ebaeafa'. However, there were unannotated tags: try --tags. Traceback (most recent call last): File "/Users/nshulga/git/pytorch/pytorch-release/./.github/scripts/generate_pytorch_version.py", line 120, in <module> main() File "/Users/nshulga/git/pytorch/pytorch-release/./.github/scripts/generate_pytorch_version.py", line 115, in main print(version_obj.get_release_version()) File "/Users/nshulga/git/pytorch/pytorch-release/./.github/scripts/generate_pytorch_version.py", line 75, in get_release_version if not get_tag(): File "/Users/nshulga/git/pytorch/pytorch-release/./.github/scripts/generate_pytorch_version.py", line 37, in get_tag dirty_tag = subprocess.check_output( File "/Users/nshulga/miniforge3/lib/python3.9/subprocess.py", line 424, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/Users/nshulga/miniforge3/lib/python3.9/subprocess.py", line 528, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['git', 'describe']' returned non-zero exit status 128. ``` After the change nightly is reported as(due to autolabelling issue, should be fixed by ttps://github.com/pytorch/test-infra/pull/1047 ): ``` % ./.github/scripts/generate_pytorch_version.py ciflow/inductor/26921+cpu ``` Even for tagged release commits version generation was wrong: ``` % git checkout release/1.13 % ./.github/scripts/generate_pytorch_version.py ciflow/periodic/79617-4848-g7c98e70d44+cpu ``` After the fix, it is as expected: ``` % ./.github/scripts/generate_pytorch_version.py 1.13.0+cpu ``` Pull Request resolved: https://github.com/pytorch/pytorch/pull/88741 Approved by: https://github.com/dagitses, https://github.com/msaroufim
2022-11-10 00:06:31 +00:00
--build-arg INSTALL_CHANNEL=$(INSTALL_CHANNEL) \
Allow cmake vars in docker build (#100867) Fixes #100866 Pull Request resolved: https://github.com/pytorch/pytorch/pull/100867 Approved by: https://github.com/malfet
2023-05-10 17:44:26 +00:00
--build-arg TRITON_VERSION=$(TRITON_VERSION) \
--build-arg CMAKE_VARS="$(CMAKE_VARS)"
docker: Add make variable to add docker build args (#48942) Summary: Adds an extra make variable 'EXTRA_DOCKER_BUILD_FLAGS' that allows us to add extra docker build flags to the docker build command. Example: make -f docker.Makefile EXTRA_DOCKER_BUILD_FLAGS=--no-cache devel-image Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/48942 Reviewed By: walterddr Differential Revision: D25376288 Pulled By: seemethere fbshipit-source-id: 9cf2c2a5e01d505fa54447604ecd653dcbdd42e1
2020-12-08 04:14:02 +00:00
EXTRA_DOCKER_BUILD_FLAGS ?=
ci: Add workflow to build official docker images with multiarch (#83437) Resolves https://github.com/pytorch/pytorch/issues/80764 Signed-off-by: Eli Uriegas <seemethere101@gmail.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/83437 Approved by: https://github.com/ZainRizvi, https://github.com/malfet
2022-08-16 20:07:16 +00:00
BUILD ?= build
# Intentionally left blank
PLATFORMS_FLAG ?=
PUSH_FLAG ?=
USE_BUILDX ?=
BUILD_PLATFORMS ?=
WITH_PUSH ?= false
# Setup buildx flags
ifneq ("$(USE_BUILDX)","")
BUILD = buildx build
ifneq ("$(BUILD_PLATFORMS)","")
PLATFORMS_FLAG = --platform="$(BUILD_PLATFORMS)"
endif
# Only set platforms flags if using buildx
ifeq ("$(WITH_PUSH)","true")
PUSH_FLAG = --push
endif
endif
Dockerfile should set the syntax directive to v1 (#125632) Fixes #125526 [#1811](https://github.com/pytorch/builder/issues/1811) Adopt syntax=docker/dockerfile:1 whcih has been stable since 2018, while still best practice to declare in 2024. - Syntax features dependent upon the [syntax directive version are documented here](https://hub.docker.com/r/docker/dockerfile). - While you can set a fixed minor version, [Docker officially advises to only pin the major version] ``` (https://docs.docker.com/build/dockerfile/frontend/#stable-channel): We recommend using docker/dockerfile:1, which always points to the latest stable release of the version 1 syntax, and receives both "minor" and "patch" updates for the version 1 release cycle. BuildKit automatically checks for updates of the syntax when performing a build, making sure you are using the most current version. ``` **Support for building with Docker prior to v23 (released on Feb 2023)** NOTE: 18.06 may not be the accurate minimum version for using docker/dockerfile:1, according to the [DockerHub tag history](https://hub.docker.com/layers/docker/dockerfile/1.0/images/sha256-92f5351b2fca8f7e2f452aa9aec1c34213cdd2702ca92414eee6466fab21814a?context=explore) 1.0 of the syntax seems to be from Dec 2018, which is probably why docker/dockerfile:experimental was paired with it in this file. Personally, I'd favor only supporting builds with Docker v23. This is only relevant for someone building this Dockerfile locally, the user could still extend the already built and published image from a registry on older versions of Docker without any concern for this directive which only applies to building this Dockerfile, not images that extend it. However if you're reluctant, you may want to refer others to [this Docker docs page](https://docs.docker.com/build/buildkit/#getting-started) where they should only need the ENV DOCKER_BUILDKIT=1, presumably the requirement for experimental was dropped with syntax=docker/dockerfile:1 with releases of Docker since Dec 2018. Affected users can often quite easily install a newer version of Docker on their OS, as per Dockers official guidance (usually via including an additional repo to the package manager). **Reference links** Since one of these was already included in the inline note (now a broken link), I've included relevant links mentioned above. You could alternatively rely on git blame with a commit message referencing the links or this PR for more information. Feel free to remove any of the reference links, they're mostly only relevant to maintainers to be aware of (which this PR itself has detailed adequately above). Pull Request resolved: https://github.com/pytorch/pytorch/pull/125632 Approved by: https://github.com/malfet
2024-05-08 01:52:53 +00:00
DOCKER_BUILD = docker $(BUILD) \
docker: Add make variable to add docker build args (#48942) Summary: Adds an extra make variable 'EXTRA_DOCKER_BUILD_FLAGS' that allows us to add extra docker build flags to the docker build command. Example: make -f docker.Makefile EXTRA_DOCKER_BUILD_FLAGS=--no-cache devel-image Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/48942 Reviewed By: walterddr Differential Revision: D25376288 Pulled By: seemethere fbshipit-source-id: 9cf2c2a5e01d505fa54447604ecd653dcbdd42e1
2020-12-08 04:14:02 +00:00
--progress=$(BUILD_PROGRESS) \
$(EXTRA_DOCKER_BUILD_FLAGS) \
ci: Add workflow to build official docker images with multiarch (#83437) Resolves https://github.com/pytorch/pytorch/issues/80764 Signed-off-by: Eli Uriegas <seemethere101@gmail.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/83437 Approved by: https://github.com/ZainRizvi, https://github.com/malfet
2022-08-16 20:07:16 +00:00
$(PLATFORMS_FLAG) \
$(PUSH_FLAG) \
docker: Add make variable to add docker build args (#48942) Summary: Adds an extra make variable 'EXTRA_DOCKER_BUILD_FLAGS' that allows us to add extra docker build flags to the docker build command. Example: make -f docker.Makefile EXTRA_DOCKER_BUILD_FLAGS=--no-cache devel-image Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/48942 Reviewed By: walterddr Differential Revision: D25376288 Pulled By: seemethere fbshipit-source-id: 9cf2c2a5e01d505fa54447604ecd653dcbdd42e1
2020-12-08 04:14:02 +00:00
--target $(BUILD_TYPE) \
-t $(DOCKER_FULL_NAME):$(DOCKER_TAG) \
$(BUILD_ARGS) .
DOCKER_PUSH = docker push $(DOCKER_FULL_NAME):$(DOCKER_TAG)
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
.PHONY: all
all: devel-image
.PHONY: devel-image
devel-image: BASE_IMAGE := $(BASE_DEVEL)
[releng] Docker release Refactor Push nightly tags step. Move cuda and cudnn version to docker tag rather then name (#116097) Follow up after : https://github.com/pytorch/pytorch/pull/116070 This PR does 2 things. 1. Refactor Push nightly tags step, don't need to extract CUDA_VERSION anymore. New tag should be in this format: ``${PYTORCH_VERSION}-cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION)-runtime`` 2. Move cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION) from docker name to tag Pull Request resolved: https://github.com/pytorch/pytorch/pull/116097 Approved by: https://github.com/jeanschmidt
2023-12-19 13:53:08 +00:00
devel-image: DOCKER_TAG := $(PYTORCH_VERSION)-cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION)-devel
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
devel-image:
$(DOCKER_BUILD)
docker.Makefile: fix phony targets (#82941) ### Description Updated phony targets in `docker.Makefile` to refer to the `make` target called directly after `.PHONY` declarations. This was probably the author's intention. ### Issue None - this is a simple fix. ### Testing * Ran `make -f docker.Makefile devel-push` * Ran `make -f docker.Makefile runtime-push` Pull Request resolved: https://github.com/pytorch/pytorch/pull/82941 Approved by: https://github.com/kit1980
2022-08-10 02:40:34 +00:00
.PHONY: devel-push
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
devel-push: BASE_IMAGE := $(BASE_DEVEL)
[releng] Docker release Refactor Push nightly tags step. Move cuda and cudnn version to docker tag rather then name (#116097) Follow up after : https://github.com/pytorch/pytorch/pull/116070 This PR does 2 things. 1. Refactor Push nightly tags step, don't need to extract CUDA_VERSION anymore. New tag should be in this format: ``${PYTORCH_VERSION}-cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION)-runtime`` 2. Move cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION) from docker name to tag Pull Request resolved: https://github.com/pytorch/pytorch/pull/116097 Approved by: https://github.com/jeanschmidt
2023-12-19 13:53:08 +00:00
devel-push: DOCKER_TAG := $(PYTORCH_VERSION)-cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION)-devel
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
devel-push:
$(DOCKER_PUSH)
Separate arm64 and amd64 docker builds (#125617) Fixes https://github.com/pytorch/pytorch/issues/125094 Please note: Docker CUDa 12.4 failure is existing issue, related to docker image not being available on gitlab: ``` docker.io/nvidia/cuda:12.4.0-cudnn8-devel-ubuntu22.04: docker.io/nvidia/cuda:12.4.0-cudnn8-devel-ubuntu22.04: not found ``` https://github.com/pytorch/pytorch/actions/runs/8974959068/job/24648540236?pr=125617 Here is the reference issue: https://gitlab.com/nvidia/container-images/cuda/-/issues/225 Tracked on our side: https://github.com/pytorch/builder/issues/1811 Pull Request resolved: https://github.com/pytorch/pytorch/pull/125617 Approved by: https://github.com/huydhn, https://github.com/malfet
2024-05-07 11:50:54 +00:00
ifeq ("$(CUDA_VERSION_SHORT)","cpu")
.PHONY: runtime-image
runtime-image: BASE_IMAGE := $(BASE_RUNTIME)
runtime-image: DOCKER_TAG := $(PYTORCH_VERSION)-runtime
runtime-image:
$(DOCKER_BUILD)
.PHONY: runtime-push
runtime-push: BASE_IMAGE := $(BASE_RUNTIME)
runtime-push: DOCKER_TAG := $(PYTORCH_VERSION)-runtime
runtime-push:
$(DOCKER_PUSH)
else
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
.PHONY: runtime-image
runtime-image: BASE_IMAGE := $(BASE_RUNTIME)
[releng] Docker release Refactor Push nightly tags step. Move cuda and cudnn version to docker tag rather then name (#116097) Follow up after : https://github.com/pytorch/pytorch/pull/116070 This PR does 2 things. 1. Refactor Push nightly tags step, don't need to extract CUDA_VERSION anymore. New tag should be in this format: ``${PYTORCH_VERSION}-cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION)-runtime`` 2. Move cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION) from docker name to tag Pull Request resolved: https://github.com/pytorch/pytorch/pull/116097 Approved by: https://github.com/jeanschmidt
2023-12-19 13:53:08 +00:00
runtime-image: DOCKER_TAG := $(PYTORCH_VERSION)-cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION)-runtime
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
runtime-image:
$(DOCKER_BUILD)
docker.Makefile: fix phony targets (#82941) ### Description Updated phony targets in `docker.Makefile` to refer to the `make` target called directly after `.PHONY` declarations. This was probably the author's intention. ### Issue None - this is a simple fix. ### Testing * Ran `make -f docker.Makefile devel-push` * Ran `make -f docker.Makefile runtime-push` Pull Request resolved: https://github.com/pytorch/pytorch/pull/82941 Approved by: https://github.com/kit1980
2022-08-10 02:40:34 +00:00
.PHONY: runtime-push
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
runtime-push: BASE_IMAGE := $(BASE_RUNTIME)
[releng] Docker release Refactor Push nightly tags step. Move cuda and cudnn version to docker tag rather then name (#116097) Follow up after : https://github.com/pytorch/pytorch/pull/116070 This PR does 2 things. 1. Refactor Push nightly tags step, don't need to extract CUDA_VERSION anymore. New tag should be in this format: ``${PYTORCH_VERSION}-cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION)-runtime`` 2. Move cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION) from docker name to tag Pull Request resolved: https://github.com/pytorch/pytorch/pull/116097 Approved by: https://github.com/jeanschmidt
2023-12-19 13:53:08 +00:00
runtime-push: DOCKER_TAG := $(PYTORCH_VERSION)-cuda$(CUDA_VERSION_SHORT)-cudnn$(CUDNN_VERSION)-runtime
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
runtime-push:
$(DOCKER_PUSH)
Separate arm64 and amd64 docker builds (#125617) Fixes https://github.com/pytorch/pytorch/issues/125094 Please note: Docker CUDa 12.4 failure is existing issue, related to docker image not being available on gitlab: ``` docker.io/nvidia/cuda:12.4.0-cudnn8-devel-ubuntu22.04: docker.io/nvidia/cuda:12.4.0-cudnn8-devel-ubuntu22.04: not found ``` https://github.com/pytorch/pytorch/actions/runs/8974959068/job/24648540236?pr=125617 Here is the reference issue: https://gitlab.com/nvidia/container-images/cuda/-/issues/225 Tracked on our side: https://github.com/pytorch/builder/issues/1811 Pull Request resolved: https://github.com/pytorch/pytorch/pull/125617 Approved by: https://github.com/huydhn, https://github.com/malfet
2024-05-07 11:50:54 +00:00
endif
docker: Refactor Dockerfile process for official images (#32515) Summary: ## Commit Message: Refactors Dockerfile to be as parallel as possible with caching and adds a new Makefile to build said Dockerfile. Also updated the README.md to reflect the changes as well as updated some of the verbage around running our latest Docker images. Adds the new Dockerfile process to our CircleCI workflows ## How to build: Building the new images is pretty simple, just requires `docker` > 18.06 since the new build process relies on `buildkit` caching and multi-stage build resolving. ### Development images For `runtime` images: ``` make -f docker.Makefile runtime-image ``` For `devel` images: ``` make -f docker.Makefile devel-image ``` Builds are tagged as follows: ```bash docker.io/${docker_user:-whoami}/pytorch:$(git describe --tags)-${image_type} ``` Example: ``` docker.io/seemethere/pytorch:v1.4.0a0-2225-g9eba97b61d-runtime ``` ### Official images Official images are the ones hosted on [`docker.io/pytorch/pytorch`](https://hub.docker.com/r/pytorch/pytorch) To do official images builds you can simply add set the `BUILD_TYPE` variable to `official` and it will do the correct build without building the local binaries: Example: ``` make -f docker.Makefile BUILD_TYPE=official runtime-image ``` ## How to push: Pushing is also super simple (And will automatically tag the right thing based off of the git tag): ``` make -f docker.Makefile runtime-push make -f docker.Makefile devel-push ``` Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/32515 Differential Revision: D19558619 Pulled By: seemethere fbshipit-source-id: a06b25cd39ae9890751a60f8f36739ad6ab9ac99
2020-01-24 18:24:46 +00:00
.PHONY: clean
clean:
-docker rmi -f $(shell docker images -q $(DOCKER_FULL_NAME))
Reference in a new issue Copy permalink