From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 93F40158099 for ; Fri, 1 Dec 2023 05:53:27 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id DA94C2BC021; Fri, 1 Dec 2023 05:53:26 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id BDC682BC021 for ; Fri, 1 Dec 2023 05:53:26 +0000 (UTC) Received: from oystercatcher.gentoo.org (oystercatcher.gentoo.org [148.251.78.52]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id EFCCE335D12 for ; Fri, 1 Dec 2023 05:53:25 +0000 (UTC) Received: from localhost.localdomain (localhost [IPv6:::1]) by oystercatcher.gentoo.org (Postfix) with ESMTP id 548F3FB7 for ; Fri, 1 Dec 2023 05:53:24 +0000 (UTC) From: "Alfredo Tupone" To: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: 8bit Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Alfredo Tupone" Message-ID: <1701409984.6066231bec8d7a83aff48ff16eb28e44eadd6ef4.tupone@gentoo> Subject: [gentoo-commits] repo/gentoo:master commit in: sci-libs/caffe2/, sci-libs/caffe2/files/ X-VCS-Repository: repo/gentoo X-VCS-Files: sci-libs/caffe2/caffe2-2.0.1-r5.ebuild sci-libs/caffe2/files/caffe2-2.0.1-cudaExtra.patch X-VCS-Directories: sci-libs/caffe2/ sci-libs/caffe2/files/ X-VCS-Committer: tupone X-VCS-Committer-Name: Alfredo Tupone X-VCS-Revision: 6066231bec8d7a83aff48ff16eb28e44eadd6ef4 X-VCS-Branch: master Date: Fri, 1 Dec 2023 05:53:24 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply X-Archives-Salt: c36008cb-546b-4478-97db-9094b68d22be X-Archives-Hash: 6ac8e0da85c089b11e56815cf14591c7 commit: 6066231bec8d7a83aff48ff16eb28e44eadd6ef4 Author: Alfredo Tupone gentoo org> AuthorDate: Fri Dec 1 05:52:18 2023 +0000 Commit: Alfredo Tupone gentoo org> CommitDate: Fri Dec 1 05:53:04 2023 +0000 URL: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6066231b sci-libs/caffe2: install nvfuser and functorch files Closes: https://bugs.gentoo.org/914572 Signed-off-by: Alfredo Tupone gentoo.org> sci-libs/caffe2/caffe2-2.0.1-r5.ebuild | 210 +++++++++++++++++++++ sci-libs/caffe2/files/caffe2-2.0.1-cudaExtra.patch | 28 +++ 2 files changed, 238 insertions(+) diff --git a/sci-libs/caffe2/caffe2-2.0.1-r5.ebuild b/sci-libs/caffe2/caffe2-2.0.1-r5.ebuild new file mode 100644 index 000000000000..c062b38f1e86 --- /dev/null +++ b/sci-libs/caffe2/caffe2-2.0.1-r5.ebuild @@ -0,0 +1,210 @@ +# Copyright 2022-2023 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +EAPI=8 + +PYTHON_COMPAT=( python3_{9..11} ) +inherit python-single-r1 cmake cuda flag-o-matic prefix + +MYPN=pytorch +MYP=${MYPN}-${PV} + +DESCRIPTION="A deep learning framework" +HOMEPAGE="https://pytorch.org/" +SRC_URI="https://github.com/pytorch/${MYPN}/archive/refs/tags/v${PV}.tar.gz + -> ${MYP}.tar.gz" + +LICENSE="BSD" +SLOT="0" +KEYWORDS="~amd64" +IUSE="cuda distributed fbgemm ffmpeg gloo mpi nnpack +numpy opencl opencv openmp qnnpack tensorpipe xnnpack" +RESTRICT="test" +REQUIRED_USE=" + ${PYTHON_REQUIRED_USE} + ffmpeg? ( opencv ) + mpi? ( distributed ) + tensorpipe? ( distributed ) + distributed? ( tensorpipe ) + gloo? ( distributed ) +" # ?? ( cuda rocm ) + +# CUDA 12 not supported yet: https://github.com/pytorch/pytorch/issues/91122 +RDEPEND=" + ${PYTHON_DEPS} + dev-cpp/gflags:= + >=dev-cpp/glog-0.5.0 + dev-libs/cpuinfo + dev-libs/libfmt + dev-libs/protobuf:= + dev-libs/pthreadpool + dev-libs/sleef + sci-libs/lapack + >=sci-libs/onnx-1.12.0 + sci-libs/foxi + cuda? ( + =dev-libs/cudnn-8* + dev-libs/cudnn-frontend:0/8 + =sci-libs/XNNPACK-2022.12.22 ) +" +DEPEND=" + ${RDEPEND} + dev-cpp/eigen + cuda? ( dev-libs/cutlass ) + dev-libs/psimd + dev-libs/FP16 + dev-libs/FXdiv + dev-libs/pocketfft + dev-libs/flatbuffers + sci-libs/kineto + $(python_gen_cond_dep ' + dev-python/pyyaml[${PYTHON_USEDEP}] + dev-python/pybind11[${PYTHON_USEDEP}] + ') +" + +S="${WORKDIR}"/${MYP} + +PATCHES=( + "${FILESDIR}"/${PN}-2.0.0-gentoo.patch + "${FILESDIR}"/${PN}-1.13.0-install-dirs.patch + "${FILESDIR}"/${PN}-1.12.0-glog-0.6.0.patch + "${FILESDIR}"/${PN}-1.13.1-tensorpipe.patch + "${FILESDIR}"/${PN}-2.0.0-gcc13.patch + "${FILESDIR}"/${PN}-2.0.0-cudnn_include_fix.patch + "${FILESDIR}"/${P}-cudaExtra.patch +) + +src_prepare() { + filter-lto #bug 862672 + sed -i \ + -e "/third_party\/gloo/d" \ + cmake/Dependencies.cmake \ + || die + cmake_src_prepare + pushd torch/csrc/jit/serialization || die + flatc --cpp --gen-mutable --scoped-enums mobile_bytecode.fbs || die + popd + # prefixify the hardcoded paths, after all patches are applied + hprefixify \ + aten/CMakeLists.txt \ + caffe2/CMakeLists.txt \ + cmake/Metal.cmake \ + cmake/Modules/*.cmake \ + cmake/Modules_CUDA_fix/FindCUDNN.cmake \ + cmake/Modules_CUDA_fix/upstream/FindCUDA/make2cmake.cmake \ + cmake/Modules_CUDA_fix/upstream/FindPackageHandleStandardArgs.cmake \ + cmake/public/LoadHIP.cmake \ + cmake/public/cuda.cmake \ + cmake/Dependencies.cmake \ + torch/CMakeLists.txt \ + CMakeLists.txt +} + +src_configure() { + if use cuda && [[ -z ${TORCH_CUDA_ARCH_LIST} ]]; then + ewarn "WARNING: caffe2 is being built with its default CUDA compute capabilities: 3.5 and 7.0." + ewarn "These may not be optimal for your GPU." + ewarn "" + ewarn "To configure caffe2 with the CUDA compute capability that is optimal for your GPU," + ewarn "set TORCH_CUDA_ARCH_LIST in your make.conf, and re-emerge caffe2." + ewarn "For example, to use CUDA capability 7.5 & 3.5, add: TORCH_CUDA_ARCH_LIST=7.5 3.5" + ewarn "For a Maxwell model GPU, an example value would be: TORCH_CUDA_ARCH_LIST=Maxwell" + ewarn "" + ewarn "You can look up your GPU's CUDA compute capability at https://developer.nvidia.com/cuda-gpus" + ewarn "or by running /opt/cuda/extras/demo_suite/deviceQuery | grep 'CUDA Capability'" + fi + + local mycmakeargs=( + -DBUILD_CUSTOM_PROTOBUF=OFF + -DBUILD_SHARED_LIBS=ON + + -DUSE_CCACHE=OFF + -DUSE_CUDA=$(usex cuda) + -DUSE_CUDNN=$(usex cuda) + -DUSE_FAST_NVCC=$(usex cuda) + -DTORCH_CUDA_ARCH_LIST="${TORCH_CUDA_ARCH_LIST:-3.5 7.0}" + -DBUILD_NVFUSER=$(usex cuda) + -DUSE_DISTRIBUTED=$(usex distributed) + -DUSE_MPI=$(usex mpi) + -DUSE_FAKELOWP=OFF + -DUSE_FBGEMM=$(usex fbgemm) + -DUSE_FFMPEG=$(usex ffmpeg) + -DUSE_GFLAGS=ON + -DUSE_GLOG=ON + -DUSE_GLOO=$(usex gloo) + -DUSE_KINETO=OFF # TODO + -DUSE_LEVELDB=OFF + -DUSE_MAGMA=OFF # TODO: In GURU as sci-libs/magma + -DUSE_MKLDNN=OFF + -DUSE_NCCL=OFF # TODO: NVIDIA Collective Communication Library + -DUSE_NNPACK=$(usex nnpack) + -DUSE_QNNPACK=$(usex qnnpack) + -DUSE_XNNPACK=$(usex xnnpack) + -DUSE_SYSTEM_XNNPACK=$(usex xnnpack) + -DUSE_TENSORPIPE=$(usex tensorpipe) + -DUSE_PYTORCH_QNNPACK=OFF + -DUSE_NUMPY=$(usex numpy) + -DUSE_OPENCL=$(usex opencl) + -DUSE_OPENCV=$(usex opencv) + -DUSE_OPENMP=$(usex openmp) + -DUSE_ROCM=OFF # TODO + -DUSE_SYSTEM_CPUINFO=ON + -DUSE_SYSTEM_PYBIND11=ON + -DUSE_UCC=OFF + -DUSE_VALGRIND=OFF + -DPYBIND11_PYTHON_VERSION="${EPYTHON#python}" + -DPYTHON_EXECUTABLE="${PYTHON}" + -DUSE_ITT=OFF + -DBLAS=Eigen # avoid the use of MKL, if found on the system + -DUSE_SYSTEM_EIGEN_INSTALL=ON + -DUSE_SYSTEM_PTHREADPOOL=ON + -DUSE_SYSTEM_FXDIV=ON + -DUSE_SYSTEM_FP16=ON + -DUSE_SYSTEM_GLOO=ON + -DUSE_SYSTEM_ONNX=ON + -DUSE_SYSTEM_SLEEF=ON + + -Wno-dev + -DTORCH_INSTALL_LIB_DIR="${EPREFIX}"/usr/$(get_libdir) + -DLIBSHM_INSTALL_LIB_SUBDIR="${EPREFIX}"/usr/$(get_libdir) + ) + + if use cuda; then + addpredict "/dev/nvidiactl" # bug 867706 + addpredict "/dev/char" + + mycmakeargs+=( + -DCMAKE_CUDA_FLAGS="$(cuda_gccdir -f | tr -d \")" + ) + fi + cmake_src_configure +} + +src_install() { + cmake_src_install + + insinto "/var/lib/${PN}" + doins "${BUILD_DIR}"/CMakeCache.txt + + rm -rf python + mkdir -p python/torch/include || die + mv "${ED}"/usr/lib/python*/site-packages/caffe2 python/ || die + mv "${ED}"/usr/include/torch python/torch/include || die + cp torch/version.py python/torch/ || die + python_domodule python/caffe2 + python_domodule python/torch +} diff --git a/sci-libs/caffe2/files/caffe2-2.0.1-cudaExtra.patch b/sci-libs/caffe2/files/caffe2-2.0.1-cudaExtra.patch new file mode 100644 index 000000000000..2366c04c1b40 --- /dev/null +++ b/sci-libs/caffe2/files/caffe2-2.0.1-cudaExtra.patch @@ -0,0 +1,28 @@ +--- a/third_party/nvfuser/CMakeLists.txt 2023-11-30 21:42:07.336946970 +0100 ++++ b/third_party/nvfuser/CMakeLists.txt 2023-11-30 21:46:35.101749250 +0100 +@@ -18,7 +18,7 @@ + set(NVFUSER_ROOT ${PROJECT_SOURCE_DIR}) + set(NVFUSER_SRCS_DIR "${NVFUSER_ROOT}/csrc") + set(TORCH_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/../..") +-set(TORCH_INSTALL_LIB_DIR ${TORCH_ROOT}/torch/lib) ++set(TORCH_INSTALL_LIB_DIR ${CMAKE_INSTALL_LIBDIR}) + + # --- build nvfuser_codegen library + +@@ -213,7 +213,7 @@ + message(STATUS "somehow this is happening") + set_target_properties(${NVFUSER} PROPERTIES LINK_FLAGS ${TORCH_PYTHON_LINK_FLAGS}) + endif() +- install(TARGETS ${NVFUSER} EXPORT NvfuserTargets DESTINATION ${TORCH_ROOT}/nvfuser/) ++ install(TARGETS ${NVFUSER} EXPORT NvfuserTargets DESTINATION "${TORCH_INSTALL_LIB_DIR}") + endif() + + # --- generate runtime files +--- a/functorch/CMakeLists.txt 2023-11-30 20:30:45.805209036 +0100 ++++ b/functorch/CMakeLists.txt 2023-11-30 20:31:13.284766157 +0100 +@@ -35,4 +35,4 @@ + if(NOT ${TORCH_PYTHON_LINK_FLAGS} STREQUAL "") + set_target_properties(${PROJECT_NAME} PROPERTIES LINK_FLAGS ${TORCH_PYTHON_LINK_FLAGS}) + endif() +-install(TARGETS ${PROJECT_NAME} DESTINATION "${CMAKE_CURRENT_SOURCE_DIR}") ++install(TARGETS ${PROJECT_NAME} DESTINATION "${CMAKE_INSTALL_LIBDIR}")