From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 1F78715800A for ; Sat, 22 Jul 2023 15:44:18 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 8754BE086E; Sat, 22 Jul 2023 15:44:14 +0000 (UTC) Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 5BDFFE07F0 for ; Sat, 22 Jul 2023 15:44:14 +0000 (UTC) Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-99357737980so475442666b.2 for ; Sat, 22 Jul 2023 08:44:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1690040653; x=1690645453; h=content-disposition:mime-version:message-id:subject:to:from:date :from:to:cc:subject:date:message-id:reply-to; bh=VJCLH2gor8hrKOARmfjju7d6geO1ZGeM4v/WjwCpKVI=; b=hornBMHI3pMveMGa/I66wURHMXr2dVSbyhcMX6nyeL5bz6ExTiDTZee0O2Ru4AVoTl yO1GaRxRm5syGe473CavbyK2R/tqQz2L9B4JAETXZa6YwUTl0DwYcA0kZogTb+OFUe96 /yoOTUihQXa8V8UDjHPyihneGctgptqgdOWi39SE2jI1g77o2Z1IVVHaLq/UCU1YTQ1/ pGZvuapUN3AR70N0WkxsuaPsPDtkTa5T/OTpSssISGlKc8EHp0KXwL6oJymkRRzw5gHv 7VTsQSXHt7beSkTtMupqAVZkwM+M/T7WXBIYDbOKWw5JkVEntOYD3MoBY0sNNlGTLob/ GIJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690040653; x=1690645453; h=content-disposition:mime-version:message-id:subject:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VJCLH2gor8hrKOARmfjju7d6geO1ZGeM4v/WjwCpKVI=; b=eripODUMH9WD+inUKKadX6LSWgrBscfDtGBOV1WJF2VWQfRQZP5lfBPAQctBqvY4KB HiUSe4Jq8eA2CFf8ki2nmsSrW8gE5aOvMrEsU5JIPUT+dt+Ffda6zfQswKkiirB+uNuf EEV+9bsyOF2G4Rotz0jQFA9dd0luT+U1mP+PBwB92vtig/cL/AW0xqP2rFnieqeiFqYl aEzurnR8SDvO3DLOAMZPZ8K8Li0cCSWE1HG+gYt3ph4U/0r/l28NiocroyERx1wZj4kF /nWS1nMySukayqZwRJpZENc/IxTBFjzdweQ/G8OhUhNXIGVcX4PKYet2WOZoHWt4ykQ4 AWZg== X-Gm-Message-State: ABy/qLYGeKWVhPLGZDcKIFr043bfc4uSrd3Lr7sAjEpIaiSrVNEzdC14 mv6xeZKGW/QiDBuis0bkcrkoWmnbesYolA== X-Google-Smtp-Source: APBJJlGybMmX7ZHF7Cqd6G5yAC8NT4cCPfnjLsbPJbAtuygxrGgw5c2TPFML2fcOjMxzzaHCqDtHgg== X-Received: by 2002:a17:906:64c1:b0:99b:4668:865f with SMTP id p1-20020a17090664c100b0099b4668865fmr4683123ejn.10.1690040652655; Sat, 22 Jul 2023 08:44:12 -0700 (PDT) Received: from NeoTokyo (2a02-a210-2d40-3a80-e45a-723b-28c8-0551.cable.dynamic.v6.ziggo.nl. [2a02:a210:2d40:3a80:e45a:723b:28c8:551]) by smtp.gmail.com with ESMTPSA id o13-20020a1709061d4d00b00992f8110a2bsm3672480ejh.150.2023.07.22.08.44.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 22 Jul 2023 08:44:12 -0700 (PDT) Date: Sat, 22 Jul 2023 17:44:11 +0200 From: Sebastian Parborg To: gentoo-dev@lists.gentoo.org Subject: [gentoo-dev] [PATCH] git-r3.eclass: Add Git LFS support Message-ID: Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Archives-Salt: ce79731b-c5bc-46af-9754-523b85e66480 X-Archives-Hash: 59e7f9fdb2b385636b15b4aca26e6d21 Github PR can be found at: https://github.com/gentoo/gentoo/pull/31905 Signed-off-by: Sebastian Parborg --- eclass/git-r3.eclass | 90 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/eclass/git-r3.eclass b/eclass/git-r3.eclass index 5ac141962b12..83c5ab590c39 100644 --- a/eclass/git-r3.eclass +++ b/eclass/git-r3.eclass @@ -10,6 +10,21 @@ # Third generation eclass for easing maintenance of live ebuilds using # git as remote repository. +# @ECLASS_VARIABLE: EGIT_LFS +# @PRE_INHERIT +# @DEFAULT_UNSET +# @DESCRIPTION: +# If set, git lfs support will be enabled. +# Set before inheriting this eclass. + +# @ECLASS_VARIABLE: _NUM_LFS_FILTERS_FOUND +# @INTERNAL +# @DEFAULT_UNSET +# @DESCRIPTION: +# This is used to provide QA warnings if a repo has git lfs filters +# defined but EGIT_LFS is not turned on and vice versa. +# If non-empty, then the repo likely needs EGIT_LFS to clone properly. + case ${EAPI} in 6|7|8) ;; *) die "${ECLASS}: EAPI ${EAPI:-0} not supported" ;; @@ -22,8 +37,10 @@ PROPERTIES+=" live" if [[ ${EAPI} != 6 ]]; then BDEPEND=">=dev-vcs/git-1.8.2.1[curl]" + [[ ${EGIT_LFS} ]] && BDEPEND+=" dev-vcs/git-lfs" else DEPEND=">=dev-vcs/git-1.8.2.1[curl]" + [[ ${EGIT_LFS} ]] && DEPEND+=" dev-vcs/git-lfs" fi # @ECLASS_VARIABLE: EGIT_CLONE_TYPE @@ -81,6 +98,28 @@ fi # fatal errors rather than 'non-pretty versions'. : "${EGIT_MIN_CLONE_TYPE:=shallow}" +# @ECLASS_VARIABLE: EGIT_LFS_CLONE_TYPE +# @USER_VARIABLE +# @DESCRIPTION: +# Type of lfs clone that should be used against the remote repository. +# This can be either of: 'mirror', 'single', 'shallow'. +# +# This works a bit differently than EGIT_CLONE_TYPE. +# +# The 'mirror' type clones all LFS files that is available from the +# cloned repo. Is is mostly useful for backup or rehosting purposes as +# the disk usage will be excessive. +# +# The 'single' type clones only the LFS files from the current commit. +# However unlike 'shallow', it will not cleanup stale LFS files. +# +# The 'shallow' type clones only the LFS files from the current commit. +# LFS files that are not referenced by the current commit and more than +# a few days old will be automatically removed to save disk space. +# This is the recommended mode for LFS repos to prevent excessive disk +# usage. +: "${EGIT_LFS_CLONE_TYPE:=shallow}" + # @ECLASS_VARIABLE: EGIT3_STORE_DIR # @USER_VARIABLE # @DEFAULT_UNSET @@ -760,6 +799,35 @@ git-r3_fetch() { fi fi + if [[ ${EGIT_LFS} ]]; then + # Fetch the LFS files from the current ref (if any) + local lfs_fetch_command=( git lfs fetch "${r}" ) + + case "${EGIT_LFS_CLONE_TYPE}" in + shallow) + lfs_fetch_command+=( + --prune + ) + ;; + single) + ;; + mirror) + lfs_fetch_command+=( + --all + ) + ;; + *) + die "Invalid EGIT_LFS_CLONE_TYPE=${EGIT_LFS_CLONE_TYPE}" + esac + + set -- "${lfs_fetch_command[@]}" + echo "${@}" >&2 + "${@}" || die + elif [[ -d ${GIT_DIR}/lfs && ${EGIT_LFS_CLONE_TYPE} == shallow ]]; then + # Cleanup the LFS files from old checkouts if LFS support has been turned off. + rm -fr ${GIT_DIR}/lfs || die + fi + success=1 break done @@ -875,6 +943,11 @@ git-r3_checkout() { # non-empty directories. git init --quiet -b __init__ || die + if [[ ${EGIT_LFS} ]]; then + # The "skip-repo" flag will just skip the installation of the pre-push hooks. + # We don't use these hook as we don't do any pushes + git lfs install --local --skip-repo || die + fi # setup 'alternates' to avoid copying objects echo "${orig_repo}/objects" > "${GIT_DIR}"/objects/info/alternates || die # now copy the refs @@ -905,6 +978,16 @@ git-r3_checkout() { fi echo "${@}" >&2 "${@}" || die "git checkout ${remote_ref:-${new_commit_id}} failed" + + # If any filters in any of the ".gitattributes" files specifies lfs, + # then this repo is most likely storing files with git lfs. + local has_git_lfs_filters=$( + git grep "filter=lfs" -- ".gitattributes" "**/.gitattributes" + ) + if [[ $has_git_lfs_filters ]]; then + # This is used for issuing QA warnings regarding LFS files in the repo (or lack thereof) + _EGIT_LFS_FILTERS_FOUND="yes" + fi } git-r3_sub_checkout unset -f git-r3_sub_checkout @@ -1054,6 +1137,13 @@ git-r3_src_unpack() { _git-r3_env_setup git-r3_src_fetch git-r3_checkout + + if [[ ! ${EGIT_LFS} && ${_EGIT_LFS_FILTERS_FOUND} ]]; then + eqawarn "QA Notice: There are Git LFS filters setup in the cloned repo, consider using EGIT_LFS!" + fi + if [[ ${EGIT_LFS} && ! ${_EGIT_LFS_FILTERS_FOUND} ]]; then + eqawarn "QA Notice: There are no Git LFS filters setup in the cloned repo. EGIT_LFS will do nothing!" + fi } # https://bugs.gentoo.org/show_bug.cgi?id=482666 -- 2.41.0