From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 96BF9158086 for ; Fri, 12 Nov 2021 00:15:25 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 7592C2BC02F; Fri, 12 Nov 2021 00:15:24 +0000 (UTC) Received: from smtp.gentoo.org (woodpecker.gentoo.org [140.211.166.183]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 527112BC02F for ; Fri, 12 Nov 2021 00:15:23 +0000 (UTC) Received: from oystercatcher.gentoo.org (unknown [IPv6:2a01:4f8:202:4333:225:90ff:fed9:fc84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id E5B7D342C00 for ; Fri, 12 Nov 2021 00:15:20 +0000 (UTC) Received: from localhost.localdomain (localhost [IPv6:::1]) by oystercatcher.gentoo.org (Postfix) with ESMTP id 29F5CD3 for ; Fri, 12 Nov 2021 00:15:19 +0000 (UTC) From: "Robin H. Johnson" To: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: 8bit Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Robin H. Johnson" Message-ID: <1636676105.8eb8dd235dfeb80df3740369343177f3cb64dfa8.robbat2@gentoo> Subject: [gentoo-commits] proj/releng:master commit in: scripts/ X-VCS-Repository: proj/releng X-VCS-Files: scripts/copy_buildsync.sh X-VCS-Directories: scripts/ X-VCS-Committer: robbat2 X-VCS-Committer-Name: Robin H. Johnson X-VCS-Revision: 8eb8dd235dfeb80df3740369343177f3cb64dfa8 X-VCS-Branch: master Date: Fri, 12 Nov 2021 00:15:19 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply X-Archives-Salt: f5bfa1c9-5690-48b8-ae66-2fb11baf6565 X-Archives-Hash: 7beba4646a0c7cc16c1fbc119eb37223 commit: 8eb8dd235dfeb80df3740369343177f3cb64dfa8 Author: Robin H. Johnson gentoo org> AuthorDate: Fri Nov 12 00:15:05 2021 +0000 Commit: Robin H. Johnson gentoo org> CommitDate: Fri Nov 12 00:15:05 2021 +0000 URL: https://gitweb.gentoo.org/proj/releng.git/commit/?id=8eb8dd23 scripts/copy_buildsync: try to find dead links and textfiles Signed-off-by: Robin H. Johnson gentoo.org> scripts/copy_buildsync.sh | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/scripts/copy_buildsync.sh b/scripts/copy_buildsync.sh index 7b4dfbdc..e224f75e 100755 --- a/scripts/copy_buildsync.sh +++ b/scripts/copy_buildsync.sh @@ -164,10 +164,12 @@ process_arch() { # New variant preserve code find_variants=( '(' -iname '*.iso' -o -name 'netboot-*' -o "${EXTENSIONS[@]}" ')' ) variants=$(find 20* "${find_variants[@]}" -printf '%f\n' 2>/dev/null | sed -e 's,-20[012][0-9]\{5\}.*,,g' -r | sort -u) + # This file specifies which variants are still in use. keepfile="${tmpdir}/.keep.${ARCH}.txt" keepfile_tmp=$(mktemp -p "${tmpdir}" -t ".keep.${ARCH}.txt.XXXXXX") echo -n '' >"${keepfile_tmp}" chmod 644 "${keepfile_tmp}" + for v in $variants ; do # FIXME: trace the $a variable in this! variant_path=$(find 20* -iname "${v}-20*" "${find_variants[@]}" -print 2>/dev/null | sed -e "s,.*/$a/autobuilds/,,g" | sort -k1,1 -t/ | tail -n1 ) @@ -188,6 +190,8 @@ process_arch() { echo "${variant_path}" | sed -e 's,/.*,,g' -e 's,^,/,g' -e 's,$,$,g' >>"${keepfile_tmp}" mv -f "${f_tmp}" "${f}" done + + # Refresh keepfile mv -f "${keepfile_tmp}" "${keepfile}" # ================================================================ @@ -206,6 +210,37 @@ process_arch() { $DEBUGP rm $VERBOSEP -rf "$(pwd)"/"${i}" done + # Preserve the keepfile for review + cp -lf "${keepfile}" "${outdir}" + + # Find the dead links for cleanup + _dead="${tmpdir}"/dead-link + find -L $(pwd) -type l >"${_dead}" + if test -s "${_dead}"; then + echo "copy_buildsync: dead links to verify:" 1>&2 + cat "${_dead}" 1>&2 + fi + + # Find the dead latest txt files + _dead="${tmpdir}"/dead-latest + for f in $(find "${outdir}" -name 'latest*txt' ) ; do + d=$(dirname $f) + find $d \ + | fgrep -f <(awk '/^#/{next} {print $1}' $f) \ + | sed "s,${d}/,,g" \ + | sort \ + | comm -13 \ + - \ + <(sort $f |grep -v -e '^#' |awk '{print $1}') \ + | fgrep -l -f - $f \ + | xargs -n1 --no-ruN-if-empty readlink -f + done >"${_dead}" + if test -s "${_dead}"; then + echo "copy_buildsync: dead latest*txt files to verify:" 1>&2 + cat "${_dead}" 1>&2 + fi + + # Cleanup tmpdir $DEBUGP rm $VERBOSEP -rf "${tmpdir}" else