From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 3460E138335 for ; Wed, 21 Nov 2018 19:13:21 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id C7593E0880; Wed, 21 Nov 2018 19:13:19 +0000 (UTC) Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 9E1DBE0880 for ; Wed, 21 Nov 2018 19:13:19 +0000 (UTC) Received: from oystercatcher.gentoo.org (oystercatcher.gentoo.org [148.251.78.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id EDCA1335C8C for ; Wed, 21 Nov 2018 19:13:17 +0000 (UTC) Received: from localhost.localdomain (localhost [IPv6:::1]) by oystercatcher.gentoo.org (Postfix) with ESMTP id D8562456 for ; Wed, 21 Nov 2018 19:13:15 +0000 (UTC) From: "Matt Turner" To: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: 8bit Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Matt Turner" Message-ID: <1542827246.c9ad265f80178b8786974e0d1e250daa42e6a742.mattst88@gentoo> Subject: [gentoo-commits] proj/releng:master commit in: scripts/ X-VCS-Repository: proj/releng X-VCS-Files: scripts/copy_buildsync.sh X-VCS-Directories: scripts/ X-VCS-Committer: mattst88 X-VCS-Committer-Name: Matt Turner X-VCS-Revision: c9ad265f80178b8786974e0d1e250daa42e6a742 X-VCS-Branch: master Date: Wed, 21 Nov 2018 19:13:15 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org X-Archives-Salt: 3c493536-138c-49a1-9a80-cfe522a97f15 X-Archives-Hash: 89f99279eafd76055da89e2f0f1fb8dc commit: c9ad265f80178b8786974e0d1e250daa42e6a742 Author: Matt Turner gentoo org> AuthorDate: Wed Nov 21 06:43:38 2018 +0000 Commit: Matt Turner gentoo org> CommitDate: Wed Nov 21 19:07:26 2018 +0000 URL: https://gitweb.gentoo.org/proj/releng.git/commit/?id=c9ad265f scripts/copy_buildsync: try to exclude rsync tempfile In-progress uploads are named .. and were mistakenly recognized by copy_buildsync.sh as targets to sweep to the mirrors. When it did, it deleted the file causing the in-progress upload to fail. Since the script is run by cron every 15 minutes, this would lead to a large portion of the generated installation media to fail to upload, and worse there is no pattern to it -- it just happens whenever an upload spans the 15 minute boundary. The fix is to ignore hidden files and to tell rsync (via the filter rules) to exclude them from its sweep to the mirrors. Closes: https://bugs.gentoo.org/670458 Signed-off-by: Matt Turner gentoo.org> Signed-off-by: Robin H. Johnson gentoo.org> scripts/copy_buildsync.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/copy_buildsync.sh b/scripts/copy_buildsync.sh index d67cf6ee..6c6085ff 100755 --- a/scripts/copy_buildsync.sh +++ b/scripts/copy_buildsync.sh @@ -70,14 +70,14 @@ copy_arch_to_outgoing() { fi # Copying - for i in $(find ${indir} -type f | egrep -- '-20[0123][0-9]{5}(([0-9]{6})|(T[0-9]{6}Z))?' | sed -e 's:^.*-\(20[^.]\+\).*$:\1:' | sort -ur); do + for i in $(find ${indir} -not -path '*/\.*' -type f | egrep -- '-20[0123][0-9]{5}(([0-9]{6})|(T[0-9]{6}Z))?' | sed -e 's:^.*-\(20[^.]\+\).*$:\1:' | sort -ur); do #echo "Doing $i" t="${outdir}/${i}" mkdir -p ${t} 2>/dev/null - rsync "${RSYNC_OPTS[@]}" --temp-dir=${tmpdir} --partial-dir=${tmpdir} ${indir}/ --filter "S *${i}*" --filter 'S **/' --filter 'H *' ${t} + rsync "${RSYNC_OPTS[@]}" --temp-dir=${tmpdir} --partial-dir=${tmpdir} ${indir}/ --filter '- **/.*' --filter "S *${i}*" --filter 'S **/' --filter 'H *' ${t} rc=$? if [ $rc -eq 0 ]; then - find ${indir} -type f -name "*${i}*" -print0 | xargs -0 --no-run-if-empty $DEBUGP rm $VERBOSEP -f + find ${indir} -not -path '*/\.*' -type f -name "*${i}*" -print0 | xargs -0 --no-run-if-empty $DEBUGP rm $VERBOSEP -f else echo "Not deleting ${indir}/*${i}*, rsync failed!" 1>&2 fail=1