From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id CB4C7138334 for ; Fri, 11 Jan 2019 10:17:01 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 9CC97E09A5; Fri, 11 Jan 2019 10:17:00 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 51BA0E09A5 for ; Fri, 11 Jan 2019 10:17:00 +0000 (UTC) Received: from gentoo.org (unknown [IPv6:2001:980:3ff0:64:5054:ff:fe0b:7015]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: grobian) by smtp.gentoo.org (Postfix) with ESMTPSA id A9918335CC0 for ; Fri, 11 Jan 2019 10:16:58 +0000 (UTC) Date: Fri, 11 Jan 2019 11:16:49 +0100 From: Fabian Groffen To: gentoo-portage-dev@lists.gentoo.org Subject: Re: [gentoo-portage-dev] [PATCH v5] collision_protect: use dynamic report interval Message-ID: <20190111101649.GA16745@gentoo.org> References: <20190109083323.3590-1-grobian@gentoo.org> <20190110153013.6186-1-grobian@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-portage-dev@lists.gentoo.org Reply-to: gentoo-portage-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="sm4nu43k4a2Rpi4c" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.2 (SunOS 5.11, VIM - Vi IMproved 8.1) Organization: Gentoo Foundation, Inc. X-Archives-Salt: b81bce5a-c56d-499d-947f-1a573db39a7f X-Archives-Hash: 787717f2ba6924affcca858c80734544 --sm4nu43k4a2Rpi4c Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Pushed, thanks On 10-01-2019 20:37:16 -0800, Zac Medico wrote: > On 1/10/19 7:30 AM, Fabian Groffen wrote: > > The reporting of files remaining can look somewhat odd since the report > > interval is hardcoded to be per 1000 objects. Adjust this interval to > > be time based. This means that modern (fast) machines likely will never > > see the countdown messages at all. On slow setups the message will be > > informative that there is progress, albeit rather slowly. While at it, > > report percentage done. > >=20 > > Output before this patch: > >=20 > > * checking 6158 files for package collisions > > 5158 files remaining ... > > 4158 files remaining ... > > 3158 files remaining ... > > 2158 files remaining ... > > 1158 files remaining ... > > 158 files remaining ... > >=20 > > Possible output after this patch on a slower machine: > >=20 > > * checking 6158 files for package collisions > > 48% done, 3145 files remaining ... > > 96% done, 192 files remaining ... > > 100% done > >=20 > > Signed-off-by: Fabian Groffen > > --- > > lib/portage/dbapi/vartree.py | 15 +++++++++++++-- > > 1 file changed, 13 insertions(+), 2 deletions(-) > >=20 > > diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py > > index 4b91caea8..909c0e473 100644 > > --- a/lib/portage/dbapi/vartree.py > > +++ b/lib/portage/dbapi/vartree.py > > @@ -35,6 +35,7 @@ portage.proxy.lazyimport.lazyimport(globals(), > > 'portage.util.install_mask:install_mask_dir,InstallMask', > > 'portage.util.listdir:dircache,listdir', > > 'portage.util.movefile:movefile', > > + 'portage.util.monotonic:monotonic', > > 'portage.util.path:first_existing,iter_parents', > > 'portage.util.writeable_check:get_ro_checker', > > 'portage.util._xattr:xattr', > > @@ -3475,13 +3476,21 @@ class dblink(object): > > symlink_collisions =3D [] > > destroot =3D self.settings['ROOT'] > > totfiles =3D len(file_list) + len(symlink_list) > > + previous =3D monotonic() > > + progress_shown =3D False > > + report_interval =3D 1.7 # seconds > > + falign =3D len("%d" % totfiles) > > showMessage(_(" %s checking %d files for package collisions\n") % \ > > (colorize("GOOD", "*"), totfiles)) > > for i, (f, f_type) in enumerate(chain( > > ((f, "reg") for f in file_list), > > ((f, "sym") for f in symlink_list))): > > - if i % 1000 =3D=3D 0 and i !=3D 0: > > - showMessage(_("%d files remaining ...\n") % (totfiles - i)) > > + current =3D monotonic() > > + if current - previous > report_interval: > > + showMessage(_("%3d%% done, %*d files remaining ...\n") % > > + (i * 100 / totfiles, falign, totfiles - i)) > > + previous =3D current > > + progress_shown =3D True > > =20 > > dest_path =3D normalize_path( > > os.path.join(destroot, f.lstrip(os.path.sep))) > > @@ -3570,6 +3579,8 @@ class dblink(object): > > break > > if stopmerge: > > collisions.append(f) > > + if progress_shown: > > + showMessage(_("100% done\n")) > > return collisions, dirs_ro, symlink_collisions, plib_collisions > > =20 > > def _lstat_inode_map(self, path_iter): > >=20 >=20 > Looks good! > --=20 > Thanks, > Zac >=20 --=20 Fabian Groffen Gentoo on a different level --sm4nu43k4a2Rpi4c Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEELUvHd/Gtp7LaU1vuzpXahU5EQpMFAlw4bREACgkQzpXahU5E QpNKPAf/cZGy8siPHQwu9RKa+tqH1xU0wAA6VYa28SlR2L/l+KFb/bWT9PEYAslS VJH3Hc1xUR6gmo8Kb8AI400MnhfGUOBw2LiYxXCpy4asAFOugX4XDtJtjU62+ZcW miSZMiSSodNgT+Ooid3WWRXtBMq94NrYT95Q5bCyJeGeeSW0yPnSg/1cmCt3G67l KpDDJuNVA+6Updl2qgVpGEMfZqhrWAaSo1JlKLpULZVas8XNXMXPVIXBRlIKmzvr SU6OoHAh9liOg+OSOcaO9KkSw34yEiGYPG6TZemW4MQbQ34H3kxN43zbHGwgwIeU isxG1UwwcItNV9w5m+diKIiX8xk5xw== =kK5L -----END PGP SIGNATURE----- --sm4nu43k4a2Rpi4c--