From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id B3C21138334 for ; Wed, 9 Jan 2019 08:09:42 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 09EDCE0CAC; Wed, 9 Jan 2019 08:09:42 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id BF142E0CAC for ; Wed, 9 Jan 2019 08:09:39 +0000 (UTC) Received: from gentoo.org (unknown [IPv6:2001:980:3ff0:64:5054:ff:fe0b:7015]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: grobian) by smtp.gentoo.org (Postfix) with ESMTPSA id B1D90335C94 for ; Wed, 9 Jan 2019 08:09:37 +0000 (UTC) Date: Wed, 9 Jan 2019 09:09:28 +0100 From: Fabian Groffen To: gentoo-portage-dev@lists.gentoo.org Subject: Re: [gentoo-portage-dev] [PATCH v3] collision_protect: use dynamic report interval Message-ID: <20190109080928.GA7770@gentoo.org> References: <20190107143113.7068-1-grobian@gentoo.org> <20190108134207.15339-1-grobian@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-portage-dev@lists.gentoo.org Reply-to: gentoo-portage-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="J/dobhs11T7y2rNN" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.2 (SunOS 5.11, VIM - Vi IMproved 8.1) Organization: Gentoo Foundation, Inc. X-Archives-Salt: 2b0c1315-73a7-4f5d-8e2e-2b082958da7b X-Archives-Hash: d5f4e7f56e72e0a93870b1881db0b7cf --J/dobhs11T7y2rNN Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 08-01-2019 20:59:34 +0000, M. J. Everitt wrote: > On 08/01/19 19:15, Zac Medico wrote: > > On 1/8/19 5:42 AM, Fabian Groffen wrote: > >> The reporting of files remaining can look somewhat odd since the report > >> interval is hardcoded to be per 1000 objects. Adjust this interval to > >> be time based. This means that modern (fast) machines likely will nev= er > >> see the countdown messages at all. On slow setups the message will be > >> informative that there is progress, albeit rather slowly. While at it, > >> report percentage done. > >> > >> Output before this patch: > >> > >> * checking 6158 files for package collisions > >> 5158 files remaining ... > >> 4158 files remaining ... > >> 3158 files remaining ... > >> 2158 files remaining ... > >> 1158 files remaining ... > >> 158 files remaining ... > >> > >> Possible output after this patch on a slower machine: > >> > >> * checking 6158 files for package collisions > >> 48% done, 3145 files remaining ... > >> 96% done, 192 files remaining ... > >> 100% done > >> > >> Signed-off-by: Fabian Groffen > >> --- > >> lib/portage/dbapi/vartree.py | 11 +++++++++-- > >> 1 file changed, 9 insertions(+), 2 deletions(-) > >> > >> diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.= py > >> index 4b91caea8..244195fad 100644 > >> --- a/lib/portage/dbapi/vartree.py > >> +++ b/lib/portage/dbapi/vartree.py > >> @@ -3475,13 +3475,18 @@ class dblink(object): > >> symlink_collisions =3D [] > >> destroot =3D self.settings['ROOT'] > >> totfiles =3D len(file_list) + len(symlink_list) > >> + tnow =3D time.time() > >> + tinterv =3D 2 # seconds > >> + ninterv =3D tnow + tinterv > >> showMessage(_(" %s checking %d files for package collisions\n") % \ > >> (colorize("GOOD", "*"), totfiles)) > >> for i, (f, f_type) in enumerate(chain( > >> ((f, "reg") for f in file_list), > >> ((f, "sym") for f in symlink_list))): > >> - if i % 1000 =3D=3D 0 and i !=3D 0: > >> - showMessage(_("%d files remaining ...\n") % (totfiles - i)) > >> + if time.time() > ninterv: > >> + showMessage(_("%3d%% done, %d files remaining ...\n") % > >> + (i * 100 / totfiles, totfiles - i)) > >> + ninterv =3D time.time() + tinterv > >> =20 > >> dest_path =3D normalize_path( > >> os.path.join(destroot, f.lstrip(os.path.sep))) > >> @@ -3570,6 +3575,8 @@ class dblink(object): > >> break > >> if stopmerge: > >> collisions.append(f) > >> + if tnow + tinterv < ninterv: > >> + showMessage(_("100% done\n")) > >> return collisions, dirs_ro, symlink_collisions, plib_collisions > >> =20 > >> def _lstat_inode_map(self, path_iter): > >> > > Please replace time.time() with portage.util.monotonic.monotonic(). Will do. > It's a bit of a nit-pick, granted, but can we ensure that the count-down's > remain padded/justified such that the numbers line up for easy at-a-glance > inspection ? The optimal standard looks somewhat like the pre-merge Sizes: >=20 > =C2=A0* Final size of build directory: 2696 KiB (2.6 MiB) > =C2=A0* Final size of installed tree:=C2=A0 5372 KiB (5.2 MiB) >=20 > Otherwise, I think this will be quite helpful. Thanks. I think I understand what you mean, let me see if I can do something without too much complexity. Thanks, Fabian --=20 Fabian Groffen Gentoo on a different level --J/dobhs11T7y2rNN Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEELUvHd/Gtp7LaU1vuzpXahU5EQpMFAlw1rDUACgkQzpXahU5E QpPRJwf/W7jM5/o3mWZgghTyJQUWFopxpBEDArDhe9fWjI6CUWbXRVwlG7qMs70E vsAzn3uqDHnnNRKcMHOrUzbpXtMamPpFIVAWnH3rmqzCPDrlhGX96ylIl0vc6US7 Cx/+qUnBi2U9U0HzDQuAtHyajHLOZjlpTg0hRYtzacIepNbNofCguCxz4QF0AU6E +c+bUW3N32MBdFMf2AqzCWp4bJUjYQK6nUxUQ+7Wg3BQa3OY9IwVlgr0yfdDHNCg wCjGHr5g5YpQZ8/wQ0aC0Ng0UUUdaIFTSh5EJ0ZjKsvr7MQdo/KHPWTmWBem5NtP ornKd6besbm2jVM4Xex9YRHukrBaMQ== =zPvs -----END PGP SIGNATURE----- --J/dobhs11T7y2rNN--