From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 3A93F138334 for ; Tue, 8 Jan 2019 19:15:41 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 0630EE097A; Tue, 8 Jan 2019 19:15:40 +0000 (UTC) Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id B0733E097A for ; Tue, 8 Jan 2019 19:15:39 +0000 (UTC) Received: from [10.128.13.179] (unknown [100.42.98.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: zmedico) by smtp.gentoo.org (Postfix) with ESMTPSA id 5B55D335D06; Tue, 8 Jan 2019 19:15:37 +0000 (UTC) Subject: Re: [gentoo-portage-dev] [PATCH v3] collision_protect: use dynamic report interval To: gentoo-portage-dev@lists.gentoo.org, Fabian Groffen References: <20190107143113.7068-1-grobian@gentoo.org> <20190108134207.15339-1-grobian@gentoo.org> From: Zac Medico Openpgp: preference=signencrypt Autocrypt: addr=zmedico@gentoo.org; prefer-encrypt=mutual; keydata= mQINBFs7tmwBEADTzG+IcYtRfTfKryU7sUH7LlV1M+TdaCMfIkY4x6RyHXkaaqYuQ+U9HKn0 +m5FcZsZ1Ojik+We3Tz0F6kDbam6EWzBxmsLb/IHeUEsvsuLzuBQjiD9zzqGocZiPWr+uWJs AdbueS72R7FPXJPDUEPrJ9GdhGFyYARveY9cmdisOwcDOiSFfBjk3/89t4gROn4KUhezVuO9 VS14gVSns1561CJjlB47HkSBu4+FuzrfVygg4xitWAH119Ehw0vJcgkTw4Bqhk01Iw9us80m dFyU8JbJ0CVYe30gYKFFbnXoiT6xLLogKOkv0goPFxaXcMwWM9ei3SjAGVqgN6i8VnO7kquV LwkTe6ntEK0iY+l4qTKuyIOQLpCbWNI0eVwlx5b/pY2pt5TEGWAPMCZGjlidMx0aDcVX4oji 2/xegFAcxALrfOX3kj2FZ9kNAqLZu26AfqtslIqlBEAb5sZwPr351msBIdbaWX2UNw21I478 7eQ7UfohwXQHlXdhc/wop3VDkDzLBnvlK4ozSJI/9T5F/+9yEZvc6DKUWdEfD12o2El5hHan gCUQWDBKqZb1wcekK8KY2tmH8BBQi7k52IWYLJYfJdir/XpGm5SsDpf3zvDcIFXqFHAG7w7b fhriM+6oBOeIO9ew1Xj3swbRhDwdzRUhu7Uqayq1vdvKqGkgcQARAQABtCNaYWNoYXJ5IE1l ZGljbyA8em1lZGljb0BnZW50b28ub3JnPokCVAQTAQgAPgIbAwULCQgHAwUVCgkICwUWAwIB AAIeAQIXgBYhBEdYrNjamv1GpqYECtYQzzch54rNBQJbZpeGBQkEzWcaAAoJENYQzzch54rN iTEQALgfn8NqcY1P3VgE8n/ypYfCa8YhOhZcSlVOnIil9u4F0OPUP9/TFALaW2RRctUm7URP Oe6gMd8mzFauU1q8FKHW8vo6i48Oqb3RmkH4PNbH0EHfM1e3pAq+E6Bi08y7HzYUm5PKr+m3 oLNFvqZuG3RKgmKQTm8E56IpI8rODnVmBkLQEGkdi9hDB1Zkm9dlT+eRGglHAhnpb+AweOzh dvqdfu0SKEoyLjCvRIFltrtNIuKWpjqOVAaMU0hLfDlRVJR0oTThe7P0vhulKZtWS/L7wXJr e7NGIunM09JIAjYAXX5hmY9L0oKnp0WK5PksBKIu8W8TUwzVmTXNAs7qfEQQJ3LvAQo3xifa l2n0ixdoozltU8afxJohz8OKrWOAzE8kqWa+H+t1XZtQAy7yZA3x51vyp4hquyAMYrZbU8hC 0x/l6KM/qGI7RPohw5VxgneQCAbpu/G+3DLVH5QjPR+tKPtpFkEpxrbTNUhaOUqFipX92382 w2CCxDrNrBYbnFYKmVthZqVxrUuDfi4VIgeBkGzDKPwqjSkbdZ0/I5GAmjZh0aJWrRnzCum/ hZHHHcU+wOdXBwwGBhRf9SIaxCfaxw0PB+CZXsswvuaANah2b32EFTPcFCCgFvoGMNSSHFuo JR3PAo6qvwGhYdmTI/Na57P1FF+g3VojnAnK2eIEuQINBFs7tmwBEACfZb/EePObKC8tuVFl IMQaTRzm7q5THbwQvbdKdw/31cdYJaZZ7BFgnSBq2CBYDPxcm+TxvvjgNTam8kZGdEpCm3/v P1YdJSoiYbisJubV7JiAOoAmGtaDOVX5thpv28HL17wqK2d5jgqMTLFeT3hyaro5cjGAQlhT NOyfwVkdFMFLjOhGzq7aki4UZ9ieS/IedoyZvOblf79d5PV1xI0mf2w5yoHcfZRv7nwI9XMz nqlgC9/RQP7O+WfCl2pl9gyd0hc3uqUWl0ke9xHvVxlEdORH/f+OzeybYj9r80GkC6MeqiG8 qzuWO0IZZ5Gvzwkq0KgRAefr9Z/4vmyOiVhJrvk1gLp6VEdB54fUO3MG/PcnYiasRJOUwRQQ 1+q25w5i9ooZRk0LQ6oNnWbeVj+Pwn0mzL0/GjkdPsYclxzOLSlRAwkisepVl62L2m+XwXsZ j9DS3vIHEbbfCmKayBuGCXQEpJMmEnGqFLnOzv29cOutm3BnGXcWAg5aJjiV5PKEmr/4g9/E J8lphGnNCRQ2DvUzO1tlHpmT+JvDBwAgo5rfq5wjjeiS578kAgPoitOBzp/z5YM5FXX+shEd i5/ratErb9ysUfr3Z45YV3yM6MHgCpEg8+5k6fM6ey4tRnxXAYdJ4XzaSyML9fsnEg9aVaCP iRl+fuLSuF1QJf7LwwARAQABiQI8BBgBCAAmAhsMFiEER1is2Nqa/UampgQK1hDPNyHnis0F Als7trcFCQHhM8sACgkQ1hDPNyHnis0Wdw/9ERSLUHWjGW+oRJPC80RsSZO+9gbdo6AT8GP0 L3S1M60d/C4Pp9lT7sGvN31Q2h4/i1AGz2vsvjA4RUwuWI/M2F660ZmqbuLA4BbS/gjA+Wrf ZB8TDEoy7k0IO+ga0J6ENgxSOk0w1ZS+LYVrM8r7C73ZckAU2aknGOHvfIdRocpb464/JA+j kP8ENcXA0NVT17Te6K3OxpN3fBsHP7w9zbYaV3Ex3suXhLx0/+xxjLPTfwobEy3kTv5Qqumq Ph7TlXvc/9Ku4GwgAFUscLG5ZhsDULIWsRr0tyum0IYI3lpQnMykZiDCoiq7HETTwJeZPpJk PlVc7Qyy0eqcKyoiqd/ttr5tPwRFWk4OYmqzZFiZZ6sG7Zw68UXKHuhCV1MgXKPqS3M8Q6V+ nGL+ZwLE1GkvqwaPgn7uzJpfO+HzoWn3yxjNvh5plsZbM/kqCTMyByB0LQPZdjkO30GvV6dU wamblYkpytpZ5cmbIRITzsxPkOuWwXkLoz6y664JfH/Te3xVQ2rqZkuDCFrwy5nOqk92w7OY WGqyDOSL4tyCdcsXVoF5HXcHQLTv5WzCQ2MAQUW6zflyirYygTfJ3ds5mk5/J52UKEhIHPzJ Hc26Ubfvo38XuE0FJjb7xoZzCoSTpCVZ2H0f2cJOBLEVN9bnbelNlmp0m6mPAHfDrfsVR/u5 Ag0EWzvRagEQANK1C/HvZgnFVa+3tFmS4OVnCRO611C7WXubm5Y2xj/Lh2LOWv5TeTtTp5FR S3961b429TbJNv0q9N4mDi6XOGpZvWLkfiDw/VT9I+48B2eVXKx5N2H48S7t1Knwut6vuTEx 14MGiZToCs2Tu1fEUnaBv2Hg60ysVfplDAQadixzboLHM1DxLYn1W/cAUrhXAW5uNQlyE0Ze kiB10JxbnAurdpRSu4X81IrGJDK/oNAgAcWRieVX5J1N5LStPrloFKf+Dtl2z49WibXVTjwF Ir/BZYprkTTgNzeM3VRPnyRz7IA1pMkVX0r4C0O38mqHyDCpkM/TKsOpNMDqYcllD8Fa/Zyg S08RLymWvRXWn7Sz7MlZE9CCQ2aG+N20esYH3nwrYsdUdj87/nSwqYKhTWwBBIWtrrCHw56O ZMTXznX3OkLOBB0gXH92G5dKv4azBeAZzR6c+qvP6PxgVuAlvlnFbzgW5m4CrkTsEwSh/s2Y d9sJPctzkPkif6tDWk0qKy9lRwTKyOK0xqwZGcJBLdqInU05DLJlrf2QJKBS2SE1tEparQvT 8/+EBpzGk0omSxGmDxLW6EY1CTXV+LfbRYz42Y71f7aVNNqfF/CbJYLupe9DadjvceRM/ZFG WO6SzWoO7ed5uT2i8M33tC2EnK/BL1oZ5Wr7Kv5XED10JI+5ABEBAAGJBNIEGAEIACYWIQRH WKzY2pr9RqamBArWEM83IeeKzQUCWzvRagIbAgUJAeEzgAKgCRDWEM83IeeKzcHUIAQZAQgA fRYhBPDoF2pbVs6oErqQrtB1+4wQSj0gBQJbO9FqXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90 YXRpb25zLm9wZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGMEU4MTc2QTVCNTZDRUE4MTJCQTkw QUVEMDc1RkI4QzEwNEEzRDIwAAoJENB1+4wQSj0gJEMP/iP89Et7jH3Ai9w1bQ19T0ChOD4O 36IjisSTGTYsoeIAUgka/XFmuQSAgeTj5Tc73/Ny0V5jscjmtVKhUbFkLGetHuA0gHJpxEen j0zedAIQ36ToU0gia7Qsk7cpJJJRge3fdaUgb4aUcU5fTg2lkPdsMqW2B1/djlKz6T2cqX8y omvByiBdRSWVJlRmgqYQTJyQa6zLERWcknCG9T2oD9y9VJlVW3kcLT2QmB6a/3q/M0gCUBl8 qGHZYDGv+0su4SeppQ5BAOu9PXNV8pGdqTMsirQ+E20K5QprzmPg5kVG4x8qG1UpRpC/hJ9E 75wax9o4WrqHH4NmhvnL/Pfs2jK7PJLi/xRqvO65wGGO+4JW0kailhjvLoRYJi0B6ir5JhtK 44kctcqrn7QygO9ZB5r2/i1LyZrA+LWKLcpd7xxrraunFidoc63Mz5QadvWGcOduM6DOKS4b sLgr52V+413UkdtvBru/4hA9s1C8Dpc6DT0p0wFSJ7EBKE4dlu9VCnJ9+7X6HVMoq7aMj7ZW qJ1S1Jp/m6uJwzBDexcTokj8rTPBzbIo+3EfaGWMk47VoQgccee5B1jvxmLxhIbEd+LUc9ek jDQ0FETN7dOBuIIg5+Z6IdAycsLMHfNeSTP0jMHHZBKbptQwiaQ/hieMbSraN6fjozta4jw8 mUBoOJTgyjAP/2nbBGiOp/s8CkNEfbqFlLLVSnpN70PHPQ+FOKHe+FFg/bhyf6L0mGShJ8rH c8o7Re1Im/ztjj9hrGUfdicPFrU88Y+XgRGqb8DxjlyYSOVwIi6abyH5TgP1LNasBc23+cY/ nf2+n7Eq6/AOdgQRCKe5bIHXl94889HHVMQyfR+AvRZ9AiHNd7AGbhi2aMKsHgrnhuTcFO/d TR8NHolevl/zyOHmlcKaDUahjgTEK/SguUjmFHYDQYFDCrQj6U0o8uZtLA23wpiGgG1ajqpG w0VpKjkFzXcCNVeVme7XreRBxwstOda4Y00JUfOTGpR0+/bGRnJaQvBFV87WbBksVCa2McuY uzEqnkYwE4M/Hua9CRXReyvG31JRDbMubqXZjgisOtjIaNn7jvbweLoJhSzINkBB28tJiV6P ODhMx9XC5taaVLiPpABL/xzbqtHi6LCwXAYVlOS+ceiR2WIUigL866Ze607UePBJVQvsCuIS y7PV9kVbfN1/xKls9fiB87tv5XpyEUR9s+91Az6d8kterzLKqwxWSfp+HSepYuWOhlLsywUr BAJMnLR1yogNFuwyFkXIwKy/kNj4lIy+wLGNCQod1H84OLg1iHElIA+wxAWL7xnTYPxKX102 YsIUXy5A5ebVtwxpLuu8Yd6CnrTOvyLKqukoXSwP+16Mjo7C Message-ID: Date: Tue, 8 Jan 2019 11:15:35 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-portage-dev@lists.gentoo.org Reply-to: gentoo-portage-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 In-Reply-To: <20190108134207.15339-1-grobian@gentoo.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="S8ONmfR55a5MhcRfJvPq2p5WNbyH9pvT9" X-Archives-Salt: 9c50e680-ca41-4dcc-922c-8d4200355a54 X-Archives-Hash: f635bfa90a69731c7f2acdbe87d1b0a4 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --S8ONmfR55a5MhcRfJvPq2p5WNbyH9pvT9 Content-Type: multipart/mixed; boundary="sc4bXsE9DFeUPAKTMbbcWoiB1ArDWq7Ss"; protected-headers="v1" From: Zac Medico To: gentoo-portage-dev@lists.gentoo.org, Fabian Groffen Message-ID: Subject: Re: [gentoo-portage-dev] [PATCH v3] collision_protect: use dynamic report interval References: <20190107143113.7068-1-grobian@gentoo.org> <20190108134207.15339-1-grobian@gentoo.org> In-Reply-To: <20190108134207.15339-1-grobian@gentoo.org> --sc4bXsE9DFeUPAKTMbbcWoiB1ArDWq7Ss Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 1/8/19 5:42 AM, Fabian Groffen wrote: > The reporting of files remaining can look somewhat odd since the report= > interval is hardcoded to be per 1000 objects. Adjust this interval to > be time based. This means that modern (fast) machines likely will neve= r > see the countdown messages at all. On slow setups the message will be > informative that there is progress, albeit rather slowly. While at it,= > report percentage done. >=20 > Output before this patch: >=20 > * checking 6158 files for package collisions > 5158 files remaining ... > 4158 files remaining ... > 3158 files remaining ... > 2158 files remaining ... > 1158 files remaining ... > 158 files remaining ... >=20 > Possible output after this patch on a slower machine: >=20 > * checking 6158 files for package collisions > 48% done, 3145 files remaining ... > 96% done, 192 files remaining ... > 100% done >=20 > Signed-off-by: Fabian Groffen > --- > lib/portage/dbapi/vartree.py | 11 +++++++++-- > 1 file changed, 9 insertions(+), 2 deletions(-) >=20 > diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.p= y > index 4b91caea8..244195fad 100644 > --- a/lib/portage/dbapi/vartree.py > +++ b/lib/portage/dbapi/vartree.py > @@ -3475,13 +3475,18 @@ class dblink(object): > symlink_collisions =3D [] > destroot =3D self.settings['ROOT'] > totfiles =3D len(file_list) + len(symlink_list) > + tnow =3D time.time() > + tinterv =3D 2 # seconds > + ninterv =3D tnow + tinterv > showMessage(_(" %s checking %d files for package collisions\n") % \= > (colorize("GOOD", "*"), totfiles)) > for i, (f, f_type) in enumerate(chain( > ((f, "reg") for f in file_list), > ((f, "sym") for f in symlink_list))): > - if i % 1000 =3D=3D 0 and i !=3D 0: > - showMessage(_("%d files remaining ...\n") % (totfiles - i)) > + if time.time() > ninterv: > + showMessage(_("%3d%% done, %d files remaining ...\n") % > + (i * 100 / totfiles, totfiles - i)) > + ninterv =3D time.time() + tinterv > =20 > dest_path =3D normalize_path( > os.path.join(destroot, f.lstrip(os.path.sep))) > @@ -3570,6 +3575,8 @@ class dblink(object): > break > if stopmerge: > collisions.append(f) > + if tnow + tinterv < ninterv: > + showMessage(_("100% done\n")) > return collisions, dirs_ro, symlink_collisions, plib_collisions > =20 > def _lstat_inode_map(self, path_iter): >=20 Please replace time.time() with portage.util.monotonic.monotonic(). --=20 Thanks, Zac --sc4bXsE9DFeUPAKTMbbcWoiB1ArDWq7Ss-- --S8ONmfR55a5MhcRfJvPq2p5WNbyH9pvT9 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQKTBAEBCgB9FiEE8OgXaltWzqgSupCu0HX7jBBKPSAFAlw09tdfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEYw RTgxNzZBNUI1NkNFQTgxMkJBOTBBRUQwNzVGQjhDMTA0QTNEMjAACgkQ0HX7jBBK PSCMtBAApC4vaTbD3U6APOF8MA7ZrHfZQZrnbop45y3676x3qckT4Zm0KTmtY+9H vLF6OcfK9h5Oa+6SP2hU7U4oj70MjsqksekP7S+AaYkc1q9DqgDiHXy6HjUh9KQB x2QbGuLso9xUI2wyshWvQxoxztnyAyQvDysdjNqjHq3e0D71SRQh+82FG6ZV9xt5 tM5txH+Xv3USi0cBO+36WDGTPqq0hdbU5fzhq54PcHpPskEtKmP9qT7NgV/2pLYK kiob/l0n6pkabAtkpECtlElnlEueZi//cX1Ty66b7PHC8POl/O6sRC/iqKlBAMzH Uy/lYEKQ41b/1vZqXomZjhGCg5ZyH0pEJUppXlASlLx/3z0KGwECMKmC16bCtIBV 3giW9+Y579x49tDwNpMWVL4TLirjq/mVAnDiTLaXca+hA+xUNOEV8mabzl0mVLZf rIDdgQ/2gAP/O7hyZ4ZHo7N9Qw3eRw7LOUjonX4yO5+BuqW3H02Ph8RpWLctN2Ww guWsfnIi8DfHPDKg8Dug8PcZtSeDuTOF5QsPQT60n1axrxEBdRJ5whrjM8+Qupvx 5NUZPeT/0CFNCtM9nD0YWUpOcRzGVGPgOdMWndPOj6XPfJtIO3yNNEBwMFexVkdK FLZWCBDu7ZegoMk4LDj8e/rhE089sVc/7uWEX+SyXWNjsGa5Doc= =yIcc -----END PGP SIGNATURE----- --S8ONmfR55a5MhcRfJvPq2p5WNbyH9pvT9--