* [gentoo-portage-dev] [PATCH 0/3] Implement a more verbose User-Agent HTTP-header @ 2013-08-14 20:10 Mark Kubacki 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 1/3] Send exact version with " Mark Kubacki ` (2 more replies) 0 siblings, 3 replies; 10+ messages in thread From: Mark Kubacki @ 2013-08-14 20:10 UTC (permalink / raw To: gentoo-portage-dev; +Cc: W-Mark Kubacki From: W-Mark Kubacki <wmark@hurrikane.de> Before this patch series Portage identified itself as > Gentoo Portage Now it will be the output of ```emerge --version```, i.e.: > Gentoo Portage 2.1.13.7 (default/linux/amd64/13.0, gcc-4.6.2, > glibc-2.18, 3.9.0-rc8-mark-signed+ x86_64, > Intel(R) Core(TM) i7-3770T CPU @ 2.50GHz) That longer string makes debugging easier and helps deciding when to deploy changes on the Gentoo binhost's side. W-Mark Kubacki (3): Send exact version with User-Agent HTTP-header Send output of ```emerge --version``` as User-Agent HTTP-header Add CPU model name to output of getportageversion as fifth element pym/_emerge/actions.py | 4 +++- pym/portage/dbapi/bintree.py | 8 +++++++- pym/portage/util/_urlopen.py | 5 +++-- 3 files changed, 13 insertions(+), 4 deletions(-) -- 1.8.3.2 ^ permalink raw reply [flat|nested] 10+ messages in thread
* [gentoo-portage-dev] [PATCH 1/3] Send exact version with User-Agent HTTP-header 2013-08-14 20:10 [gentoo-portage-dev] [PATCH 0/3] Implement a more verbose User-Agent HTTP-header Mark Kubacki @ 2013-08-14 20:10 ` Mark Kubacki 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 2/3] Send output of ```emerge --version``` as " Mark Kubacki 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element Mark Kubacki 2 siblings, 0 replies; 10+ messages in thread From: Mark Kubacki @ 2013-08-14 20:10 UTC (permalink / raw To: gentoo-portage-dev; +Cc: W-Mark Kubacki From: W-Mark Kubacki <wmark@hurrikane.de> Signed-off-by: W-Mark Kubacki <wmark@hurrikane.de> --- pym/portage/util/_urlopen.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/pym/portage/util/_urlopen.py b/pym/portage/util/_urlopen.py index 768ccb8..798e7b4 100644 --- a/pym/portage/util/_urlopen.py +++ b/pym/portage/util/_urlopen.py @@ -6,6 +6,7 @@ import sys from datetime import datetime from time import mktime from email.utils import formatdate, parsedate +from portage import VERSION try: from urllib.request import urlopen as _urlopen @@ -34,7 +35,7 @@ def urlopen(url, if_modified_since=None): url = urllib_parse.urlunparse((parse_result.scheme, netloc, parse_result.path, parse_result.params, parse_result.query, parse_result.fragment)) password_manager = urllib_request.HTTPPasswordMgrWithDefaultRealm() request = urllib_request.Request(url) - request.add_header('User-Agent', 'Gentoo Portage') + request.add_header('User-Agent', 'Gentoo Portage '+VERSION) if if_modified_since: request.add_header('If-Modified-Since', _timestamp_to_http(if_modified_since)) if parse_result.username is not None: -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [gentoo-portage-dev] [PATCH 2/3] Send output of ```emerge --version``` as User-Agent HTTP-header 2013-08-14 20:10 [gentoo-portage-dev] [PATCH 0/3] Implement a more verbose User-Agent HTTP-header Mark Kubacki 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 1/3] Send exact version with " Mark Kubacki @ 2013-08-14 20:10 ` Mark Kubacki 2013-08-15 6:25 ` Zac Medico 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element Mark Kubacki 2 siblings, 1 reply; 10+ messages in thread From: Mark Kubacki @ 2013-08-14 20:10 UTC (permalink / raw To: gentoo-portage-dev; +Cc: W-Mark Kubacki From: W-Mark Kubacki <wmark@hurrikane.de> The remote server logs will read like this: 2001:db8::4 - - [14/Aug/2013:20:58:02 +0200] "GET /Packages HTTP/1.1" 304 0 "-" "Gentoo Portage 2.1.13.7 (default/linux/amd64/13.0, gcc-4.6.2, glibc-2.18, 3.9.0-rc8-mark-signed+ x86_64)" This will enable administrators to… • check if a machine uses the correct binhost • decide when new server-side features can be enabled (Portage ver.) • notify the machine's administrator of malconfigurations (such as wrong GCC libraries, profile, obsolete kernel) Construction of this header happens before "Packages" are fetched from a remote binhost. Signed-off-by: W-Mark Kubacki <wmark@hurrikane.de> --- pym/portage/dbapi/bintree.py | 8 +++++++- pym/portage/util/_urlopen.py | 4 ++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/pym/portage/dbapi/bintree.py b/pym/portage/dbapi/bintree.py index 61ac6b5..44ee8e6 100644 --- a/pym/portage/dbapi/bintree.py +++ b/pym/portage/dbapi/bintree.py @@ -20,6 +20,7 @@ portage.proxy.lazyimport.lazyimport(globals(), 'portage.util.listdir:listdir', 'portage.util._urlopen:urlopen@_urlopen', 'portage.versions:best,catpkgsplit,catsplit,_pkg_str', + '_emerge.actions:load_emerge_config,getportageversion', ) from portage.cache.mappings import slot_dict_class @@ -892,8 +893,13 @@ class binarytree(object): # Don't use urlopen for https, since it doesn't support # certificate/hostname verification (bug #469888). if parsed_url.scheme not in ('https',): + trees = load_emerge_config().trees + user_agent = "Gentoo "+getportageversion(self.settings["PORTDIR"], None, + self.settings.profile_path, self.settings["CHOST"], + trees[self.settings['EROOT']]["vartree"].dbapi) try: - f = _urlopen(url, if_modified_since=local_timestamp) + f = _urlopen(url, user_agent=user_agent, + if_modified_since=local_timestamp) if hasattr(f, 'headers') and f.headers.get('timestamp', ''): remote_timestamp = f.headers.get('timestamp') except IOError as err: diff --git a/pym/portage/util/_urlopen.py b/pym/portage/util/_urlopen.py index 798e7b4..9876f29 100644 --- a/pym/portage/util/_urlopen.py +++ b/pym/portage/util/_urlopen.py @@ -26,7 +26,7 @@ if sys.hexversion >= 0x3000000: # and the file-'mtime' TIMESTAMP_TOLERANCE=5 -def urlopen(url, if_modified_since=None): +def urlopen(url, user_agent=None, if_modified_since=None): parse_result = urllib_parse.urlparse(url) if parse_result.scheme not in ("http", "https"): return _urlopen(url) @@ -35,7 +35,7 @@ def urlopen(url, if_modified_since=None): url = urllib_parse.urlunparse((parse_result.scheme, netloc, parse_result.path, parse_result.params, parse_result.query, parse_result.fragment)) password_manager = urllib_request.HTTPPasswordMgrWithDefaultRealm() request = urllib_request.Request(url) - request.add_header('User-Agent', 'Gentoo Portage '+VERSION) + request.add_header('User-Agent', user_agent or 'Gentoo Portage '+VERSION) if if_modified_since: request.add_header('If-Modified-Since', _timestamp_to_http(if_modified_since)) if parse_result.username is not None: -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [gentoo-portage-dev] [PATCH 2/3] Send output of ```emerge --version``` as User-Agent HTTP-header 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 2/3] Send output of ```emerge --version``` as " Mark Kubacki @ 2013-08-15 6:25 ` Zac Medico 0 siblings, 0 replies; 10+ messages in thread From: Zac Medico @ 2013-08-15 6:25 UTC (permalink / raw To: gentoo-portage-dev, Mark Kubacki On 08/14/2013 01:10 PM, Mark Kubacki wrote: > @@ -892,8 +893,13 @@ class binarytree(object): > # Don't use urlopen for https, since it doesn't support > # certificate/hostname verification (bug #469888). > if parsed_url.scheme not in ('https',): > + trees = load_emerge_config().trees > + user_agent = "Gentoo "+getportageversion(self.settings["PORTDIR"], None, > + self.settings.profile_path, self.settings["CHOST"], > + trees[self.settings['EROOT']]["vartree"].dbapi) Generally, your patches look reasonable. However, it's a waste to call load_emerge_config() here, since the caller has certainly loaded the config already. I guess we could have the caller pass the result of getportageversion() as an argument to the binarytree.populate() method. -- Thanks, Zac ^ permalink raw reply [flat|nested] 10+ messages in thread
* [gentoo-portage-dev] [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element 2013-08-14 20:10 [gentoo-portage-dev] [PATCH 0/3] Implement a more verbose User-Agent HTTP-header Mark Kubacki 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 1/3] Send exact version with " Mark Kubacki 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 2/3] Send output of ```emerge --version``` as " Mark Kubacki @ 2013-08-14 20:10 ` Mark Kubacki 2013-08-14 20:27 ` [gentoo-portage-dev] " Mark Kubacki ` (2 more replies) 2 siblings, 3 replies; 10+ messages in thread From: Mark Kubacki @ 2013-08-14 20:10 UTC (permalink / raw To: gentoo-portage-dev; +Cc: W-Mark Kubacki From: W-Mark Kubacki <wmark@hurrikane.de> It will read like this: > Portage 2.1.13.7 (default/linux/amd64/13.0, gcc-4.6.2, glibc-2.18, > 3.9.0-rc8-mark-signed+ x86_64, Intel(R) Core(TM) i7-3770T CPU @ 2.50GHz) That new fifth element will be the CPU model name of the host running Portage. It is not the target architecture! The former output is not sufficient to tell if a machine has been downgraded (i.e. 3rd gen. Core i7 running a configuration for x86 "Pentium Celeron") or to distinguish between i.e. ARM CPUs ("arm5tel" could be a lot, including incompatible instruction sets). Signed-off-by: W-Mark Kubacki <wmark@hurrikane.de> --- pym/_emerge/actions.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/pym/_emerge/actions.py b/pym/_emerge/actions.py index 52ceba4..e1873cd 100644 --- a/pym/_emerge/actions.py +++ b/pym/_emerge/actions.py @@ -3017,9 +3017,11 @@ def getportageversion(portdir, _unused, profile, chost, vardb): gccver = getgccversion(chost) unameout=platform.release()+" "+platform.machine() + cpu_model_name=platform.processor() return "Portage %s (%s, %s, %s, %s)" % \ - (portage.VERSION, profilever, gccver, ",".join(libcver), unameout) + (portage.VERSION, profilever, gccver, ",".join(libcver), unameout, + cpu_model_name) def git_sync_timestamps(portdb, portdir): """ -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [gentoo-portage-dev] Re: [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element Mark Kubacki @ 2013-08-14 20:27 ` Mark Kubacki 2013-08-15 8:00 ` [gentoo-portage-dev] " Fabian Groffen 2013-08-15 10:58 ` Alexander Berntsen 2 siblings, 0 replies; 10+ messages in thread From: Mark Kubacki @ 2013-08-14 20:27 UTC (permalink / raw To: gentoo-portage-dev [-- Attachment #1: Type: text/plain, Size: 354 bytes --] On 2013-08-14 22:10, Mark Kubacki wrote: > > return "Portage %s (%s, %s, %s, %s)" % \ > - (portage.VERSION, profilever, gccver, ",".join(libcver), unameout) > + (portage.VERSION, profilever, gccver, ",".join(libcver), unameout, > + cpu_model_name) That one is missing a %s: + return "Portage %s (%s, %s, %s, %s, %s)" % \ -- Mark [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4339 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [gentoo-portage-dev] [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element Mark Kubacki 2013-08-14 20:27 ` [gentoo-portage-dev] " Mark Kubacki @ 2013-08-15 8:00 ` Fabian Groffen 2013-08-24 22:39 ` Mike Frysinger 2013-08-15 10:58 ` Alexander Berntsen 2 siblings, 1 reply; 10+ messages in thread From: Fabian Groffen @ 2013-08-15 8:00 UTC (permalink / raw To: gentoo-portage-dev; +Cc: W-Mark Kubacki [-- Attachment #1: Type: text/plain, Size: 1863 bytes --] On 14-08-2013 22:10:40 +0200, Mark Kubacki wrote: > From: W-Mark Kubacki <wmark@hurrikane.de> > > It will read like this: > > Portage 2.1.13.7 (default/linux/amd64/13.0, gcc-4.6.2, glibc-2.18, > > 3.9.0-rc8-mark-signed+ x86_64, Intel(R) Core(TM) i7-3770T CPU @ 2.50GHz) > > That new fifth element will be the CPU model name of the host > running Portage. It is not the target architecture! FYI: % python3 Python 3.3.2 (default, Jul 24 2013, 11:14:02) [GCC 4.2.1 (Gentoo 4.2.1_p5666-r1, Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import platform >>> platform.release() '12.4.0' >>> platform.machine() 'x86_64' >>> platform.processor() 'i386' >>> % python Python 3.2.3 (default, May 6 2013, 21:19:05) [GCC 4.7.2] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import platform >>> platform.release() '5.11' >>> platform.machine() 'i86pc' >>> platform.processor() 'i386' >>> % python Python 3.3.2 (default, Jul 15 2013, 13:51:24) [GCC 4.7.2] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import platform >>> platform.release() '5.10' >>> platform.machine() 'sun4u' >>> platform.processor() 'sparc' >>> % python Python 3.2.5 (default, Jul 15 2013, 11:37:08) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import platform >>> platform.release() '3.8.13-gentoo' >>> platform.machine() 'x86_64' >>> platform.processor() 'AMD Athlon(tm) 64 X2 Dual Core Processor 3800+' >>> e.g. it seems to me only on Linux it gives fancy model output. Note that the first and second system were running a 64-bit Python as well as kernel. Fabian -- Fabian Groffen Gentoo on a different level [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 194 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [gentoo-portage-dev] [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element 2013-08-15 8:00 ` [gentoo-portage-dev] " Fabian Groffen @ 2013-08-24 22:39 ` Mike Frysinger 0 siblings, 0 replies; 10+ messages in thread From: Mike Frysinger @ 2013-08-24 22:39 UTC (permalink / raw To: gentoo-portage-dev; +Cc: Fabian Groffen, W-Mark Kubacki [-- Attachment #1: Type: Text/Plain, Size: 357 bytes --] On Thursday 15 August 2013 04:00:57 Fabian Groffen wrote: > e.g. it seems to me only on Linux it gives fancy model output. Note > that the first and second system were running a 64-bit Python as well as > kernel. output from the uname() syscall/C lib func is bare. the `uname` program in Gentoo contains a patch to get useful information. -mike [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [gentoo-portage-dev] [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element Mark Kubacki 2013-08-14 20:27 ` [gentoo-portage-dev] " Mark Kubacki 2013-08-15 8:00 ` [gentoo-portage-dev] " Fabian Groffen @ 2013-08-15 10:58 ` Alexander Berntsen 2013-08-24 22:38 ` Mike Frysinger 2 siblings, 1 reply; 10+ messages in thread From: Alexander Berntsen @ 2013-08-15 10:58 UTC (permalink / raw To: gentoo-portage-dev -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 (This holds for all patches) Please surround '+' with spaces[0]. Please ensure <79 width[1]. Please indent continuation lines properly[2]. [0] <http://www.python.org/dev/peps/pep-0008/#id19> [1] <http://www.python.org/dev/peps/pep-0008/#id13> [2] <http://www.python.org/dev/peps/pep-0008/#id11>, look at foo = long_function_name(). - -- Alexander alexander@plaimi.net http://plaimi.net/~alexander -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlIMtFAACgkQRtClrXBQc7UFFgD/dhU6OgwtRqF7fZ81QbC6l//V b+qNKFwGkiv8uyqzzUUBAK9Xc/w4iR40R61YXEySV+ybQ5GbrlvyjgHMzUCyobAs =nYLt -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [gentoo-portage-dev] [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element 2013-08-15 10:58 ` Alexander Berntsen @ 2013-08-24 22:38 ` Mike Frysinger 0 siblings, 0 replies; 10+ messages in thread From: Mike Frysinger @ 2013-08-24 22:38 UTC (permalink / raw To: gentoo-portage-dev; +Cc: Alexander Berntsen [-- Attachment #1: Type: Text/Plain, Size: 217 bytes --] On Thursday 15 August 2013 06:58:24 Alexander Berntsen wrote: > (This holds for all patches) > > Please surround '+' with spaces[0]. same for assignments (not to be confused with keyword args to funcs) -mike [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2013-08-24 22:39 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-08-14 20:10 [gentoo-portage-dev] [PATCH 0/3] Implement a more verbose User-Agent HTTP-header Mark Kubacki 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 1/3] Send exact version with " Mark Kubacki 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 2/3] Send output of ```emerge --version``` as " Mark Kubacki 2013-08-15 6:25 ` Zac Medico 2013-08-14 20:10 ` [gentoo-portage-dev] [PATCH 3/3] Add CPU model name to output of getportageversion as fifth element Mark Kubacki 2013-08-14 20:27 ` [gentoo-portage-dev] " Mark Kubacki 2013-08-15 8:00 ` [gentoo-portage-dev] " Fabian Groffen 2013-08-24 22:39 ` Mike Frysinger 2013-08-15 10:58 ` Alexander Berntsen 2013-08-24 22:38 ` Mike Frysinger
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox