[gentoo-portage-dev] [PATCH] If a binhost file sets a TTL header, honor it.

[gentoo-portage-dev] [PATCH] If a binhost file sets a TTL header, honor it.

[Zac Medico]

From 5400ba6ecbccf946aa4d5a8ddaaa2e1d7b784d3f Mon Sep 17 00:00:00 2001
From: David James <davidjames@google.com>
Date: Thu, 23 Oct 2014 18:40:28 -0700
Subject: [PATCH] If a binhost file sets a TTL header, honor it.

BUG=chromium:381970
TEST=Run it locally and verify TTL headers are respected.

Change-Id: I54b2afa6fefdb3de1b9a03aaa3af37fd587e1a11
Reviewed-on: https://chromium-review.googlesource.com/225279
Reviewed-by: Zac Medico <zmedico@gmail.com>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
---
 pym/portage/dbapi/bintree.py | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/pym/portage/dbapi/bintree.py b/pym/portage/dbapi/bintree.py
index 229ce3b..e6eae33 100644
--- a/pym/portage/dbapi/bintree.py
+++ b/pym/portage/dbapi/bintree.py
@@ -43,6 +43,7 @@ import subprocess
 import sys
 import tempfile
 import textwrap
+import time
 import traceback
 import warnings
 from gzip import GzipFile
@@ -879,11 +880,16 @@ class binarytree(object):
 				if e.errno != errno.ENOENT:
 					raise
 			local_timestamp = pkgindex.header.get("TIMESTAMP", None)
+			download_timestamp = float(pkgindex.header.get("DOWNLOAD_TIMESTAMP", 0))
 			remote_timestamp = None
 			rmt_idx = self._new_pkgindex()
 			proc = None
 			tmp_filename = None
 			try:
+				ttl = float(pkgindex.header.get("TTL", 0))
+				if download_timestamp and ttl and download_timestamp + ttl > time.time():
+					raise UseCachedCopyOfRemoteIndex()
+
 				# urlparse.urljoin() only works correctly with recognized
 				# protocols and requires the base url to have a trailing
 				# slash, so join manually...
@@ -1022,6 +1028,7 @@ class binarytree(object):
 					pass
 			if pkgindex is rmt_idx:
 				pkgindex.modified = False # don't update the header
+				pkgindex.header["DOWNLOAD_TIMESTAMP"] = str(long(time.time()))
 				try:
 					ensure_dirs(os.path.dirname(pkgindex_file))
 					f = atomic_ofstream(pkgindex_file)
-- 
2.0.4

[gentoo-portage-dev] Re: [PATCH] If a binhost file sets a TTL header, honor it.

[Zac Medico]

On 10/24/2014 12:24 PM, Zac Medico wrote:
> From 5400ba6ecbccf946aa4d5a8ddaaa2e1d7b784d3f Mon Sep 17 00:00:00 2001
> From: David James <davidjames@google.com>
> Date: Thu, 23 Oct 2014 18:40:28 -0700
> Subject: [PATCH] If a binhost file sets a TTL header, honor it.

The first patch needs a small tweak to avoid an UnboundLocalError when
UseCachedCopyOfRemoteIndex is raised:

From 9b31dff37f26b5114a6b2d9ad972b98947e43a53 Mon Sep 17 00:00:00 2001
From: David James <davidjames@google.com>
Date: Fri, 24 Oct 2014 12:40:30 -0700
Subject: [PATCH] Set "url" before marking the index as up to date.

Fixes UnboundLocalError: local variable 'url' referenced before
assignment.

BUG=chromium:381970
TEST=Run it locally.

Change-Id: I592bfcac4b27da6d95ae40f5f05bc31eb571702f
Reviewed-on: https://chromium-review.googlesource.com/225423
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
---
 pym/portage/dbapi/bintree.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/pym/portage/dbapi/bintree.py b/pym/portage/dbapi/bintree.py
index e6eae33..844d015 100644
--- a/pym/portage/dbapi/bintree.py
+++ b/pym/portage/dbapi/bintree.py
@@ -886,16 +886,16 @@ class binarytree(object):
 			proc = None
 			tmp_filename = None
 			try:
-				ttl = float(pkgindex.header.get("TTL", 0))
-				if download_timestamp and ttl and download_timestamp + ttl > time.time():
-					raise UseCachedCopyOfRemoteIndex()
-
 				# urlparse.urljoin() only works correctly with recognized
 				# protocols and requires the base url to have a trailing
 				# slash, so join manually...
 				url = base_url.rstrip("/") + "/Packages"
 				f = None
 
+				ttl = float(pkgindex.header.get("TTL", 0))
+				if download_timestamp and ttl and download_timestamp + ttl > time.time():
+					raise UseCachedCopyOfRemoteIndex()
+
 				# Don't use urlopen for https, since it doesn't support
 				# certificate/hostname verification (bug #469888).
 				if parsed_url.scheme not in ('https',):
-- 
2.0.4

Re: [gentoo-portage-dev] Re: [PATCH] If a binhost file sets a TTL header, honor it.

[Brian Dolbec]

On Fri, 24 Oct 2014 14:15:29 -0700
Zac Medico <zmedico@gentoo.org> wrote:

> On 10/24/2014 12:24 PM, Zac Medico wrote:
> > From 5400ba6ecbccf946aa4d5a8ddaaa2e1d7b784d3f Mon Sep 17 00:00:00
> > 2001 From: David James <davidjames@google.com>
> > Date: Thu, 23 Oct 2014 18:40:28 -0700
> > Subject: [PATCH] If a binhost file sets a TTL header, honor it.
> 
> The first patch needs a small tweak to avoid an UnboundLocalError when
> UseCachedCopyOfRemoteIndex is raised:
> 
> From 9b31dff37f26b5114a6b2d9ad972b98947e43a53 Mon Sep 17 00:00:00 2001
> From: David James <davidjames@google.com>
> Date: Fri, 24 Oct 2014 12:40:30 -0700
> Subject: [PATCH] Set "url" before marking the index as up to date.
> 
> Fixes UnboundLocalError: local variable 'url' referenced before
> assignment.
> 
> BUG=chromium:381970
> TEST=Run it locally.
> 
> Change-Id: I592bfcac4b27da6d95ae40f5f05bc31eb571702f
> Reviewed-on: https://chromium-review.googlesource.com/225423
> Tested-by: David James <davidjames@chromium.org>
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> ---
>  pym/portage/dbapi/bintree.py | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 

Squash the 2 commits together and commit please
-- 
Brian Dolbec <dolsen>