[gentoo-portage-dev] [PATCH] fetch(): fix support for digest size=None

[gentoo-portage-dev] [PATCH] fetch(): fix support for digest size=None

[Michał Górny]

It seems that the code initially supported fetching without size
'digest' but it got broken over time. Add proper conditionals to avoid
ugly failures in this case.
---
 pym/portage/checksum.py             | 2 +-
 pym/portage/package/ebuild/fetch.py | 7 ++++---
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py
index f24a90f..1efe74e 100644
--- a/pym/portage/checksum.py
+++ b/pym/portage/checksum.py
@@ -303,7 +303,7 @@ def verify_all(filename, mydict, calc_prelink=0, strict=0):
 	reason     = "Reason unknown"
 	try:
 		mysize = os.stat(filename)[stat.ST_SIZE]
-		if mydict["size"] != mysize:
+		if mydict["size"] is not None and mydict["size"] != mysize:
 			return False,(_("Filesize does not match recorded size"), mysize, mydict["size"])
 	except OSError as e:
 		if e.errno == errno.ENOENT:
diff --git a/pym/portage/package/ebuild/fetch.py b/pym/portage/package/ebuild/fetch.py
index 7b856a2..f60db9e 100644
--- a/pym/portage/package/ebuild/fetch.py
+++ b/pym/portage/package/ebuild/fetch.py
@@ -700,7 +700,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 								os.unlink(myfile_path)
 							except OSError:
 								pass
-					elif distdir_writable:
+					elif distdir_writable and size is not None:
 						if mystat.st_size < fetch_resume_size and \
 							mystat.st_size < size:
 							# If the file already exists and the size does not
@@ -806,8 +806,9 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 						# assume that it is fully downloaded.
 						continue
 					else:
-						if mystat.st_size < mydigests[myfile]["size"] and \
-							not restrict_fetch:
+						if (mydigests[myfile]["size"] is not None
+								and mystat.st_size < mydigests[myfile]["size"]
+								and not restrict_fetch):
 							fetched = 1 # Try to resume this download.
 						elif parallel_fetchonly and \
 							mystat.st_size == mydigests[myfile]["size"]:
-- 
2.3.5

Re: [gentoo-portage-dev] [PATCH] fetch(): fix support for digest size=None

[Zac Medico]

On 04/05/2015 12:59 AM, Michał Górny wrote:
> It seems that the code initially supported fetching without size
> 'digest' but it got broken over time. Add proper conditionals to avoid
> ugly failures in this case.

If we're handling this case, then we should probably not assume that the
dict contains a "size" key. So, we could check that dict.get("size") is
not None.
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] [PATCH] fetch(): fix support for digest size=None

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 665 bytes --]

Dnia 2015-04-05, o godz. 10:57:57
Zac Medico <zmedico@gentoo.org> napisał(a):

> On 04/05/2015 12:59 AM, Michał Górny wrote:
> > It seems that the code initially supported fetching without size
> > 'digest' but it got broken over time. Add proper conditionals to avoid
> > ugly failures in this case.
> 
> If we're handling this case, then we should probably not assume that the
> dict contains a "size" key. So, we could check that dict.get("size") is
> not None.

I agree. Could you take it from this point? I don't really have time to
do this, so I limit myself to solving the bugs I need to get moving :/.

-- 
Best regards,
Michał Górny

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

[gentoo-portage-dev] [PATCH v2] fetch(): fix support for digest size=None

[Zac Medico]

From: Michał Górny <mgorny@gentoo.org>

It seems that the code initially supported fetching without size
'digest' but it got broken over time. Add proper conditionals to avoid
ugly failures in this case.

Signed-off-by: Zac Medico <zmedico@gentoo.org>
---
[PATCH v2] uses dict.get() so that the "size" key is not required

https://github.com/zmedico/portage/tree/mgorny-fetch-size-missing

 pym/portage/checksum.py             | 2 +-
 pym/portage/package/ebuild/fetch.py | 7 ++++---
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py
index f24a90f..642602e 100644
--- a/pym/portage/checksum.py
+++ b/pym/portage/checksum.py
@@ -303,7 +303,7 @@ def verify_all(filename, mydict, calc_prelink=0, strict=0):
 	reason     = "Reason unknown"
 	try:
 		mysize = os.stat(filename)[stat.ST_SIZE]
-		if mydict["size"] != mysize:
+		if mydict.get("size") is not None and mydict["size"] != mysize:
 			return False,(_("Filesize does not match recorded size"), mysize, mydict["size"])
 	except OSError as e:
 		if e.errno == errno.ENOENT:
diff --git a/pym/portage/package/ebuild/fetch.py b/pym/portage/package/ebuild/fetch.py
index 7b856a2..7e4e6fe 100644
--- a/pym/portage/package/ebuild/fetch.py
+++ b/pym/portage/package/ebuild/fetch.py
@@ -700,7 +700,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 								os.unlink(myfile_path)
 							except OSError:
 								pass
-					elif distdir_writable:
+					elif distdir_writable and size is not None:
 						if mystat.st_size < fetch_resume_size and \
 							mystat.st_size < size:
 							# If the file already exists and the size does not
@@ -806,8 +806,9 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 						# assume that it is fully downloaded.
 						continue
 					else:
-						if mystat.st_size < mydigests[myfile]["size"] and \
-							not restrict_fetch:
+						if (mydigests[myfile].get("size") is not None
+								and mystat.st_size < mydigests[myfile]["size"]
+								and not restrict_fetch):
 							fetched = 1 # Try to resume this download.
 						elif parallel_fetchonly and \
 							mystat.st_size == mydigests[myfile]["size"]:
-- 
2.3.1

Re: [gentoo-portage-dev] [PATCH v2] fetch(): fix support for digest size=None

[Brian Dolbec]

On Sun,  5 Apr 2015 21:47:12 -0700
Zac Medico <zmedico@gentoo.org> wrote:

> From: Michał Górny <mgorny@gentoo.org>
> 
> It seems that the code initially supported fetching without size
> 'digest' but it got broken over time. Add proper conditionals to avoid
> ugly failures in this case.
> 
> Signed-off-by: Zac Medico <zmedico@gentoo.org>
> ---
> [PATCH v2] uses dict.get() so that the "size" key is not required
> 
> https://github.com/zmedico/portage/tree/mgorny-fetch-size-missing
> 
>  pym/portage/checksum.py             | 2 +-
>  pym/portage/package/ebuild/fetch.py | 7 ++++---
>  2 files changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py
> index f24a90f..642602e 100644
> --- a/pym/portage/checksum.py
> +++ b/pym/portage/checksum.py
> @@ -303,7 +303,7 @@ def verify_all(filename, mydict, calc_prelink=0,
> strict=0): reason     = "Reason unknown"
>  	try:
>  		mysize = os.stat(filename)[stat.ST_SIZE]
> -		if mydict["size"] != mysize:
> +		if mydict.get("size") is not None and
> mydict["size"] != mysize: return False,(_("Filesize does not match
> recorded size"), mysize, mydict["size"]) except OSError as e:
>  		if e.errno == errno.ENOENT:
> diff --git a/pym/portage/package/ebuild/fetch.py
> b/pym/portage/package/ebuild/fetch.py index 7b856a2..7e4e6fe 100644
> --- a/pym/portage/package/ebuild/fetch.py
> +++ b/pym/portage/package/ebuild/fetch.py
> @@ -700,7 +700,7 @@ def fetch(myuris, mysettings, listonly=0,
> fetchonly=0, os.unlink(myfile_path)
>  							except
> OSError: pass
> -					elif distdir_writable:
> +					elif distdir_writable and
> size is not None: if mystat.st_size < fetch_resume_size and \
>  							mystat.st_size
> < size: # If the file already exists and the size does not
> @@ -806,8 +806,9 @@ def fetch(myuris, mysettings, listonly=0,
> fetchonly=0, # assume that it is fully downloaded.
>  						continue
>  					else:
> -						if mystat.st_size <
> mydigests[myfile]["size"] and \
> -							not
> restrict_fetch:
> +						if
> (mydigests[myfile].get("size") is not None
> +								and
> mystat.st_size < mydigests[myfile]["size"]
> +								and
> not restrict_fetch): fetched = 1 # Try to resume this download.
>  						elif
> parallel_fetchonly and \ mystat.st_size == mydigests[myfile]["size"]:

OK :) merge please

-- 
Brian Dolbec <dolsen>