[gentoo-catalyst] [PATCH] Support snapshot.tar.xz format

[gentoo-catalyst] [PATCH] Support snapshot.tar.xz format

[Rick Farina]

This patch adds in support for automatically detecting and decompressing
both formats gentoo currently ships portage snapshots in. To make the
change as minimal as possible I am using recent gnu/bsd tar ability to
automatically pick the correct format and decompress. This should work
on all gentoo supported platforms including linux, freebsd, and
prefix-macosx.

Signed-off-by: Rick Farina <sidhayn@gmail.com>
---
 modules/generic_stage_target.py |   18 +++++++++++++-----
 1 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/modules/generic_stage_target.py b/modules/generic_stage_target.py
index c4ef239..b971741 100644
--- a/modules/generic_stage_target.py
+++ b/modules/generic_stage_target.py
@@ -404,12 +404,20 @@ class generic_stage_target(generic_target):
 
 	def set_snapshot_path(self):
 		self.settings["snapshot_path"]=normpath(self.settings["storedir"]+\
-			"/snapshots/portage-"+self.settings["snapshot"]+".tar.bz2")
+			"/snapshots/portage-"+self.settings["snapshot"]+".tar.xz")
 
 		if os.path.exists(self.settings["snapshot_path"]):
 			self.settings["snapshot_path_hash"]=\
 				generate_hash(self.settings["snapshot_path"],\
 				hash_function=self.settings["hash_function"],verbose=False)
+		else:
+			self.settings["snapshot_path"]=normpath(self.settings["storedir"]+\
+				"/snapshots/portage-"+self.settings["snapshot"]+".tar.bz2")
+
+			if os.path.exists(self.settings["snapshot_path"]):
+				self.settings["snapshot_path_hash"]=\
+					generate_hash(self.settings["snapshot_path"],\
+					hash_function=self.settings["hash_function"],verbose=False)
 
 	def set_snapcache_path(self):
 		if self.settings.has_key("SNAPCACHE"):
@@ -635,7 +643,7 @@ class generic_stage_target(generic_target):
 					self.settings["source_path"]+"\nto "+\
 					self.settings["chroot_path"]+\
 						" (This may take some time) ...\n"
-				unpack_cmd="tar xjpf "+self.settings["source_path"]+" -C "+\
+				unpack_cmd="tar xpf "+self.settings["source_path"]+" -C "+\
 					self.settings["chroot_path"]
 				error_msg="Tarball extraction of "+\
 					self.settings["source_path"]+" to "+\
@@ -646,7 +654,7 @@ class generic_stage_target(generic_target):
 				self.settings["source_path"]+"\nto "+\
 				self.settings["chroot_path"]+\
 				" (This may take some time) ...\n"
-			unpack_cmd="tar xjpf "+self.settings["source_path"]+" -C "+\
+			unpack_cmd="tar xpf "+self.settings["source_path"]+" -C "+\
 				self.settings["chroot_path"]
 			error_msg="Tarball extraction of "+self.settings["source_path"]+\
 				" to "+self.settings["chroot_path"]+" failed."
@@ -744,7 +752,7 @@ class generic_stage_target(generic_target):
 				read_from_clst(self.settings["snapshot_cache_path"]+\
 				"catalyst-hash")
 			destdir=self.settings["snapshot_cache_path"]
-			unpack_cmd="tar xjpf "+self.settings["snapshot_path"]+" -C "+destdir
+			unpack_cmd="tar xpf "+self.settings["snapshot_path"]+" -C "+destdir
 			unpack_errmsg="Error unpacking snapshot"
 			cleanup_msg="Cleaning up invalid snapshot cache at \n\t"+\
 				self.settings["snapshot_cache_path"]+\
@@ -760,7 +768,7 @@ class generic_stage_target(generic_target):
 			cleanup_errmsg="Error removing existing snapshot directory."
 			cleanup_msg=\
 				"Cleaning up existing portage tree (This can take a long time)..."
-			unpack_cmd="tar xjpf "+self.settings["snapshot_path"]+" -C "+\
+			unpack_cmd="tar xpf "+self.settings["snapshot_path"]+" -C "+\
 				self.settings["chroot_path"]+"/usr"
 			unpack_errmsg="Error unpacking snapshot"
 
-- 
1.7.8.6

[gentoo-catalyst] Re: [PATCH] Support snapshot.tar.xz format

[Richard Farina]

This version of the patch addresses the concerns from Matt (of me
failing to copy and paste everything) as well as updates the commit
message to show that I have no verified this should work on all gentoo
supported platforms.

Please accept.

thanks,
Zero

On 08/22/2012 12:12 PM, Rick Farina wrote:
> This patch adds in support for automatically detecting and decompressing
> both formats gentoo currently ships portage snapshots in. To make the
> change as minimal as possible I am using recent gnu/bsd tar ability to
> automatically pick the correct format and decompress. This should work
> on all gentoo supported platforms including linux, freebsd, and
> prefix-macosx.
> 
> Signed-off-by: Rick Farina <sidhayn@gmail.com>
> ---
>  modules/generic_stage_target.py |   18 +++++++++++++-----
>  1 files changed, 13 insertions(+), 5 deletions(-)
> 
> diff --git a/modules/generic_stage_target.py b/modules/generic_stage_target.py
> index c4ef239..b971741 100644
> --- a/modules/generic_stage_target.py
> +++ b/modules/generic_stage_target.py
> @@ -404,12 +404,20 @@ class generic_stage_target(generic_target):
>  
>  	def set_snapshot_path(self):
>  		self.settings["snapshot_path"]=normpath(self.settings["storedir"]+\
> -			"/snapshots/portage-"+self.settings["snapshot"]+".tar.bz2")
> +			"/snapshots/portage-"+self.settings["snapshot"]+".tar.xz")
>  
>  		if os.path.exists(self.settings["snapshot_path"]):
>  			self.settings["snapshot_path_hash"]=\
>  				generate_hash(self.settings["snapshot_path"],\
>  				hash_function=self.settings["hash_function"],verbose=False)
> +		else:
> +			self.settings["snapshot_path"]=normpath(self.settings["storedir"]+\
> +				"/snapshots/portage-"+self.settings["snapshot"]+".tar.bz2")
> +
> +			if os.path.exists(self.settings["snapshot_path"]):
> +				self.settings["snapshot_path_hash"]=\
> +					generate_hash(self.settings["snapshot_path"],\
> +					hash_function=self.settings["hash_function"],verbose=False)
>  
>  	def set_snapcache_path(self):
>  		if self.settings.has_key("SNAPCACHE"):
> @@ -635,7 +643,7 @@ class generic_stage_target(generic_target):
>  					self.settings["source_path"]+"\nto "+\
>  					self.settings["chroot_path"]+\
>  						" (This may take some time) ...\n"
> -				unpack_cmd="tar xjpf "+self.settings["source_path"]+" -C "+\
> +				unpack_cmd="tar xpf "+self.settings["source_path"]+" -C "+\
>  					self.settings["chroot_path"]
>  				error_msg="Tarball extraction of "+\
>  					self.settings["source_path"]+" to "+\
> @@ -646,7 +654,7 @@ class generic_stage_target(generic_target):
>  				self.settings["source_path"]+"\nto "+\
>  				self.settings["chroot_path"]+\
>  				" (This may take some time) ...\n"
> -			unpack_cmd="tar xjpf "+self.settings["source_path"]+" -C "+\
> +			unpack_cmd="tar xpf "+self.settings["source_path"]+" -C "+\
>  				self.settings["chroot_path"]
>  			error_msg="Tarball extraction of "+self.settings["source_path"]+\
>  				" to "+self.settings["chroot_path"]+" failed."
> @@ -744,7 +752,7 @@ class generic_stage_target(generic_target):
>  				read_from_clst(self.settings["snapshot_cache_path"]+\
>  				"catalyst-hash")
>  			destdir=self.settings["snapshot_cache_path"]
> -			unpack_cmd="tar xjpf "+self.settings["snapshot_path"]+" -C "+destdir
> +			unpack_cmd="tar xpf "+self.settings["snapshot_path"]+" -C "+destdir
>  			unpack_errmsg="Error unpacking snapshot"
>  			cleanup_msg="Cleaning up invalid snapshot cache at \n\t"+\
>  				self.settings["snapshot_cache_path"]+\
> @@ -760,7 +768,7 @@ class generic_stage_target(generic_target):
>  			cleanup_errmsg="Error removing existing snapshot directory."
>  			cleanup_msg=\
>  				"Cleaning up existing portage tree (This can take a long time)..."
> -			unpack_cmd="tar xjpf "+self.settings["snapshot_path"]+" -C "+\
> +			unpack_cmd="tar xpf "+self.settings["snapshot_path"]+" -C "+\
>  				self.settings["chroot_path"]+"/usr"
>  			unpack_errmsg="Error unpacking snapshot"
>  
> 

Re: [gentoo-catalyst] Re: [PATCH] Support snapshot.tar.xz format

[Sebastian Pipping]

On 08/22/2012 06:13 PM, Richard Farina wrote:
> This version of the patch addresses the concerns from Matt (of me
> failing to copy and paste everything) as well as updates the commit
> message to show that I have no verified this should work on all gentoo
> supported platforms.
> 
> Please accept.

Quoted patches are no fun.  Attach the patch or make the whole mail work
with git-am, please please please.

To be fair, I should mention that I hav no capacities for testing this
myself these days.  Hopefully someone else does.  Increased chances for
easy patch access ;-)

Best,



Sebastian

Re: [gentoo-catalyst] Re: [PATCH] Support snapshot.tar.xz format

[Sebastian Pipping]

On 08/24/2012 07:48 PM, Sebastian Pipping wrote:
> On 08/22/2012 06:13 PM, Richard Farina wrote:
>> This version of the patch addresses the concerns from Matt (of me
>> failing to copy and paste everything) as well as updates the commit
>> message to show that I have no verified this should work on all gentoo
>> supported platforms.
>>
>> Please accept.
> 
> Quoted patches are no fun.  Attach the patch or make the whole mail work
> with git-am, please please please.

Ignore my conmplaint if the patch was identical to the previous version
and you were just adding another note.

Best,




Sebastian

Re: [gentoo-catalyst] Re: [PATCH] Support snapshot.tar.xz format

[Richard Farina]

On 08/24/2012 01:48 PM, Sebastian Pipping wrote:
> On 08/22/2012 06:13 PM, Richard Farina wrote:
>> This version of the patch addresses the concerns from Matt (of me
>> failing to copy and paste everything) as well as updates the commit
>> message to show that I have no verified this should work on all gentoo
>> supported platforms.
>>
>> Please accept.
> 
> Quoted patches are no fun.  Attach the patch or make the whole mail work
> with git-am, please please please.

I specifically sent the patch using git-email then replied to my post to
add more detail than the commit message. This is the process I use on
kernel mailing lists and never had a complaint.  If this is not
acceptable please let me know, but I have a feeling that you read my
reply and not the first email that came through....
> 
> To be fair, I should mention that I hav no capacities for testing this
> myself these days.  Hopefully someone else does.  Increased chances for
> easy patch access ;-)

Obviously I have tested the patch, but given that it falls back to the
original behavior (also tested) it's not a very dangerous patch to apply
even without extensive testing imho.

Thanks,
Zero

Re: [gentoo-catalyst] Re: [PATCH] Support snapshot.tar.xz format

[Richard Farina]

On 08/24/2012 02:24 PM, Sebastian Pipping wrote:
> On 08/24/2012 07:48 PM, Sebastian Pipping wrote:
>> On 08/22/2012 06:13 PM, Richard Farina wrote:
>>> This version of the patch addresses the concerns from Matt (of me
>>> failing to copy and paste everything) as well as updates the commit
>>> message to show that I have no verified this should work on all gentoo
>>> supported platforms.
>>>
>>> Please accept.
>>
>> Quoted patches are no fun.  Attach the patch or make the whole mail work
>> with git-am, please please please.
> 
> Ignore my conmplaint if the patch was identical to the previous version
> and you were just adding another note.
> 
yeah that's it, sorry for the confusion you had me really concerned for
a minute.

-Zero

> Best,
> 
> 
> 
> 
> Sebastian
> 
> 

Re: [gentoo-catalyst] Re: [PATCH] Support snapshot.tar.xz format

[Jorge Manuel B. S. Vicetto]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 24-08-2012 18:38, Richard Farina wrote:
> On 08/24/2012 01:48 PM, Sebastian Pipping wrote:
>> On 08/22/2012 06:13 PM, Richard Farina wrote:
>>> This version of the patch addresses the concerns from Matt (of
>>> me failing to copy and paste everything) as well as updates the
>>> commit message to show that I have no verified this should work
>>> on all gentoo supported platforms.
>>> 
>>> Please accept.
>> 
>> Quoted patches are no fun.  Attach the patch or make the whole
>> mail work with git-am, please please please.
> 
> I specifically sent the patch using git-email then replied to my
> post to add more detail than the commit message. This is the
> process I use on kernel mailing lists and never had a complaint.
> If this is not acceptable please let me know, but I have a feeling
> that you read my reply and not the first email that came
> through....
>> 
>> To be fair, I should mention that I hav no capacities for testing
>> this myself these days.  Hopefully someone else does.  Increased
>> chances for easy patch access ;-)

Richard (Zero) is testing catalyst / working on patches with me on
#gentoo-releng

> Obviously I have tested the patch, but given that it falls back to
> the original behavior (also tested) it's not a very dangerous patch
> to apply even without extensive testing imho.

I'll commit the patch and test it before doing a new release.

> Thanks, Zero
> 

- -- 
Regards,

Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
Gentoo- forums / Userrel / Devrel / KDE / Elections / RelEng
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJQOSLSAAoJEC8ZTXQF1qEPtlgP/2A/iCZNIv1Cbxndj6JBQxOh
U5sZUWb3VFukKoup8zwRVHns4crPjS1yKwQr1/8+FA6rCPQv+Rn3q16w/UvcF1Bx
tHQpRor8sSbK0A1TUfhR+qWZbRdNroq+wAXIaHNtVMkHwYJh6wnFEIB9dOM0tajW
GTkEz96aGoobd20IFZBtLtYO0uHODiYMkcgrlc9JzhSUw/zaJJ1XCCCvnpYMyW6M
m+M/b10YLgd3+fRSDLC10km9oIsP1Hq3JrXARvNfse7WfHWbW/gCC1q+dHrxq3GW
h3m+rDNHU3qki2Y+4SWoUn7ylwXo+CUXha33Dl3AqkruAjD6RPhOgYsO/b7lasCU
WgHwJMxKdrUrb3bIo100pcFoKvBG/YAk69HBKiqKw7K1TJuUrb6hFAT9h6FiHiJ2
3fl/w8dxNW+NJAGf4YKlHuq1WwltghlHQ9zp4DauYf1kK9fGBby8upbK7At9cBsx
gu892SAtfaUCs4qbPptb86METLHLfNyXZyMNxibQzgDxHNzlqi87uz5R9UaXsj7t
XiPDVqNPo4RA3/XmBq11o7T7V2jVYn4rRvL6h5Rl64Zmst70EaT0zAsUG2qrM22W
vCo3z5lcTk2Xj10TfQhy+HNJ/G+BErto0bZcxi0nRD6MGxvuZlZRFk17saj2K8Bl
Kbt0bK+PV9iSagOGJjf5
=2W/0
-----END PGP SIGNATURE-----