From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1REBO9-0005zQ-8b for garchives@archives.gentoo.org; Thu, 13 Oct 2011 02:51:17 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 7642AE0592; Thu, 13 Oct 2011 02:51:08 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by pigeon.gentoo.org (Postfix) with ESMTP id 12535E0592 for ; Thu, 13 Oct 2011 02:51:08 +0000 (UTC) Received: from pelican.gentoo.org (unknown [66.219.59.40]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 784A31B4009 for ; Thu, 13 Oct 2011 02:51:07 +0000 (UTC) Received: from localhost.localdomain (localhost [127.0.0.1]) by pelican.gentoo.org (Postfix) with ESMTP id B13FB80042 for ; Thu, 13 Oct 2011 02:51:06 +0000 (UTC) From: "Mike Frysinger" To: gentoo-commits@lists.gentoo.org Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Mike Frysinger" Message-ID: Subject: [gentoo-commits] proj/portage:master commit in: man/ X-VCS-Repository: proj/portage X-VCS-Files: man/xpak.5 X-VCS-Directories: man/ X-VCS-Committer: vapier X-VCS-Committer-Name: Mike Frysinger X-VCS-Revision: df6c9363bec1a7706b53ef1c6aa760bb45ccad4a Date: Thu, 13 Oct 2011 02:51:06 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: quoted-printable X-Archives-Salt: X-Archives-Hash: 80e63dab5d0379bcc8a45fdb6835262d commit: df6c9363bec1a7706b53ef1c6aa760bb45ccad4a Author: Mike Frysinger gentoo org> AuthorDate: Thu Oct 13 02:50:57 2011 +0000 Commit: Mike Frysinger gentoo org> CommitDate: Thu Oct 13 02:50:57 2011 +0000 URL: http://git.overlays.gentoo.org/gitweb/?p=3Dproj/portage.git;a= =3Dcommit;h=3Ddf6c9363 xpak(5): fix grammar, spelling, style, and add more content Signed-off-by: Mike Frysinger gentoo.org> --- man/xpak.5 | 253 +++++++++++++++++++++++++-----------------------------= ------ 1 files changed, 105 insertions(+), 148 deletions(-) diff --git a/man/xpak.5 b/man/xpak.5 index 49de449..0b5b874 100644 --- a/man/xpak.5 +++ b/man/xpak.5 @@ -1,193 +1,150 @@ -.TH XPAK 5 "Apr 2009" "Portage VERSION" "Portage" +.TH XPAK 5 "Oct 2011" "Portage VERSION" "Portage" .SH NAME -xpak \- The XPAK Data Format +xpak \- The XPAK Data Format used with Portage binary packages +.SH DESCRIPTION +Every Gentoo binary package has a xpak attached to it which contains bui= ld +time information like the USE flags it was built with, the ebuild it was +built from, the environmental variables, CFLAGS, CXXFLAGS, etc... .SH NOTES .SS Data Types +The following conventions cover all occurrences in this documentation .IP Integer -every offset or length(len) value in this documentation will be an unsig= ned -32bit integer in big endian byte order(32bit unsigned big int or uint32(= big) -). +All offsets/lengths are big endian unsigned 32bit integers .IP String -All strings, mentioned in this documentation are ASCII encoded, and not -nullterminated +All strings are ASCII encoded, and not NUL terminated (quotes are for il= lustration only) .IP Values -The actual values of the individual xpak entries are stored as Strings. +The actual values of the individual xpak entries are stored as Strings .P .SS Vertical Bars -The vertical bars '|' are not part of the file format, they are merely u= sed to +The vertical bars '|' are not part of the file format; they are merely u= sed to illustrate how the offset values apply to the data. - .SH SYNOPSIS - -.IP tarball - |<-xpak_offset->| -.br -|< xpak >|"STOP" - +.IP "binpkg (tbz2)" + |<-xpak_offset->| + |< xpak >|"STOP" .IP xpak -"XPAKPACK""XPAKSTOP" - + "XPAKPACK""XPAKSTOP" .IP index -|<-------------index_len------------->| -.br -|<...>| - + |<-------------index_len------------->| + |<...>| .IP indexN - |<-name_len->| -.br -|< name >| - + |<-name_len->| + |< name >| .IP data -|<--------------data_len------------->| -.br -|<-dataN_offset->|<-dataN_len->| -.br -|< data >|< data_N >|| + |<--------------data_len------------->| + |<-dataN_offset->|<-dataN_len->| + |< data >|< data_N >|| +.SH DETAILS +.SS xpak =20 -.SH DETAILED -Every gentoo binary package has a xpak attached to it which contains bui= ld -time information like the use flags it was built with, the ebuild it was -built from, the environmental variables, CFLAGs, CXXFLAGs, .... +If you look at a Gentoo binary package (binpkg) with a hex-editor you'll +notice that after the tarball of files, you find a binary blob - the +\fIxpak\fR, an offset which holds the bytes from the start of the +\fIxpak\fR to the end of the file - \fIxpak_offset\fR and finally the +String \fI"STOP"\fR. =20 -.SS xpak + || + |<---xpak---->|"STOP" + +Here you see the \fItar\fR archive, the attached \fIxpak\fR blob, the=20 +\fIxpak_offset\fR and the string \fI"STOP"\fR at the end. This metadata +is not considered "part" of the \fIxpak\fR, but rather part of the binpk= g. =20 -If you look at a gentoo binary package (binpkg) with a hex-editor you'll -notice the behinf the data, which belongs to the tarball you find a bina= ry -blob - the -.I xpak -, an offset which holds the bytes from the start of the -.I xpak -to the end of the file -=20 -.I xpak_offset -and finally the String -.I "STOP". - - || - |<---xpak---->|"STOP"| - -Here you see the=20 -.I tbz2 -archive, and the attached=20 -.I xpak -blob, the=20 -.I xpak-offset -and -the string -.I "STOP" -at the end. - -If we read the offset value and count -.I offset -bytes backwards from the start of -.I xpak_offset -, we have found the start of the -.I xpak -Block which starts with the String -.I "XPAKPACK". -This xpak block consists of the string -.I "XPAKPACK" -, the length of the=20 -.I index -block -=20 -.I index-len -, the length of the data block - -.I data-len -, an=20 -.I index-len -bytes long binary blob with the=20 -.I index -, a=20 -.I data-len -bytes long binary blob with the -.I data -and the string=20 -.I "XPAKSTOP" -at the end: +If we read the offset value and count \fIoffset\fR bytes backwards from +the start of \fIxpak_offset\fR, we have found the start of the \fIxpak\f= R +block which starts with the String \fI"XPAKPACK"\fR. + +This xpak block consists of the string \fI"XPAKPACK"\fR, the length of t= he +\fIindex\fR block (\fIindex_len\fR), the length of the \fIdata\fR block +(\fIdata_len\fR), an \fIindex_len\fR bytes long binary blob with the +\fIindex\fR, a \fIdata_len\fR bytes long binary blob with the \fIdata\fR= , +and the string \fI"XPAKSTOP"\fR at the end: =20 ||| "XPAKPACK"|<--index-->|<--data-->|"XPAKSTOP" =20 -To actually get the=20 -.I index -and the -.I data -, we cut out -.I index_len -bytes after the end of=20 -.I data_len - for the index block and then cut out the next=20 -.I data_len -bytes for the data block. If we have done everything right up to this po= int, -the following bytes would be the ASCII formatted string -.I "XPAKSTOP" -. - -The actual data is truncated into one big block - so if we want to read = it we -need the actual positions of each information in this big data block, th= is +To actually get the \fIindex\fR and the \fIdata\fR, we cut out \fIindex_= len\fR +bytes after the end of \fIdata_len\fR for the \fIindex\fR block, and the= n cut +out the next \fIdata_len\fR bytes for the \fIdata\fR block. If we have = done +everything right up to this point, the following bytes would be the ASCI= I +formatted string \fI"XPAKSTOP"\fR. + +The actual \fIdata\fR is merged into one big block; so if we want to rea= d it, +we need the actual positions of each information in this big data block.= This information can be obtained using the indices which are stored in the -.I index -block. +\fIindex\fR block. =20 .SS Index block -The index block consists of several truncated index blocks: +The \fIindex\fR block consists of several indices: =20 |<-----------------------index_len---------------------->| || =20 -The -.I index -block holds all information we need to find the data we want in the=20 -.I data -block. It consists of truncated index elements with a length -.I index_len. -Each of those index elements stands for one information in the data bloc= k and -consists of the length of its name ( -.I name_len> -), a=20 -.I name_len - bytes long string (the Name of the data block), this index belongs to, = the -offset of the=20 -.I data -block ( -.I data_offset -) and the length of that data block ( -.I data_len -): +The \fIindex\fR block holds all the information we need to find the data= we +want in the \fIdata\fR block. It consists of multiple index elements, a= ll of +which add up to the total length \fIindex_len\fR. It is not zero delimi= ted +or anything else. + +Each of those elements corresponds to one chunk of data in the \fIdata\f= R +block: the length of that block's name (\fIname_len\fR), a \fIname_len\f= R +bytes long string, the offset of that block (\fIdataN_offset\fR) and the +length of that block (\fIdataN_len\fR): =20 || |<--name-->| =20 .SS Data block -the data block contains truncated data blocks with a total length of -.I data_len -: +The \fIdata\fR block contains multiple chunks of data with a total lengt= h of +\fIdata_len\fR: =20 |<------------------------data_len------------------------>| || =20 -This binary block is -.I data_len -bytes long and consists of truncated data. - -To select one data element, we need the -.I data_offset -and the -.I data_len -from -the -.I index -, if we have those we can count -.I data_offset -bytes from the start of the -.I data -block, and then cut out the next -.I data_len -bytes. there we got our data block: +To select one data element, we need the \fIdata_offset\fR and the +\fIdata_len\fR from the \fIindex\fR. With those, we can count +\fIdata_offset\fR bytes from the start of the \fIdata\fR block, +and then cut out the next \fIdata_len\fR bytes. Then we got our +data block: =20 |<-----dataN_offset----->|<--dataN_len->| ||| +.SH EXAMPLES +Let's say we have an xpak with two chunks of data. The first has the na= me +"file1" with the contents "ddDddDdd" and the second has the name "file2"= with +the contents "jjJjjJjj". There is no \fI"STOP"\fR or \fIxpak_offset\fR = as +this xpak is not part of a binpkg. + +Here is the hexdump output (we will break it down line by line below): + 00 58 50 41 4b 50 41 43 4b 00 00 00 20 00 00 00 10 |XPAKPACK... ....= | + 10 00 00 00 04 66 69 6c 31 00 00 00 00 00 00 00 08 |....fil1........= | + 20 00 00 00 04 66 69 6c 32 00 00 00 08 00 00 00 08 |....fil2........= | + 30 64 64 44 64 64 44 64 64 6a 6a 4a 6a 6a 4a 6a 6a |ddDddDddjjJjjJjj= | + 40 58 50 41 4b 53 54 4f 50 |XPAKSTOP| + +The \fIindex_len\fR is 32 and the \fIdata_len\fR 16 (as there are 16 byt= es: +"ddDddDdd" and "jjJjjJjj"). + |<------"XPAKPACK"----->|| 32 | 16 | + 00 58 50 41 4b 50 41 43 4b 00 00 00 20 00 00 00 10 + +Now we have the first index element with a \fIname_len\fR of 4, followed +by the name string "fil1", followed by the data1 offset of 0 and a data1 +len of 8 (since data1 has 8 bytes: "ddDddDdd"). + | 4 |<--"fil1"->||data1_off:0|data1_len:8| + 10 00 00 00 04 66 69 6c 31 00 00 00 00 00 00 00 08 + +Now we have the second index element with a \fIname_len\fR of 4, followe= d +by the name string "fil2", followed by the data2 offset of 8 and a data2 +len of 8 (since data2 has 8 bytes: "jjJjjJjj"). + | 4 |<--"fil2"->||data2_off:8|data2_len:8| + 20 00 00 00 04 66 69 6c 32 00 00 00 08 00 00 00 08 + + |<------"XPAKSTOP"----->| + 40 58 50 41 4b 53 54 4f 50 .SH AUTHORS +.nf Lars Hartmann +Mike Frysinger +.fi .SH "SEE ALSO" .BR qtbz2 (1), .BR quickpkg (1),