[gentoo-user] Downloading packages from a txt file

[gentoo-user] Downloading packages from a txt file

[Grant]

Hello, a little while ago I asked this list for help with updating a
machine with no network connection.  The solution was to get a txt
list of the package paths and then download them on a networked
machine with this command:

sort -u links.txt | xargs -n 1 -i{} sh -c 'for i in {}; do wget -c $i
&& break; done'

This worked great then, but something must have changed because that
now only downloads about 12 or so files and then quits without an
error.  There are a lot more packages in links.txt to be downloaded. 
Does anyone know how to fix this?

- Grant

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Downloading packages from a txt file

[Alex]

On Friday 19 August 2005 15:18, Grant wrote:
> sort -u links.txt | xargs -n 1 -i{} sh -c 'for i in {}; do wget -c $i
> && break; done'

How about this instead,

$ wget -ci links.txt

It always did the job for me :)

-- 
Cheers, Alex.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Downloading packages from a txt file

[Grant]

> > sort -u links.txt | xargs -n 1 -i{} sh -c 'for i in {}; do wget -c $i
> > && break; done'
> 
> How about this instead,
> 
> $ wget -ci links.txt
> 
> It always did the job for me :)
> 
> --
> Cheers, Alex.

Hi Alex,

The problem is that the links.txt file was generated by 'emerge -fp
world' and has a bunch of alternate paths for each file separated by
'%20'.  Using 'wget -ci links.txt' turns out like this:

http://gentoo.ccccom.com/distfiles/man-pages-2.07.tar.bz2%20http:/www.fr.kernel.org/pub/linux/docs/manpages/man-pages-2.07.tar.bz2%20http:/www.at.kernel.org/pub/linux/docs/manpages/man-pages-2.07.tar.bz2%20http:/www.uk.kernel.org/pub/linux/docs/manpages/man-pages-2.07.tar.bz2%20http:/www.de.kernel.org/pub/linux/docs/manpages/man-pages-2.07.tar.bz2%20http:/www.us.kernel.org/pub/linux/docs/manpages/man-pages-2.07.tar.bz2%20http:/www.kernel.org/pub/linux/docs/manpages/man-pages-2.07.tar.bz2
           => `man-pages-2.07.tar.bz2'
Resolving gentoo.ccccom.com... 206.251.252.55
Connecting to gentoo.ccccom.com[206.251.252.55]:80... connected.
HTTP request sent, awaiting response... 404 Not Found
08:26:12 ERROR 404: Not Found.

Do you know how to get 'emerge -fp world' to return only one path per
file or get wget to use the %20-separated alternate paths properly?

- Grant

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Downloading packages from a txt file

[Alex]

On Saturday 20 August 2005 15:32, Grant wrote:
> Do you know how to get 'emerge -fp world' to return only one path per
> file or get wget to use the %20-separated alternate paths properly?

hmm... if you try:

$ cat links.txt | xargs wget -c 

I know it's not delicate but it might work :)

sorry but i can't be much of a help, my bash scripting knowledge is limited...
-- 
Cheers, Alex.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Downloading packages from a txt file

[Grant]

> > Do you know how to get 'emerge -fp world' to return only one path per
> > file or get wget to use the %20-separated alternate paths properly?
> 
> hmm... if you try:
> 
> $ cat links.txt | xargs wget -c
> 
> I know it's not delicate but it might work :)
> 
> sorry but i can't be much of a help, my bash scripting knowledge is limited...
> --
> Cheers, Alex.

Hi Alex,

That works for some of them but I get a lot of:

416 Requested Range Not Satisfiable

- Grant

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Downloading packages from a txt file

[Alex]

On Monday 22 August 2005 14:52, Grant wrote:
> That works for some of them but I get a lot of:
>
> 416 Requested Range Not Satisfiable

Are the packages downloaded? Doing "cat links.txt | xargs wget -c" will try to 
download from *all* urls listed in links.txt and there are a few of them for 
each package, so it might  be downloaded after-all :)
Those error could be some bad urls in the ebuilds. Just a guess, I don't 
know...
-- 
Cheers, Alex.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Downloading packages from a txt file

[Ow Mun Heng]

On Sat, 2005-08-20 at 08:32 -0700, Grant wrote:

> Do you know how to get 'emerge -fp world' to return only one path per
> file or get wget to use the %20-separated alternate paths properly?
> 

Try this.. it's a 2 step process, I believe it can be done in 1 step
but, I'll let you figure it out.

emerge -fp mono 2> /tmp/link.txt >&1
cat /tmp/link.txt | cut -d" " -f1 | xargs wget -c

Note : If the 1st link doesn't work, then it won't try another mirror.

Be aware

-- 
Ow Mun Heng
Gentoo/Linux on DELL D600 1.4Ghz 1.5GB RAM
98% Microsoft(tm) Free!! 
Neuromancer 10:50:09 up 3 days, 15:54, 5 users, load average: 2.45,
2.31, 2.43 


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Downloading packages from a txt file

[Nick Rout]

i say plug the bloody computer into a network and be done with it. 

the darn things aren't much fun without connectivity anyway IMHO 

:-)

On Wed, 24 Aug 2005 10:50:45 +0800
Ow Mun Heng wrote:

> On Sat, 2005-08-20 at 08:32 -0700, Grant wrote:
-- 
Nick Rout <nick@rout.co.nz>

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Downloading packages from a txt file

[Nick Rout]

Seriously this time:

How about changing your strategy to this:

Get a list of the packages you want to update from the target machine.
something like:

emerge -uDp world|grep ebuild|awk '{print($4)}'>packlist

take packlist to the connected machine and type:

for package in `cat packlist` ; do DISTDIR=/where/ever/i/want emerge
--nodeps -f =$package; done

The files will then be in /where/ever/i/want and you can put them on a
cd or whatever method you are using and take them away.

--nodeps will make sure that your connected host doesn't substitute its
own idea of what the deps are (perhaps based on different USE flags)


You could also probably do something like:

for package in `cat packlist` ; do DISTDIR="/where/ever/i/want" GENTOO_MIRRORS="local
/usr/portage/distfiles" emerge --nodeps -f =$package; done

The GENTOO_MIRRORS="local /usr/portage/distfiles" should take files from
the local system in preference to downloading them, which will save your
bandwidth, although i am not 100% sure of the syntax. 

In other words let your network connected host choose where to download
from.

On Fri, 19 Aug 2005 08:18:08 -0700
Grant wrote:

> Hello, a little while ago I asked this list for help with updating a
> machine with no network connection.  The solution was to get a txt
> list of the package paths and then download them on a networked
> machine with this command:
> 
> sort -u links.txt | xargs -n 1 -i{} sh -c 'for i in {}; do wget -c $i
> && break; done'
> 
> This worked great then, but something must have changed because that
> now only downloads about 12 or so files and then quits without an
> error.  There are a lot more packages in links.txt to be downloaded. 
> Does anyone know how to fix this?
> 
> - Grant
> 
> -- 
> gentoo-user@gentoo.org mailing list

-- 
Nick Rout <nick@rout.co.nz>

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Downloading packages from a txt file

[Fernando Canizo]

El 24/ago/2005 a las 02:36 -0300, Nick me decía:
> Seriously this time:
> 
> How about changing your strategy to this:
> 
> Get a list of the packages you want to update from the target machine.
> something like:
> 
> emerge -uDp world|grep ebuild|awk '{print($4)}'>packlist
> 
> take packlist to the connected machine and type:
> 
> for package in `cat packlist` ; do DISTDIR=/where/ever/i/want emerge
> --nodeps -f =$package; done
> 
> The files will then be in /where/ever/i/want and you can put them on a
> cd or whatever method you are using and take them away.
> 
> --nodeps will make sure that your connected host doesn't substitute its
> own idea of what the deps are (perhaps based on different USE flags)

Could work, but i think he says he has debian in connected machine.

(hasn't?)

Anyway i would be great if 'emerge' has clean option like
'--print-uris' from apt-get so one could just use 'wget' or whatever
they find on connected machine. Other idea could be to have a script
that downloads the gentoo-way using what emerge print now, checking
mirrors, etc. Like a tiny version of emerge.

-- 
Fernando Canizo - http://www.lugmen.org.ar/~conan/
<ciaranm> antialiasing?
<ciaranm> the alcohol does that for you
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Downloading packages from a txt file

[Nick Rout]

On Wed, August 24, 2005 11:14 pm, Fernando Canizo said:
> El 24/ago/2005 a las 02:36 -0300, Nick me decía:
>> Seriously this time:
>>
>> How about changing your strategy to this:
>>
>> Get a list of the packages you want to update from the target machine.
>> something like:
>>
>> emerge -uDp world|grep ebuild|awk '{print($4)}'>packlist
>>
>> take packlist to the connected machine and type:
>>
>> for package in `cat packlist` ; do DISTDIR=/where/ever/i/want emerge
>> --nodeps -f =$package; done
>>
>> The files will then be in /where/ever/i/want and you can put them on a
>> cd or whatever method you are using and take them away.
>>
>> --nodeps will make sure that your connected host doesn't substitute its
>> own idea of what the deps are (perhaps based on different USE flags)
>
> Could work, but i think he says he has debian in connected machine.

yes thats a bit of a barrier, although having debian doesn't stop him
installing portage AFAIK. A long time ago in a galaxy far away there was a
howto on installing portage on other distros.

>
> (hasn't?)
>
> Anyway i would be great if 'emerge' has clean option like
> '--print-uris' from apt-get so one could just use 'wget' or whatever
> they find on connected machine.

you mean like:

emerge -fp target 2>&1|grep -v Calculating|grep -v '...done!'|sed -e 's/\
/\n/g'

The sed part transforms from a list all on one line with spaces separating
to a list one per line, ie:

url1 url2 url3

to

url1
url2
url3

Trouble is this gives a list with many alternatives for each url
(depending on the number of mirrors shown in the ebuild.

You then need a script to download each url ONLY if one of the earlier
url's for the same file hasn't worked.

>Other idea could be to have a script
> that downloads the gentoo-way using what emerge print now, checking
> mirrors, etc. Like a tiny version of emerge.
>
> --
> Fernando Canizo - http://www.lugmen.org.ar/~conan/
> <ciaranm> antialiasing?
> <ciaranm> the alcohol does that for you
> --
> gentoo-user@gentoo.org mailing list
>
>


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Downloading packages from a txt file

[Willie Wong]

On Thu, Aug 25, 2005 at 09:55:32AM +1200, Nick Rout wrote:
> On Wed, August 24, 2005 11:14 pm, Fernando Canizo said:
> The sed part transforms from a list all on one line with spaces separating
> to a list one per line, ie:
> 
> url1 url2 url3
> 
> to
> 
> url1
> url2
> url3
> 
> Trouble is this gives a list with many alternatives for each url
> (depending on the number of mirrors shown in the ebuild.
> 
> You then need a script to download each url ONLY if one of the earlier
> url's for the same file hasn't worked.
> 

I don't think you do... I think "wget -c" will only download if the
size doesn't match. And if you trust your internet connection enough,
you can use "wget -nc", which only downloads if a file of the same
name does not exist (but doesn't check for length/size). 

Though, seeing that the internet connection is on a separate machine,
maybe the OP would like to have integrity checking while downloading
(so he won't run into "D'oh, the MD5 sum doesn't match" problem). And
with my poor "skillz" in programming, I'd rather install portage on
Debian then to hack a replacement to download and check the md5 for
the packages. 

W
-- 
"Now it is such a bizarrely improbable coincidence that 
anything so mindboggingly useful could have evolved purely 
by chance that some thinkers have chosen to see it as the 
final and clinching proof of the non-existence of God.
"The argument goes something like this: `I refuse to prove 
that I exist,' says God, `for proof denies faith, and 
without faith I am nothing.'
"`But,' says Man, `The Babel fish is a dead giveaway, isn't 
it? It could not have evolved by chance. It proves you 
exist, and so therefore, by your own arguments, you don't. 
QED.'
Sortir en Pantoufles: up 13 days,  2:42
-- 
gentoo-user@gentoo.org mailing list