* [gentoo-dev] GLEP for Deltup package update system
@ 2003-07-28 22:09 John Whitney
2003-07-29 9:14 ` bdharring
2003-07-29 12:20 ` Kurt Lieber
0 siblings, 2 replies; 5+ messages in thread
From: John Whitney @ 2003-07-28 22:09 UTC (permalink / raw
To: gentoo-dev
Hi everyone,
I'd like to call your attention to GLEP 9 (http://glep.gentoo.org) which describes a non-intrusive system for patching old tarballs when updates are available. This would result in great bandwidth savings and requires no changes in the portage code. Users can simply emerge deltup and edit their FETCHCOMMAND in make.conf to enable patching. Package updates would greatly reduce mirror loads and are a must for anyone with a slow net connection. All that is required is to make patches available and make a file in the portage tree with the list of available patches. Deltup was written especially for Gentoo and is designed to patch tarballs while still retaining the correct md5sum.
You can try my demo (only a few patches available) by following Lisa's "Tiny-HOWTO: Deltup for Gentoo": http://www.thedoh.com/linux/HOWTO/deltup
All this project needs is the green light. Isn't it about time we make Gentoo more available to the bandwidth deprived?
---JJW
--
______________________________________________
http://www.linuxmail.org/
Now with e-mail forwarding for only US$5.95/yr
Powered by Outblaze
--
gentoo-dev@gentoo.org mailing list
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-dev] GLEP for Deltup package update system
2003-07-28 22:09 John Whitney
@ 2003-07-29 9:14 ` bdharring
2003-07-29 12:20 ` Kurt Lieber
1 sibling, 0 replies; 5+ messages in thread
From: bdharring @ 2003-07-29 9:14 UTC (permalink / raw
To: gentoo-dev; +Cc: John Whitney
On Monday, July 28, 2003, at 05:09 PM, John Whitney wrote:
> All this project needs is the green light. Isn't it about time we
> make Gentoo more available to the bandwidth deprived?
Taking a quick look at deltup-0.4 (most recent available via
sourceforge), it is still reliant on having an older version of bzip2
installed to handle cases where the target distfile was compressed w/
the older version of bzip2. That strikes me as side stepping the
problem- say down the line, either the bzip2 or gzip implementation was
tweaked again to produce a slightly better compression, you'd run into
the same problem again of maintaining md5 correctness.
As I had mentioned in the past on this ml, I'd think some solution
allowing for an md5 checksum of the data rather then the compressed
form of said data would be a better solution. On the plus side, it
could be extended to allow the user better control over what
compression is used on their distfiles, but that's ancillary to the
problem I mentioned above.
Other then that point of contention, kudos on bdelta's performance.
~bdh
> ---JJW
> --
> ______________________________________________
> http://www.linuxmail.org/
> Now with e-mail forwarding for only US$5.95/yr
>
> Powered by Outblaze
>
> --
> gentoo-dev@gentoo.org mailing list
>
--
gentoo-dev@gentoo.org mailing list
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-dev] GLEP for Deltup package update system
2003-07-28 22:09 John Whitney
2003-07-29 9:14 ` bdharring
@ 2003-07-29 12:20 ` Kurt Lieber
1 sibling, 0 replies; 5+ messages in thread
From: Kurt Lieber @ 2003-07-29 12:20 UTC (permalink / raw
To: John Whitney; +Cc: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 720 bytes --]
On Mon, Jul 28, 2003 at 04:09:24PM -0600 or thereabouts, John Whitney wrote:
>All that is required is to make patches available and make a file in the portage tree with the list of available patches.
Making those files available on our mirroring system is not as easy as you
might think. It is not a big-bang tranformation, meaning we will need to
continue to support people using the old way. Therefore, deltup will
require additional space on our mirrors. Before we can commit to
supporting this on our already-bulging mirror system, someone needs to
identify *exactly* what sort of space requirements this system will
require. (ok, not exactly, but at least within a 10% margin of error)
--kurt
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-dev] GLEP for Deltup package update system
@ 2003-07-29 16:45 John Whitney
2003-07-30 9:43 ` Paul de Vrieze
0 siblings, 1 reply; 5+ messages in thread
From: John Whitney @ 2003-07-29 16:45 UTC (permalink / raw
To: gentoo-dev
The space requirements are dynamic - they can be scaled to any level desired. We don't need patches for every package. This is accomplished by knowing which patches are available before trying to download them. You can provide a significant number of patches with just 100MB of space. Take a look at my demo repository: ftp://sunsite.dk/projects/deltup/patchfiles. All these patches take under 50MB and are still not fully optimized (newer patches will be about 33% more efficient). It'd be great to start out with patches for kernel, kde, gnome, gcc, mozilla, and some other big and necessary packages. Remember, there are a lot of dial-up users out there (some with only one phone line! :O :) )
A quota based system would be fine...
---JJW
> Making those files available on our mirroring system is not as easy as you
> might think. It is not a big-bang tranformation, meaning we will need to
> continue to support people using the old way. Therefore, deltup will
> require additional space on our mirrors. Before we can commit to
> supporting this on our already-bulging mirror system, someone needs to
> identify *exactly* what sort of space requirements this system will
> require. (ok, not exactly, but at least within a 10% margin of error)
> --kurt
--
______________________________________________
http://www.linuxmail.org/
Now with e-mail forwarding for only US$5.95/yr
Powered by Outblaze
--
gentoo-dev@gentoo.org mailing list
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-dev] GLEP for Deltup package update system
2003-07-29 16:45 [gentoo-dev] GLEP for Deltup package update system John Whitney
@ 2003-07-30 9:43 ` Paul de Vrieze
0 siblings, 0 replies; 5+ messages in thread
From: Paul de Vrieze @ 2003-07-30 9:43 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 1194 bytes --]
On Tuesday 29 July 2003 18:45, John Whitney wrote:
> The space requirements are dynamic - they can be scaled to any level
> desired. We don't need patches for every package. This is accomplished by
> knowing which patches are available before trying to download them. You
> can provide a significant number of patches with just 100MB of space. Take
> a look at my demo repository: ftp://sunsite.dk/projects/deltup/patchfiles.
> All these patches take under 50MB and are still not fully optimized (newer
> patches will be about 33% more efficient). It'd be great to start out with
> patches for kernel, kde, gnome, gcc, mozilla, and some other big and
> necessary packages. Remember, there are a lot of dial-up users out there
> (some with only one phone line! :O :) ) A quota based system would be
> fine...
You also might want to use download figures from a used mirror (not the
failsafe that is last in the list) to see which patches are actually wanted
by the userbase. If no-one downloads a certain patch that would signal the
time has come to remove it.
Paul
--
Paul de Vrieze
Researcher
Mail: pauldv@cs.kun.nl
Homepage: http://www.cs.kun.nl/~pauldv
[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2003-07-30 9:43 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2003-07-29 16:45 [gentoo-dev] GLEP for Deltup package update system John Whitney
2003-07-30 9:43 ` Paul de Vrieze
-- strict thread matches above, loose matches on Subject: below --
2003-07-28 22:09 John Whitney
2003-07-29 9:14 ` bdharring
2003-07-29 12:20 ` Kurt Lieber
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox