[gentoo-devhelp] gdbm ebuild with 64bit off_t

[gentoo-devhelp] gdbm ebuild with 64bit off_t

[de Almeida, Valmor F.]

Hello,

Would it be possible for the gdbm ebuild allow for a 64bit size off_t
option?

The reason is that as it stands now, gdbm-1.8.3-r3 on 32bit machines can
only work with data base files of about 2 GB since off_t is 4 bytes. If
gdbm is built with off_t size equal to 8 bytes, it will handle much
larger database files.

It would be helpful if there were a USE 64bit option for emerging gdbm.

Thanks,

--
Valmor

PS: I am willing to help if needed. However I have not written an ebuild
file before.
--
gentoo-devhelp@lists.gentoo.org mailing list

Re: [gentoo-devhelp] gdbm ebuild with 64bit off_t

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 734 bytes --]

On Tuesday 22 January 2008, de Almeida, Valmor F. wrote:
> Would it be possible for the gdbm ebuild allow for a 64bit size off_t
> option?

this isnt really an ebuild issue.  the toolchain transparently changes off_t 
and friends to the 64bit versions when using LFS flags in CPPFLAGS.

> The reason is that as it stands now, gdbm-1.8.3-r3 on 32bit machines can
> only work with data base files of about 2 GB since off_t is 4 bytes. If
> gdbm is built with off_t size equal to 8 bytes, it will handle much
> larger database files.

can you describe the issue you're looking a bit more ?

> It would be helpful if there were a USE 64bit option for emerging gdbm.

mmm, that's not the way to approach this issue
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

RE: [gentoo-devhelp] gdbm ebuild with 64bit off_t

[de Almeida, Valmor F.]

> -----Original Message-----
> From: Mike Frysinger [mailto:vapier@gentoo.org]
> Sent: Tuesday, January 22, 2008 11:15 PM
> 
> > The reason is that as it stands now, gdbm-1.8.3-r3 on 32bit machines
can
> > only work with data base files of about 2 GB since off_t is 4 bytes.
If
> > gdbm is built with off_t size equal to 8 bytes, it will handle much
> > larger database files.
> 
> can you describe the issue you're looking a bit more ?

When emerging gdbm-1.8.3-r3 on my system, off_t is 4 bytes. Therefore
the gdbm files are of about 2 GB maximum size. Trying to create
databases with greater sizes results in

gdbm fatal: write error

If gdbm is built with off_t defined as 8 bytes, much larger gdbm files
can be created. However previous database files created with off_t of 4
bytes cannot be modified or read.

I am wondering whether gdbm could be built with this flexibility.


Thanks,

--
Valmor


->emerge --info
Portage 2.1.3.19 (default-linux/x86/2007.0, gcc-4.1.2, glibc-2.6.1-r0,
2.6.22.9 i686)
=================================================================
System uname: 2.6.22.9 i686 Intel(R) Xeon(TM) CPU 2.60GHz
Timestamp of tree: Sat, 17 Nov 2007 04:30:01 +0000
app-shells/bash:     3.2_p17
dev-java/java-config: 1.3.7, 2.0.33-r1
dev-lang/python:     2.4.4-r6
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 1.12.9-r2
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.6.3, 1.7.9-r1, 1.9.6-r2, 1.10
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.3.16
sys-devel/libtool:   1.5.24
virtual/os-headers:  2.6.22-r2
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O3 -march=pentium4 -pipe"
CHOST="i686-pc-linux-gnu"

> 
> > It would be helpful if there were a USE 64bit option for emerging
gdbm.
> 
> mmm, that's not the way to approach this issue
> -mike
--
gentoo-devhelp@lists.gentoo.org mailing list

Re: [gentoo-devhelp] gdbm ebuild with 64bit off_t

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

On Tuesday 22 January 2008, de Almeida, Valmor F. wrote:
> When emerging gdbm-1.8.3-r3 on my system, off_t is 4 bytes. Therefore
> the gdbm files are of about 2 GB maximum size. Trying to create
> databases with greater sizes results in
>
> gdbm fatal: write error
>
> If gdbm is built with off_t defined as 8 bytes, much larger gdbm files
> can be created. However previous database files created with off_t of 4
> bytes cannot be modified or read.

just modify the gdbm ebuild, add "inherit flag-o-matic", and then at the top 
of src_compile, add append-lfs-flags.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

RE: [gentoo-devhelp] gdbm ebuild with 64bit off_t

[de Almeida, Valmor F.]

> -----Original Message-----
> From: Mike Frysinger [mailto:vapier@gentoo.org]
> Sent: Wednesday, January 23, 2008 12:23 AM
> 
> just modify the gdbm ebuild, add "inherit flag-o-matic", and then at
the
> top
> of src_compile, add append-lfs-flags.
> -mike

I did and I get

         ->emerge --pretend --verbose gdbm

           These are the packages that would be merged, in order:

           Calculating dependencies \!!! Digest verification failed:
           !!! /usr/portage/sys-libs/gdbm/gdbm-1.8.3-r3.ebuild
           !!! Reason: Filesize does not match recorded size
           !!! Got: 1216
           !!! Expected: 1173
           /
           !!! All ebuilds that could satisfy "gdbm" have been masked.
           !!! One of the following masked packages is required to
complete                your request:
           - sys-libs/gdbm-1.8.3-r3 (masked by: corruption)


Do I need to include gdbm in /etc/portage/package.unmask ?

Also, is the change you suggested going to be part of the next versions
of the gdbm ebuild?

Thanks,

--
Valmor
--
gentoo-devhelp@lists.gentoo.org mailing list

Re: [gentoo-devhelp] gdbm ebuild with 64bit off_t

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 399 bytes --]

On Wednesday 23 January 2008, de Almeida, Valmor F. wrote:
>            !!! Digest verification failed:
>            !!! /usr/portage/sys-libs/gdbm/gdbm-1.8.3-r3.ebuild
>            !!! Reason: Filesize does not match recorded size
>            !!! Got: 1216
>            !!! Expected: 1173

well, you obviously need to rebuild the digest or temporarily remove strict 
from FEATURES
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

RE: [gentoo-devhelp] gdbm ebuild with 64bit off_t

[de Almeida, Valmor F.]

> -----Original Message-----
> From: Mike Frysinger [mailto:vapier@gentoo.org]
> Sent: Wednesday, January 23, 2008 10:02 AM
> 
> well, you obviously need to rebuild the digest or temporarily remove
> strict
> from FEATURES
> -mike

Never dealt with ebuild files before; did some reading and then

cd /usr/portage/sys-libs/gdbm/
ebuild gdbm-1.8.3-r3.ebuild digest
emerge gdbm

Noticed that gcc now uses

-D_FILE_OFFSET_BITS=64

Thanks for your help. I am wondering whether the mods you suggested will
be part of gdbm ebuild files in the future.

--
Valmor

--
gentoo-devhelp@lists.gentoo.org mailing list

Re: [gentoo-devhelp] gdbm ebuild with 64bit off_t

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 551 bytes --]

On Wednesday 23 January 2008, de Almeida, Valmor F. wrote:
> > -----Original Message-----
> > From: Mike Frysinger [mailto:vapier@gentoo.org]
> > Sent: Wednesday, January 23, 2008 10:02 AM
> >
> > well, you obviously need to rebuild the digest or temporarily remove
> > strict from FEATURES
>
> Never dealt with ebuild files before; did some reading and then
>
> cd /usr/portage/sys-libs/gdbm/
> ebuild gdbm-1.8.3-r3.ebuild digest
> emerge gdbm
>
> Noticed that gcc now uses
>
> -D_FILE_OFFSET_BITS=64

ok ... but does that solve your problem ?
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

RE: [gentoo-devhelp] gdbm ebuild with 64bit off_t

[de Almeida, Valmor F.]

> -----Original Message-----
> From: Mike Frysinger [mailto:vapier@gentoo.org]
> Sent: Wednesday, January 23, 2008 5:33 PM
> 
> > Never dealt with ebuild files before; did some reading and then
> >
> > cd /usr/portage/sys-libs/gdbm/
> > ebuild gdbm-1.8.3-r3.ebuild digest
> > emerge gdbm
> >
> > Noticed that gcc now uses
> >
> > -D_FILE_OFFSET_BITS=64
> 
> ok ... but does that solve your problem ?
> -mike

Yes, this does solve gdbm's problem. I tried the newly emerged gdbm and
was able to create large data bases beyond the ~2GB file size limit. I
generated 3.1GB databases and twice as large.

Will the ebuild change be a part of future ebuild versions?

I have exchanged e-mails with the gdbm maintainer and he said databases
written with 32bit off_t will not be read/modified by gdbm when compiled
with 64bit off_t.

Thanks,

--
Valmor
--
gentoo-devhelp@lists.gentoo.org mailing list

Re: [gentoo-devhelp] gdbm ebuild with 64bit off_t

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1068 bytes --]

On Friday 25 January 2008, de Almeida, Valmor F. wrote:
> > -----Original Message-----
> > From: Mike Frysinger [mailto:vapier@gentoo.org]
> > Sent: Wednesday, January 23, 2008 5:33 PM
> >
> > > Never dealt with ebuild files before; did some reading and then
> > >
> > > cd /usr/portage/sys-libs/gdbm/
> > > ebuild gdbm-1.8.3-r3.ebuild digest
> > > emerge gdbm
> > >
> > > Noticed that gcc now uses
> > >
> > > -D_FILE_OFFSET_BITS=64
> >
> > ok ... but does that solve your problem ?
>
> Yes, this does solve gdbm's problem. I tried the newly emerged gdbm and
> was able to create large data bases beyond the ~2GB file size limit. I
> generated 3.1GB databases and twice as large.
>
> Will the ebuild change be a part of future ebuild versions?
>
> I have exchanged e-mails with the gdbm maintainer and he said databases
> written with 32bit off_t will not be read/modified by gdbm when compiled
> with 64bit off_t.

that's because you're only changing how gdbm interfaces with the C library.  
you arent changing the gdbm API/ABI.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

RE: [gentoo-devhelp] gdbm ebuild with 64bit off_t

[de Almeida, Valmor F.]

> -----Original Message-----
> From: Mike Frysinger [mailto:vapier@gentoo.org]
> Sent: Saturday, January 26, 2008 5:17 AM
> 
> > >
> > > ok ... but does that solve your problem ?
> >
> > Yes, this does solve gdbm's problem. I tried the newly emerged gdbm
and
> > was able to create large data bases beyond the ~2GB file size limit.
I
> > generated 3.1GB databases and twice as large.
> >
> > Will the ebuild change be a part of future ebuild versions?

Will I have to modify the gdbm ebuild in the future as I update my
gentoo systems? Or will the ebuild be modified in future versions?

Thanks,

--
Valmor

> >
> > I have exchanged e-mails with the gdbm maintainer and he said
databases
> > written with 32bit off_t will not be read/modified by gdbm when
compiled
> > with 64bit off_t.
> 
> that's because you're only changing how gdbm interfaces with the C
> library.
> you arent changing the gdbm API/ABI.
> -mike
--
gentoo-devhelp@lists.gentoo.org mailing list

Re: [gentoo-devhelp] gdbm ebuild with 64bit off_t

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 226 bytes --]

On Sunday 27 January 2008, de Almeida, Valmor F. wrote:
> Will I have to modify the gdbm ebuild in the future as I update my
> gentoo systems? Or will the ebuild be modified in future versions?

it's already in the tree
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]