[gentoo-user] GDBM incompatibility woes; any experts out there?

[gentoo-user] GDBM incompatibility woes; any experts out there?

[Kevin O'Gorman]

[-- Attachment #1.1: Type: text/plain, Size: 3946 bytes --]

Around 2002, I started working on a project that involved a few simple
database tables, and I wanted it
simple, so I used python and the gdbm module.  Since then, all has been
well.  Now I find that not only
do the gdbm modules of python and perl reject my files, but so does a C
program that uses the distributed
libgdbm.

Okay, so you think I broke my files somehow.  I was afraid that was true, so
I used the gdbm source
I had in distfiles, configured it with default setup but did not install
it.  Instead I compiled it with my
C test program, and set out to find the problem in the data, more or less
expecting
to spend a long time in the debugger.  But lo and behold, it worked just
fine.  Now I'm suspecting that
the ebuild does something (Large File support?) to the GDBM that it didn't
used to do.

I did not know when I started how much configuration information I was going
to need, and so there's
a configuation database.  As it happens, I never put more than one record in
it, so it's perfect for
simple testing.  The database is called dbhex.control, and the single record
key is "control".  I've
attached it.

I used this Makefile, with no targets or rules, just to get the flags I
want: I have my testfile 'testgdbm.c' in
the same directory with the makefile, and a gdbm-1.8.3 directory.

I make testgdbm, run "testgdbm dbhex.control" and get exactly what I
should.  If I link against the
Gentoo gdbm distribution, I get error 22 "invalid argument".  Not knowing
much about ebuilds, I'm not sure
how to tell what has changed.

Can anybody help me with:
  1) why it fails now with Gentoo tools.
  2) the best way to get it working again, preferably with both python and
C.  I expect there's either
      a compatibility flag, or I may need to do a file conversion.  Overall,
my databases run to about
      3 gigabytes, so it's doable either way.

Thanks in advance for any help.

# Makefile for tests
CC=gcc
CFLAGS=-Wall -g -m32 -ansi
LDFLAGS=-m32 gdbm-1.8.3/global.o gdbm-1.8.3/gdbmopen.o
gdbm-1.8.3/gdbmerrno.o gdbm-1.8.3/gdbmclose.o gdbm-1.8.3/update.o
gdbm-1.8.3/falloc.o gdbm-1.8.3/bucket.o gdbm-1.8.3/gdbmfetch.o
gdbm-1.8.3/findkey.o gdbm-1.8.3/version.o gdbm-1.8.3/gdbmseq.o
gdbm-1.8.3/hash.o

My test file:

/**
 * @file
 *
 * Program to test minimal functionality of the gdbm library on a known gdbm
file.
 *
 * Last Modified: Mon Aug  9 12:01:32 PDT 2010</pre>
 * @author Kevin O'Gorman
 */

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <gdbm.h>
#include <string.h>
#include <errno.h>

void
fatal(void) {
  fprintf(stderr, "Fatal function called\n");
  fprintf(stderr, "Errno is %d\n", errno);
  exit(EXIT_FAILURE);
}

/** The main thing.
 * @param argc the number of tokens on the input line.
 * @param argv an array of tokens.
 * @return 0 on success, 1-255 on failure
 */
int
main(int argc, char *argv[])
{
  datum key;
  datum value;
  datum nextkey;
  char longbucket[4096];

  printf("Running with GDBM: %s\n", gdbm_version);
  if (argc !=2) {
    fprintf(stderr, "Usage:\n   %s filename\n", argv[0]);
    exit(EXIT_FAILURE);
  }

  errno = 0;
  GDBM_FILE control = gdbm_open(argv[1], 1024, GDBM_READER, 0666, fatal);
  if (control == NULL) {
    perror("gdbm");
    fprintf(stderr, "Open returned NULL\n");
    fprintf(stderr, "Errno is %d\n", errno);
    exit(EXIT_FAILURE);
  }
  printf("is open\n");

  key = gdbm_firstkey(control);
  while (key.dptr) {
    memcpy(longbucket, key.dptr, key.dsize);
    longbucket[key.dsize] = '\0';
    printf("Key: %s", longbucket);
    value = gdbm_fetch(control, key);
    memcpy(longbucket, value.dptr, value.dsize);
    longbucket[value.dsize] = '\0';
    printf(", val: \"%s\"\n", longbucket);
    free(value.dptr);
    nextkey = gdbm_nextkey(control, key);
    free(key.dptr);
    key = nextkey;
  }

  gdbm_close(control);
  printf("That's all, folks...\n");

  return EXIT_SUCCESS;
}

/* vim: set et ai sts=2 sw=2: */


-- 
Kevin O'Gorman, PhD

[-- Attachment #1.2: Type: text/html, Size: 4765 bytes --]

[-- Attachment #2: dbhex.control --]
[-- Type: application/octet-stream, Size: 12428 bytes --]

[-- Attachment #3: Makefile --]
[-- Type: application/octet-stream, Size: 320 bytes --]

# Makefile for tests
CC=gcc
CFLAGS=-Wall -g -m32 -ansi
LDFLAGS=-m32 gdbm-1.8.3/global.o gdbm-1.8.3/gdbmopen.o gdbm-1.8.3/gdbmerrno.o gdbm-1.8.3/gdbmclose.o gdbm-1.8.3/update.o gdbm-1.8.3/falloc.o gdbm-1.8.3/bucket.o gdbm-1.8.3/gdbmfetch.o gdbm-1.8.3/findkey.o gdbm-1.8.3/version.o gdbm-1.8.3/gdbmseq.o gdbm-1.8.3/hash.o

[-- Attachment #4: testgdbm.c --]
[-- Type: text/x-c, Size: 1728 bytes --]

/**
 * @file
 *
 * Program to test minimal functionality of the gdbm library on a known gdbm file.
 *
 * Last Modified: Mon Aug  9 12:01:32 PDT 2010</pre>
 * @author Kevin O'Gorman
 */

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <gdbm.h>
#include <string.h>
#include <errno.h>

#define DBFILE "dbgames/ogdb-0000"

void
fatal(void) {
  fprintf(stderr, "Fatal function called\n");
  fprintf(stderr, "Errno is %d\n", errno);
  exit(EXIT_FAILURE);
}

/** The main thing.
 * @param argc the number of tokens on the input line.
 * @param argv an array of tokens.
 * @return 0 on success, 1-255 on failure
 */
int
main(int argc, char *argv[])
{
  datum key;
  datum value;
  datum nextkey;
  char longbucket[4096];

  printf("Running with GDBM: %s\n", gdbm_version);
  if (argc !=2) {
    fprintf(stderr, "Usage:\n   %s filename\n", argv[0]);
    exit(EXIT_FAILURE);
  }

  errno = 0;
  GDBM_FILE control = gdbm_open(argv[1], 1024, GDBM_READER, 0666, fatal);
  if (control == NULL) {
    perror("gdbm");
    fprintf(stderr, "Open returned NULL\n");
    fprintf(stderr, "Errno is %d\n", errno);
    exit(EXIT_FAILURE);
  }
  printf("is open\n");
  
  key = gdbm_firstkey(control);
  while (key.dptr) {
    memcpy(longbucket, key.dptr, key.dsize);
    longbucket[key.dsize] = '\0';
    printf("Key: %s", longbucket);
    value = gdbm_fetch(control, key);
    memcpy(longbucket, value.dptr, value.dsize);
    longbucket[value.dsize] = '\0';
    printf(", val: \"%s\"\n", longbucket);
    free(value.dptr);
    nextkey = gdbm_nextkey(control, key);
    free(key.dptr);
    key = nextkey;
  }

  gdbm_close(control);
  printf("That's all, folks...\n");
  
  return EXIT_SUCCESS;
}

/* vim: set et ai sts=2 sw=2: */

[gentoo-user] Re: GDBM incompatibility woes; any experts out there?

[walt]

On 08/09/2010 12:33 PM, Kevin O'Gorman wrote:

 > ...
> Now I find that not only
> do the gdbm modules of python and perl reject my files, but so does a C program that uses the distributed
> libgdbm.

You didn't say how long ago the problem started, but looking at the
files in sys-libs/gdbm I see nothing newer than March 20.  Is your
problem newer than March 20?

Have you tried running your test program with strace?

Re: [gentoo-user] Re: GDBM incompatibility woes; any experts out there?

[Kevin O'Gorman]

[-- Attachment #1: Type: text/plain, Size: 1003 bytes --]

On Mon, Aug 9, 2010 at 2:49 PM, walt <w41ter@gmail.com> wrote:

> On 08/09/2010 12:33 PM, Kevin O'Gorman wrote:
>
> > ...
>
>  Now I find that not only
>> do the gdbm modules of python and perl reject my files, but so does a C
>> program that uses the distributed
>> libgdbm.
>>
>
> You didn't say how long ago the problem started, but looking at the
> files in sys-libs/gdbm I see nothing newer than March 20.  Is your
> problem newer than March 20?
>
> Have you tried running your test program with strace?
>
>
I hadn't done anything with that application in over a year, so I did not
have any way to narrow it down.
As it happens, I had a sudden rush of brains to the head and read the ewarn
message that comes out
when you compile gdbm, to the effect that 32-bit systems may have to
rebuild, etc, etc.

As I suspected, it was LFS-related.

Write it off as a case of RTFLog.

Now all I have to do is discover why an ewarn wasn't emailed to me -- I
thought I had that set up.

-- 
Kevin O'Gorman, PhD

[-- Attachment #2: Type: text/html, Size: 1528 bytes --]