[gentoo-soc] Auto dependency builder progress report. Week 5.

["Александр Берсенев" <bay@hackerdom.ru>]

What has been done this week:
- rewritten stage-logging algorithm. New one is quicker and more reliable.
- changed type of internal communication sockets from SOCK_STREAM to
SOCK_SEQPACKET. This gives no performance improvement but code became
simpler.
- move from select to epoll for socket events notifications in python
server part.
- fully rewritten ld_preload hooking library, here is a commit:
http://git.overlays.gentoo.org/gitweb/?p=proj/autodep.git;a=commitdiff;h=d8d0c8f39f95af3647f73b5781199899e9b9529d
.
- test of both approaches has been unified.

---
I've spent a lot of time debugging very interesting issue and I want
to describe it here.

One of approaches I use for logging filesystem events is ld_preload
approach. I set LD_PRELOAD environment variable to tell OS to load my
library before starting an application. My library has own functions
like open, read and write and they are launched before glibc's. This
functions tells main python process about an event through unix domain
sockets. Name of socket file is stored in LOG_SOCKET environment
variable. When proccess trying to launch another process LD_PRELOAD
and LOG_SOCKET are inherited so my library is loaded before this
another process too.

But I've met a problem: in "prerm" stage of building connects to
socket always were refused!

First I think it is a bug in python server part. I moved it back from
epoll to select method for socket events notifications. Bug not
disappeared. Then I double-checked the code. All seems to be good, but
bug remains. Then I reverted few commits and this not helps.

Then I begin to examine the logging library. I think it is some
threading issue. I forced library code execute in one thread. This
gives no effect. Then I wrote small unix-socket client. Connections of
prerm stage was refused but in same time my client was connecting
successfully. In netstat socket was "listening".

Then I began to log all environment variables and find then someone
changes my LOG_SOCKET variable during "prerm" stage. I renamed it to
LOG_SOCKET_<LONG_STRING> in all sources hoping this is a
name-collision. But variable continues changing its value. The most
interesting fact: value was changing to another valid socket name, for
example from "/tmp/tmpFSDhf/socket_for_logging" to
"/tmp/tmpHDvJ5/socket_for_logging". But nobody listened
"/tmp/tmpHDvJ5/socket_for_logging". I started to believe in the higher
powers in this moment because last socket file was created five days
ago and notebook has been rebooted several times these days.

But the explanation was simple: portage system remembers all
environment variables while installing a package and restores them
while removing this package!!

Best,

Alexander Bersenev