[gentoo-user] df and du difference

[gentoo-user] df and du difference

[platoali]

Hi, 

I've a strange problem with  my root partion:

 # du -hxs /
188M    /


and 

# du -hx --max-dep=1 /
24M     /root
4.0K    /cdrom
19M     /etc
76K     /.nvclock
12K     /media
100K    /chroot
4.0K    /home
4.0K    /usr
1.9M    /package
5.5M    /bin
4.0K    /windows2
125M    /lib
4.0K    /service
4.0K    /opt
4.0K    /var
4.0K    /command
12M     /sbin
4.0K    /tmp
0       /dev
1.3M    /lost+found
0       /proc
4.0K    /boot
4.0K    /mnt
4.0K    /windows
40K     /.subversion
0       /sys
4.0K    /boot2
188M    /


but when I run df:
df -h               
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3             2.0G  640M  1.3G  35% /
.....



the difference between du and df is about 640 - 188 = 452 MB. and "df" is 
showing  that my root is full  2.4 times more than "du".

which one is the correct one? I've another server that this difference is 
about 7 GiG and on that server root is 80% full.  The type of partition is 
ext3.

Do others have this kind of inconsistancy on their systems?

Best regards
Platoali

Re: [gentoo-user] df and du difference

[Volker Armin Hemmann]

On Samstag, 16. August 2008, platoali@gmail.com wrote:
> Hi,
>
> I've a strange problem with  my root partion:
>
>  # du -hxs /
> 188M    /
>
>
> and
>
> # du -hx --max-dep=1 /
> 24M     /root
> 4.0K    /cdrom
> 19M     /etc
> 76K     /.nvclock
> 12K     /media
> 100K    /chroot
> 4.0K    /home
> 4.0K    /usr
> 1.9M    /package
> 5.5M    /bin
> 4.0K    /windows2
> 125M    /lib
> 4.0K    /service
> 4.0K    /opt
> 4.0K    /var
> 4.0K    /command
> 12M     /sbin
> 4.0K    /tmp
> 0       /dev
> 1.3M    /lost+found
> 0       /proc
> 4.0K    /boot
> 4.0K    /mnt
> 4.0K    /windows
> 40K     /.subversion
> 0       /sys
> 4.0K    /boot2
> 188M    /
>
>
> but when I run df:
> df -h
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/sda3             2.0G  640M  1.3G  35% /
> .....
>
>
>
> the difference between du and df is about 640 - 188 = 452 MB. and "df" is
> showing  that my root is full  2.4 times more than "du".
>
> which one is the correct one? I've another server that this difference is
> about 7 GiG and on that server root is 80% full.  The type of partition is
> ext3.

df is 'more correct'.

>
> Do others have this kind of inconsistancy on their systems?

yes

Re: [gentoo-user] df and du difference

[Sebastian Günther]

[-- Attachment #1: Type: text/plain, Size: 982 bytes --]

* platoali@gmail.com (platoali@gmail.com) [16.08.08 07:51]:
> Hi, 
> 
> I've a strange problem with  my root partion:
> 
> the difference between du and df is about 640 - 188 = 452 MB. and "df" is 
> showing  that my root is full  2.4 times more than "du".
> 
> which one is the correct one? I've another server that this difference is 
> about 7 GiG and on that server root is 80% full.  The type of partition is 
> ext3.
> 

df shows you the available space on the fs and du the size of the files 
inside it.

The difference is caused by the journal and the 5% reserved for the 
superuser, which du does not take in account

> Do others have this kind of inconsistancy on their systems?
> 
I would think everyone, who does not have changed the default settings 
from mkfs.ext3.

> Best regards
> Platoali
> 

HTH
Sebastian

-- 
 " Religion ist das Opium des Volkes. "      Karl Marx

 SEB@STI@N GÜNTHER         mailto:samson@guenther-roetgen.de

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

[gentoo-user] Re: df and du difference

[platoali]

 Sebastian Günther wrote:
> df shows you the available space on the fs and du the size of the files
> inside it.
>
> The difference is caused by the journal and the 5% reserved for the
> superuser, which du does not take in account
>
> > Do others have this kind of inconsistancy on their systems?
>
> I would think everyone, who does not have changed the default settings
> from mkfs.ext3.

I've another question. On my server root is 80% full and last weed it was 98% 
full. if it get to 100% , How can I delete or flush Journals to free some 
space?

best wishes
Platoali 

Re: [gentoo-user] Re: df and du difference

[Sebastian Günther]

[-- Attachment #1: Type: text/plain, Size: 615 bytes --]

* platoali@gmail.com (platoali@gmail.com) [16.08.08 10:08]:
> 
> I've another question. On my server root is 80% full and last weed it was 98% 
> full. if it get to 100% , How can I delete or flush Journals to free some 
> space?
> 
That is what the 5% are for, as you saw there where stated as not 
available but they are for the superuser for such things.

BTW: Why is your root so full, or didn't you partionate your disk?

> best wishes
> Platoali 
> 

HTH
Sebastian

-- 
 " Religion ist das Opium des Volkes. "      Karl Marx

 SEB@STI@N GÜNTHER         mailto:samson@guenther-roetgen.de

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-user] Re: df and du difference

[Dale]

platoali@gmail.com wrote:
>  Sebastian Günther wrote:
>   
>> df shows you the available space on the fs and du the size of the files
>> inside it.
>>
>> The difference is caused by the journal and the 5% reserved for the
>> superuser, which du does not take in account
>>
>>     
>>> Do others have this kind of inconsistancy on their systems?
>>>       
>> I would think everyone, who does not have changed the default settings
>> from mkfs.ext3.
>>     
>
> I've another question. On my server root is 80% full and last weed it was 98% 
> full. if it get to 100% , How can I delete or flush Journals to free some 
> space?
>
> best wishes
> Platoali 
>
>
>   

I think this may help you get more information.  What exactly does your 
"server" have installed?  What is it used for?  Web server?  File 
server?  DVR?  Could it be that some log file is growing and taking up 
that space?  How is your system partitioned?

I'm not guru but some more info may help.

Dale

:-)  :-) 

[gentoo-user] Re: df and du difference

[Platoali]

Sebastian Günther wrote:
> That is what the 5% are for, as you saw there where stated as not
> available but they are for the superuser for such things.

So there is no way to free some space from journals.

>
> BTW: Why is your root so full, or didn't you partionate your disk?

I did not partitioned it myself. This server is inherited to me from last 
admin.

~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              19G   14G  3.5G  81% /
varrun                2.0G   76K  2.0G   1% /var/run
varlock               2.0G     0  2.0G   0% /var/lock
udev                  2.0G   88K  2.0G   1% /dev
devshm                2.0G     0  2.0G   0% /dev/shm
/dev/sdb5              93G   59G   27G  69% /mnt/backup
/dev/mapper/main-usr   15G  601M   14G   5% /usr
/dev/mapper/main-var   30G  1.7G   27G   6% /var
/dev/mapper/main-db    69G  9.5G   56G  15% /var/lib/postgresql
/dev/sdc1              68G   35G   30G  55% /home/archive

~#  du --max-dep 1  -c  -hx  /                        
4.2M    /etc                                                           
36M     /tftpboot                                                      
16K     /lost+found                                                    
3.8G    /tmp                                                           
18M     /boot                                                          
1.4G    /home                                                          
8.0K    /mnt                                                           
12K     /media                                                         
254M    /root                                                          
4.0K    /var                                                           
4.0K    /srv                                                           
0       /sys                                                           
4.0K    /initrd
77M     /lib
0       /proc
4.0K    /opt
4.0K    /usr
6.4M    /sbin
3.5M    /bin
0       /dev
5.5G    /
5.5G    total

Last week, I was alarmed  that / root is 98 percent full. but I could not find 
any reason why server is full. and a restart freed  8 gig of space. but now it 
is again getting full slowly. 

Any comment?

best wishes
Platoali

Re: [gentoo-user] Re: df and du difference

[Dale]

Platoali wrote:
> Sebastian Günther wrote:
>   
>> That is what the 5% are for, as you saw there where stated as not
>> available but they are for the superuser for such things.
>>     
>
> So there is no way to free some space from journals.
>
>   
>> BTW: Why is your root so full, or didn't you partionate your disk?
>>     
>
> I did not partitioned it myself. This server is inherited to me from last 
> admin.
>
> ~# df -h
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/sda1              19G   14G  3.5G  81% /
> varrun                2.0G   76K  2.0G   1% /var/run
> varlock               2.0G     0  2.0G   0% /var/lock
> udev                  2.0G   88K  2.0G   1% /dev
> devshm                2.0G     0  2.0G   0% /dev/shm
> /dev/sdb5              93G   59G   27G  69% /mnt/backup
> /dev/mapper/main-usr   15G  601M   14G   5% /usr
> /dev/mapper/main-var   30G  1.7G   27G   6% /var
> /dev/mapper/main-db    69G  9.5G   56G  15% /var/lib/postgresql
> /dev/sdc1              68G   35G   30G  55% /home/archive
>
> ~#  du --max-dep 1  -c  -hx  /                        
> 4.2M    /etc                                                           
> 36M     /tftpboot                                                      
> 16K     /lost+found                                                    
> 3.8G    /tmp                                                           
> 18M     /boot                                                          
> 1.4G    /home                                                          
> 8.0K    /mnt                                                           
> 12K     /media                                                         
> 254M    /root                                                          
> 4.0K    /var                                                           
> 4.0K    /srv                                                           
> 0       /sys                                                           
> 4.0K    /initrd
> 77M     /lib
> 0       /proc
> 4.0K    /opt
> 4.0K    /usr
> 6.4M    /sbin
> 3.5M    /bin
> 0       /dev
> 5.5G    /
> 5.5G    total
>
> Last week, I was alarmed  that / root is 98 percent full. but I could not find 
> any reason why server is full. and a restart freed  8 gig of space. but now it 
> is again getting full slowly. 
>
> Any comment?
>
> best wishes
> Platoali
>
>   

Sebastian may have more and better ideas but if a reboot gave you some space back, then you should check the tmp directories that are usually cleared when rebooting.  I notice that in your list /tmp takes up 3.8Gb which is a good bit. May want to see what is in there.

Just my thoughts.

Dale

:-)  :-) 

[gentoo-user] Re: df and du difference

[Francesco Talamona]

On Saturday 16 August 2008, Dale wrote:
> Sebastian may have more and better ideas but if a reboot gave you
> some space back, then you should check the tmp directories that are
> usually cleared when rebooting.  I notice that in your list /tmp
> takes up 3.8Gb which is a good bit. May want to see what is in there.
>
> Just my thoughts.

Absolutely right! Double check what's stuffing /tmp. You also could try 
to mount /tmp on a larger partition (like /usr and /var).

Ciao
	Francesco

-- 
Linux Version 2.6.26-gentoo, Compiled #2 PREEMPT Sat Aug 9 20:21:11 CEST 
2008
One 1GHz AMD Athlon 64 Processor, 2GB RAM, 2004.04 Bogomips Total
aemaeth

Re: [gentoo-user] df and du difference

[Ward Poelmans]

On Sat, Aug 16, 2008 at 07:50,  <platoali@gmail.com> wrote:

> the difference between du and df is about 640 - 188 = 452 MB. and "df" is
> showing  that my root is full  2.4 times more than "du".
>
> which one is the correct one? I've another server that this difference is
> about 7 GiG and on that server root is 80% full.  The type of partition is
> ext3.

Next to the difference due journaling etc, there is one important
difference between du en df:
deleted files held open by a running process. du doesn't count these
files, df does.
You can find those files with lsof | grep "deleted". Try closing the
process with deleted files and suddenly your du en df will give the
same free diskspace.
Ofcourse, a reboot does also the trick.

Ward

Re: [gentoo-user] Re: df and du difference

[Sebastian Günther]

[-- Attachment #1: Type: text/plain, Size: 2993 bytes --]

* Platoali (platoali@gmail.com) [16.08.08 11:14]:
> Sebastian Günther wrote:
> > That is what the 5% are for, as you saw there where stated as not
> > available but they are for the superuser for such things.
> 
> So there is no way to free some space from journals.
> 
> >
> > BTW: Why is your root so full, or didn't you partionate your disk?
> 
> I did not partitioned it myself. This server is inherited to me from last 
> admin.
> 
> ~# df -h
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/sda1              19G   14G  3.5G  81% /
> varrun                2.0G   76K  2.0G   1% /var/run
> varlock               2.0G     0  2.0G   0% /var/lock
> udev                  2.0G   88K  2.0G   1% /dev
> devshm                2.0G     0  2.0G   0% /dev/shm
> /dev/sdb5              93G   59G   27G  69% /mnt/backup
> /dev/mapper/main-usr   15G  601M   14G   5% /usr
> /dev/mapper/main-var   30G  1.7G   27G   6% /var
> /dev/mapper/main-db    69G  9.5G   56G  15% /var/lib/postgresql
> /dev/sdc1              68G   35G   30G  55% /home/archive
> 
> ~#  du --max-dep 1  -c  -hx  /                        
> 4.2M    /etc                                                           
> 36M     /tftpboot                                                      
> 16K     /lost+found                                                    
> 3.8G    /tmp                                                           
There is definetly to much in it...

> 18M     /boot                                                          
> 1.4G    /home                                                          
From the df I would have thought here is more in it...

Are there any normal users on this machine

> 8.0K    /mnt                                                           
> 12K     /media                                                         
> 254M    /root                                                          
> 4.0K    /var                                                           
> 4.0K    /srv                                                           
> 0       /sys                                                           
> 4.0K    /initrd
> 77M     /lib
> 0       /proc
> 4.0K    /opt
> 4.0K    /usr
> 6.4M    /sbin
> 3.5M    /bin
> 0       /dev
> 5.5G    /
> 5.5G    total
> 
OK here is a diference to big to be normal between df and du.

14GB against 5.5GB

We are definetly missing something...

> Last week, I was alarmed  that / root is 98 percent full. but I could not find 
> any reason why server is full. and a restart freed  8 gig of space. but now it 
> is again getting full slowly. 
> 
That's /tmp: try to watch, what actually is writing in it.

> Any comment?
> 
> best wishes
> Platoali
> 


There is something wrong in the state of denmark...
Sebastian

-- 
 " Religion ist das Opium des Volkes. "      Karl Marx

 SEB@STI@N GÜNTHER         mailto:samson@guenther-roetgen.de

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

[gentoo-user] Re: df and du difference

[Platoali]

 Ward Poelmans wrote:
> You can find those files with lsof | grep "deleted". Try closing the
> process with deleted files and suddenly your du en df will give the
> same free diskspace.
> Ofcourse, a reboot does also the trick.



lsof | grep -i deleted
...
 /dev/console (deleted)
mysqld     5679    mysql    5u      REG        8,1          0    1009860 
/tmp/iby8kN8L (deleted)
mysqld     5679    mysql    6u      REG        8,1          0    1009861 
/tmp/ib3OyWjn (deleted)
mysqld     5679    mysql    7u      REG        8,1          0    1009862 
/tmp/ibCqa6uY (deleted)
mysqld     5679    mysql    8u      REG        8,1          0    1009863 
/tmp/ibnDCmHz (deleted)
mysqld     5679    mysql   12u      REG        8,1          0    1009864 
/tmp/ibaQcs5a (deleted)
...


Nothing so big. just about 20 lines and the biggest ones are these.

This server hosts accounting software for an ISP:  just a couple  python 
scripts, apache with PHP and a small Postgresql database.

Bests
Platoali
t

[gentoo-user] Re: df and du difference

[Platoali]

 Sebastian Günther wrote:
> OK here is a diference to big to be normal between df and du.
>
> 14GB against 5.5GB
>
> We are definetly missing something...
>

Yes, that is the strange thing. 

> > Last week, I was alarmed  that / root is 98 percent full. but I could not
> > find any reason why server is full. and a restart freed  8 gig of space.
> > but now it is again getting full slowly.
>
> That's /tmp: try to watch, what actually is writing in it.

I will  add a new hard and mount /tmp to it. I thing that is them most 
sensible solution.



Thanks 
Platoali

Re: [gentoo-user] Re: df and du difference

[Sebastian Günther]

[-- Attachment #1: Type: text/plain, Size: 1259 bytes --]

* Platoali (platoali@gmail.com) [16.08.08 13:13]:
>  Sebastian Günther wrote:
> > OK here is a diference to big to be normal between df and du.
> >
> > 14GB against 5.5GB
> >
> > We are definetly missing something...
> >
> 
> Yes, that is the strange thing. 
> 
This should definetly be investigated. This could be a hint that there 
is someone else using this server, e.g. it could be hacked...

> > > Last week, I was alarmed  that / root is 98 percent full. but I could not
> > > find any reason why server is full. and a restart freed  8 gig of space.
> > > but now it is again getting full slowly.
> >
> > That's /tmp: try to watch, what actually is writing in it.
> 
> I will  add a new hard and mount /tmp to it. I thing that is them most 
> sensible solution.
> 

No, you should definetly find out, who is writing such an enourmous 
amount of data into your /tmp.

This is not OK, especially, if you can't find, what this actually is.

> 
> 
> Thanks 
> Platoali
> 

A sysadmin has always to be paranoid. And if I don't know what's going 
on THEY are involved...

concerned
Sebastian

-- 
 " Religion ist das Opium des Volkes. "      Karl Marx

 SEB@STI@N GÜNTHER         mailto:samson@guenther-roetgen.de

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-user] df and du difference

[Paul Colquhoun]

On Sat, 16 Aug 2008, Ward Poelmans wrote:
> On Sat, Aug 16, 2008 at 07:50,  <platoali@gmail.com> wrote:
> 
> > the difference between du and df is about 640 - 188 = 452 MB.
> > and "df" is showing  that my root is full  2.4 times more than "du".
> >
> > which one is the correct one? I've another server that this
> > difference is about 7 GiG and on that server root is 80% full. The
> > type of partition is ext3.
> 
> Next to the difference due journaling etc, there is one important
> difference between du en df:
> deleted files held open by a running process. du doesn't count these
> files, df does.
> You can find those files with lsof | grep "deleted". Try closing the
> process with deleted files and suddenly your du en df will give the
> same free diskspace.
> Ofcourse, a reboot does also the trick.
> 
> Ward


Actually, there is one more way to hide a file from du

If there is a file in the /var directory *BEFORE* the /var partition is 
mounted onto the directory, then du won't find it, but df will know 
about the space it is using.

You will probably need to boot from a live CD of some sort to be able to 
umount the partitions and check the underlying directory, but it might 
be worth it there is still space unaccounted for after a reboot.


-- 
Reverend Paul Colquhoun, ULC.    http://andor.dropbear.id.au/~paulcol
     Asking for technical help in newsgroups?  Read this first:
        http://catb.org/~esr/faqs/smart-questions.html#intro

Re: [gentoo-user] df and du difference

[Alan McKinnon]

On Sunday 17 August 2008 01:18:21 Paul Colquhoun wrote:
> Actually, there is one more way to hide a file from du
>
> If there is a file in the /var directory *BEFORE* the 
/var partition is
> mounted onto the directory, then du won't find it, but 
df will know
> about the space it is using.
>
> You will probably need to boot from a live CD of some 
sort to be able to
> umount the partitions and check the underlying 
directory, but it might
> be worth it there is still space unaccounted for after a 
reboot.

There's a much easier way. As root:

mount -o bind / /path/to/some/arb/dir

see man mount
-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] df and du difference

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 754 bytes --]

On Sat, 16 Aug 2008 10:20:49 +0430, platoali@gmail.com wrote:

> the difference between du and df is about 640 - 188 = 452 MB. and "df"
> is showing  that my root is full  2.4 times more than "du".

There is another possibility that no one seems t have mentioned. You may
have files hidden inside a mount point that du is unable to see. To test

mkdir /mnt/tmp
mount --bind / /mnt/tmp

then run your du commands over /mnt/tmp.


-- 
Neil Bothwick

"There are two ways of constructing a software design: One way is to make
it so simple that there are obviously no deficiencies, and the other way
is to make it so complicated that there are no obvious deficiencies.
The first method is far more difficult" -C.A.R. Hoare
Neil Bothwick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-user] df and du difference

[Norberto Bensa]

Quoting Paul Colquhoun <paulcol@andor.dropbear.id.au>:

> On Sat, 16 Aug 2008, Ward Poelmans wrote:
>> On Sat, Aug 16, 2008 at 07:50,  <platoali@gmail.com> wrote:
>>
>> > the difference between du and df is about 640 - 188 = 452 MB.
>> > and "df" is showing  that my root is full  2.4 times more than "du".

Normal...

>> Next to the difference due journaling etc, there is one important
>> difference between du en df:

Hm. Yeah... Maybe. Journaling add -in my experience- about 32MB.

>> deleted files held open by a running process. du doesn't count these
>> files, df does.

Yeap.

> If there is a file in the /var directory *BEFORE* the /var partition is
> mounted onto the directory, then du won't find it, but df will know
> about the space it is using.

Yes

But you're all missing rounding errors. If you do:

du -hcs /path/to/some/dir

You'll end up with a different result if you instead do:

du -bcs /path/to/some/dir

Real example:

zoolook@melnitz ~ $ du -bcs Desktop/
1289720534      Desktop/
1289720534      total

zoolook@melnitz ~ $ du -hcs Desktop/
1.3G    Desktop/
1.3G    total

If you do the math, 1.3G is more or less 1395864372 bytes; ~110MB in  
diference and same tool.

Regards,
Norberto

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

[gentoo-user] Re: df and du difference

[Platoali]

 Alan McKinnon wrote:
> On Sunday 17 August 2008 01:18:21 Paul Colquhoun wrote:
> > Actually, there is one more way to hide a file from du
> >
> > If there is a file in the /var directory *BEFORE* the
>
> /var partition is
>
> > mounted onto the directory, then du won't find it, but
>
> df will know
>
> > about the space it is using.
> >
> > You will probably need to boot from a live CD of some
>
> sort to be able to
>
> > umount the partitions and check the underlying
>
> directory, but it might
>
> > be worth it there is still space unaccounted for after a
>
> reboot.
>
> There's a much easier way. As root:
>
> mount -o bind / /path/to/some/arb/dir
>
> see man mount


Thank you very much. That was the problem. some files have been hidden in 
/mnt/backup.

I deleted them and problem is solved.

Thanks again
Platoali

Re: [gentoo-user] Re: df and du difference

[kashani]

Platoali wrote:
>  /dev/console (deleted)
> mysqld     5679    mysql    5u      REG        8,1          0    1009860 
> /tmp/iby8kN8L (deleted)
> mysqld     5679    mysql    6u      REG        8,1          0    1009861 
> /tmp/ib3OyWjn (deleted)
> mysqld     5679    mysql    7u      REG        8,1          0    1009862 
> /tmp/ibCqa6uY (deleted)
> mysqld     5679    mysql    8u      REG        8,1          0    1009863 
> /tmp/ibnDCmHz (deleted)
> mysqld     5679    mysql   12u      REG        8,1          0    1009864 
> /tmp/ibaQcs5a (deleted)
> ...
> 
> 
> Nothing so big. just about 20 lines and the biggest ones are these.
> 
> This server hosts accounting software for an ISP:  just a couple  python 
> scripts, apache with PHP and a small Postgresql database.

	You're going to have to rebuild this server because someone is 
eventually going to break it.

The number one rule of shared database servers is never put /tmp inside 
/ because eventually some idiot will kick off some poorly thought out 
job to crunch some numbers and he will fill /tmp and therefore / and 
break your server. /tmp should always be it's own partition in this type 
of environment. I have also found 5GB to be a good size as well since 
most crazy jobs would die around 4GB on 32 bit systems.

kashani