[gentoo-user] File system full issues

[gentoo-user] File system full issues

[Mark Knecht]

Hi,
   On this machine the file system reports it's 100% full even after
I've removed 500MB of stuff. What can I do to clean this up?


dragonfly / # df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/hda8              9621848   9161608         0 100% /

Thanks,
Mark

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] File system full issues

[Mike Williams]

On Wednesday 15 February 2006 11:23, Mark Knecht wrote:
>    On this machine the file system reports it's 100% full even after
> I've removed 500MB of stuff. What can I do to clean this up?

Remove more.

I suspect that's an ext{2,3} filesystem, which has, by default, 5% set aside 
for use only by the superuser. You've gone into that 5%, and 500MB isn't 
enough to get you out, hence it still appears 100% full.

-- 
Mike Williams

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] File system full issues

[Mark Knecht]

On 2/15/06, Mike Williams <mike@gaima.co.uk> wrote:
> On Wednesday 15 February 2006 11:23, Mark Knecht wrote:
> > On this machine the file system reports it's 100% full even after
> > I've removed 500MB of stuff. What can I do to clean this up?
>
> Remove more.
>
> I suspect that's an ext{2,3} filesystem, which has, by default, 5% set aside
> for use only by the superuser. You've gone into that 5%, and 500MB isn't
> enough to get you out, hence it still appears 100% full.
>
> --
> Mike Williams

OK, good info - but what can I remove? Or more important how can I
find what's talking up too much space. /home, /usr/portage and /var
are on partitions of their own. There is about 200MB of Java stuff in
/opt and I deleted everything in /tmp before I wrote the first note.

I do appear to have about 250MB of KDE stuff in /usr/kde. We don't use
KDE but there are some KDE type apps, like k3b, on this machine.

I have about 1.1GB in /usr/lib but I wouldn't know how to touch that by hand.

/usr/share has about 850MB in it. Again, I wouldn't know how to touch
that by hand.

I appear to have kdebase, kdelibs, kdebase-pam and kde-env installed.
Which of those could come out without causing major problems for a
non-KDE user? (I did install KDE about a month ago, just to try it
out, but we don't use it. I suppose I could remove them all and then
go through a revdep-rebuild process...

The issue here is that this machine is both my wife's desktop machine
as well as our MythTV backend server. There is no video on the
machine. It's kept elsewhere on an NFS mount, but MythTV has stopped
working and the best guess the Myth folks had so far was lack of disk
space. When I first looked the machine was very full, but now with the
current clear space it's still having troubles.

Thanks,
Mark

-- 
gentoo-user@gentoo.org mailing list

[gentoo-user] Re: File system full issues

[James]

Mark Knecht <markknecht <at> gmail.com> writes:


> OK, good info - but what can I remove? Or more important how can I
> find what's talking up too much space. /home, /usr/portage and /var
> are on partitions of their own. There is about 200MB of Java stuff in
> /opt and I deleted everything in /tmp before I wrote the first note.

Well the first thing you need to do, is run a 'df' and see which
partitions are full. Then used these tools to find files by size and
date. Let's assume we're talking about /usr/portage/distfiles

for example:

REMOVING LARGE FILES IN /usr/portage/distfiles
find ./ -size +100000  -exec ls -lag {} \; | less       <lists large files>
find ./ -size +100000 -print -exec rm {} \;     <prints & removes large files>


You can then changethe size and work your way down. 

Now let's look at old files in /usr/portage/disfiles

REMOVING OLD FILES IN /usr/protage/distfiles
find ./ -mtime +180 -exec ls -lag {} \; | less
find ./ -mtime +180 -print -exec rm {} \;


It really helps if you do this a the 90% point, and avoid
sluggish behavior.

Look at /home too. If you have one big partition, as recommended
in the handbook then removing any files will help.

/usr/src/* is another place to remove kernel sources, similarly,
/boot/

Be cautions!

hth,
James

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: File system full issues

[Maarten]

James wrote:
> Mark Knecht <markknecht <at> gmail.com> writes:
> 
> 
> 
>>OK, good info - but what can I remove? Or more important how can I
>>find what's talking up too much space. /home, /usr/portage and /var
>>are on partitions of their own. There is about 200MB of Java stuff in
>>/opt and I deleted everything in /tmp before I wrote the first note.

The first question to ask is "Did it fill up slowly or has it filled up
 fairly quickly?"  If it was quick, chances are a runaway process did
something, like write a big coredump, a log which filled up due to
errors, a file copy that didn't fit, stuff like that.

Now you mention that the most likely point for that trouble are
accounted for (/tmp, /var). You also mention you use MythTV over the
network. So I would look at the (NFS?) mountpoint first; it happened to
me that the share wasn't mounted, and mythtv then happily fills up the
partition on which the mountpoint resides, ie. /.
So run a 'du -s' on /mnt/* (if that is your mountpoint) after unmounting
all network shares (a mounted dir can hide a file!).
Next, run du -s on the files and dirs in /root/. Maybe logging or cache
was written there (don't forget the dot-entries!)
Also, look into /usr/src and remove sources you can miss.

If nothing helps there, you best run a 'du -s /*' but you better umount
everything that isn't essential first.  And even then, it's sloooow...

In the case it filled up slowly you are worse off. You then know you
have to remove stuff you may need instead of just finding 'the culprit'.
Look carefully into the entire tree and decide what you need and what
you can do without. Likely candidates are Howtos and such, but they
reside under /usr/share so it's likely they aren't on your / filesystem.
They are small, too, so you don't gain that much.
One you have space, you can gain much more by gzipping stuff (especially
mysqldumps, logs, etc) but as you stated /var is on it's own partition I
fear there will not be much to gzip anyway...

One last remark: It happened to me on occasion that a filesystem keeps
at 100% full until you reboot the box. It may be that a process still
has files open, or that an fsck is in order, or whatever.  This is the
reason you should never let / fill up; it doesn't always recover very
gracefully, at least that has been my experience.  I still have to guess
at the exact reason that stays-at-100% happens, if anyone can explain...

Good luck,
Maarten

> 
> Well the first thing you need to do, is run a 'df' and see which
> partitions are full. Then used these tools to find files by size and
> date. Let's assume we're talking about /usr/portage/distfiles
> 
> for example:
> 
> REMOVING LARGE FILES IN /usr/portage/distfiles
> find ./ -size +100000  -exec ls -lag {} \; | less       <lists large files>
> find ./ -size +100000 -print -exec rm {} \;     <prints & removes large files>
> 
> 
> You can then changethe size and work your way down. 
> 
> Now let's look at old files in /usr/portage/disfiles
> 
> REMOVING OLD FILES IN /usr/protage/distfiles
> find ./ -mtime +180 -exec ls -lag {} \; | less
> find ./ -mtime +180 -print -exec rm {} \;
> 
> 
> It really helps if you do this a the 90% point, and avoid
> sluggish behavior.
> 
> Look at /home too. If you have one big partition, as recommended
> in the handbook then removing any files will help.
> 
> /usr/src/* is another place to remove kernel sources, similarly,
> /boot/
> 
> Be cautions!
> 
> hth,
> James
> 

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: File system full issues

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 360 bytes --]

On Wed, 15 Feb 2006 19:09:05 +0100, Maarten wrote:

> So run a 'du -s' on /mnt/* (if that is your mountpoint) after unmounting
> all network shares (a mounted dir can hide a file!).

Or you could save unmounting anything by doing

mount --bind / /mnt/tmp
du -sch /mnt/tmp/*


-- 
Neil Bothwick

Quantum Physics: The dreams that stuff is made of

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Re: File system full issues

[Mark Knecht]

On 2/15/06, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Wed, 15 Feb 2006 19:09:05 +0100, Maarten wrote:
>
> > So run a 'du -s' on /mnt/* (if that is your mountpoint) after unmounting
> > all network shares (a mounted dir can hide a file!).
>
> Or you could save unmounting anything by doing
>
> mount --bind / /mnt/tmp
> du -sch /mnt/tmp/*
>
>
> --
> Neil Bothwick

Hi Neil and others,
   In the end there were a few reasons disk space got filled up. The
main culprit was that watching live TV within MythTV creates a ring
buffer file. In my case it was about 750MB. Normally this file is
erased but for some reason it was left behind sometime recently. Add a
few new emerges to the system and we went over the top.

   After removing a few unnecessary packages and finding the one large
file I was back to 83% disk usage. At that point MythTV started doing
it's updates again and everything is back to normal.

   Thanks to those who answered. I've saved your responses for this
sort of problem in the future.

Cheers,
Mark

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] File system full issues

[Jürgen Pierau]

Mark Knecht wrote:

>OK, good info - but what can I remove? Or more important how can I
>find what's talking up too much space. /home, /usr/portage and /var
>are on partitions of their own. There is about 200MB of Java stuff in
>/opt and I deleted everything in /tmp before I wrote the first note.
>
>I do appear to have about 250MB of KDE stuff in /usr/kde. We don't use
>KDE but there are some KDE type apps, like k3b, on this machine.
>  
>
One thing that frequently gets me is /usr/src. Every version of the 
Linux kernel takes up something between 200 and 300 MB (more depending 
on how you formatted the underlying partition). So if you inadvertently 
have created a nice archive of kernel sources from the last two years in 
there, unmerge all the old ones. sys-kernel/gentoo-sources is a slotted 
install, so each update ADDS the new kernel sources and doesn't remove 
the old ones.

Just my 0.02$
Jürgen
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] File system full issues

[Frédéric Grosshans]

Le mercredi 15 février 2006 à 04:42 -0800, Mark Knecht a écrit :

> OK, good info - but what can I remove? Or more important how can I
> find what's talking up too much space. 

I know you've already solved that problem, but I think the following
might be interesting.

I found xdiskusage to be a very practical tool to findout where space is
wasted on a disk. It's basically a tool giving a graphical output to du,
showing how the space is shared by directory and subdirectories (and
files with the -a option).

	Fred

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] File system full issues

[Mark Knecht]

On 2/16/06, Frédéric Grosshans
<frederic.grosshans.1995@polytechnique.org> wrote:
> Le mercredi 15 février 2006 à 04:42 -0800, Mark Knecht a écrit :
>
> > OK, good info - but what can I remove? Or more important how can I
> > find what's talking up too much space.
>
> I know you've already solved that problem, but I think the following
> might be interesting.
>
> I found xdiskusage to be a very practical tool to findout where space is
> wasted on a disk. It's basically a tool giving a graphical output to du,
> showing how the space is shared by directory and subdirectories (and
> files with the -a option).
>
>         Fred

Thanks Fred. It looks like a helpful little app. I certainly would
have found the offending MythTV buffer file more quickly with it.

Cheers,
Mark

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] File system full issues

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]

On Thu, 16 Feb 2006 07:50:01 -0800, Mark Knecht wrote:

> > I found xdiskusage to be a very practical tool to findout where space
> > is wasted on a disk. It's basically a tool giving a graphical output
> > to du, showing how the space is shared by directory and
> > subdirectories (and files with the -a option).

> Thanks Fred. It looks like a helpful little app. I certainly would
> have found the offending MythTV buffer file more quickly with it.

Filelight is another useful program here... and with more eye-candy :)


-- 
Neil Bothwick

"MSDOS didn't get as bad as it is overnight -- it took over ten years
of careful development."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-user] File system full issues

[Frédéric Grosshans]

Le jeudi 16 février 2006 à 16:32 +0000, Neil Bothwick a écrit :
> On Thu, 16 Feb 2006 07:50:01 -0800, Mark Knecht wrote:
> 
> > > I found xdiskusage to be a very practical tool to findout where space
> > > is wasted on a disk. 

[...]
> Filelight is another useful program here... and with more eye-candy :)
Stupid question : Is there a gnome equivalent ? (I like candy !)
Obvious answer : apparently not easily found with Google. But I just ask
to be proven wrong...

	Fred

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] File system full issues

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 359 bytes --]

On Fri, 17 Feb 2006 16:19:22 +0100, Frédéric Grosshans wrote:

> > Filelight is another useful program here... and with more eye-candy :)

> Stupid question : Is there a gnome equivalent ?

Not that I know of.

> (I like candy !)

So do I, but I don't like GNOME

SCNR :)


-- 
Neil Bothwick

Politically Incorrect -- and damn proud of it!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-user] File system full issues

[Frédéric Grosshans]

Le vendredi 17 février 2006 à 16:07 +0000, Neil Bothwick wrote, using 

Sylpheed-Claws 2.0.0 (GTK+ 2.8.12;
x86_64-pc-linux-gnu)
> So do I, but I don't like GNOME
> SCNR :)

But apparently, you like GTK+ software enough to use it to write this
Gnome-bashing answer ;) 

SCNR ...

	Fred

PS: I know GTK != Gnome. I suppose you use XFCE. 

	

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] File system full issues

[Mark Knecht]

On 2/17/06, Frédéric Grosshans
<frederic.grosshans.1995@polytechnique.org> wrote:
>
> PS: I know GTK != Gnome. I suppose you use XFCE.
>
Or possibly some minimalistic environment like fluxbox?

I'd use fluxbox or something like that if there was a good way to
manage menus. Last time I used it as my main environment, now over two
years ago, they made you edit files by hand to get menus which was a
drag.

Cheers,
Mark

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] File system full issues

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

On Fri, 17 Feb 2006 17:49:07 +0100, Frédéric Grosshans wrote:

> > So do I, but I don't like GNOME
> > SCNR :)
> 
> But apparently, you like GTK+ software enough to use it to write this
> Gnome-bashing answer ;) 

Good catch :)

> PS: I know GTK != Gnome. I suppose you use XFCE. 

I use KDE for the desktop, but whatever program best suits the task in
hand. I also use The GIMP, Gnucash, VMWare and Unison, and probably
several other GTK apps I can't think of right now.

I'd say it was impossible to limit yourself to only GTK or only QT
without severely limiting your software choices.


-- 
Neil Bothwick

User-friendly: (adj.) trivialized, slow, incapable, and boring.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-user] File system full issues

[Pshem Kowalczyk]

On 16/02/06, Mark Knecht <markknecht@gmail.com> wrote:
> Hi,
>    On this machine the file system reports it's 100% full even after
> I've removed 500MB of stuff. What can I do to clean this up?
>
>
> dragonfly / # df
> Filesystem           1K-blocks      Used Available Use% Mounted on
> /dev/hda8              9621848   9161608         0 100% /
>

If the filesystem is ext2 or ext3 it's pretty likely that this 500MB
is the 5% reserved for root :-)
So you can't see it, but the space is there. Google around this, as
I'm not sure whether root should see this 5% as available or not.

regards
pshemko

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] File system full issues

[Alexander Skwar]

Pshem Kowalczyk wrote:

> So you can't see it, but the space is there. Google around this, as
> I'm not sure whether root should see this 5% as available or not.

No, I don't think that root should be able to *SEE* the
space - but root should be able to *USE* the space.

Alexander Skwar
-- 
The haughty do but build castle walls behind which they seek to hide their
doubts and fears.

  -- Bene Gesserit Axiom
-- 
gentoo-user@gentoo.org mailing list