[gentoo-amd64] Whats using all my disc space?

[gentoo-amd64] Whats using all my disc space?

[Paul Stear]

Hello all,
How do I list the contents of a disc in size order?
I need to find out the largest files on a disc. It's my home dir which is 
98% full, that's over 180GB used, normally its less than half of this.
I just can't find anything that is very large.

Thanks in advance for any help
Paul
-- 
This message has been sent using kmail with gentoo linux

Re: [gentoo-amd64] Whats using all my disc space?

[Greg]

Try du -h

du shows disk usage, and the -h flag should be for human readable output as applied to the sizes I believe.

Re: [gentoo-amd64] Whats using all my disc space?

[Mark Haney]

Paul Stear wrote:
> Hello all,
> How do I list the contents of a disc in size order?
> I need to find out the largest files on a disc. It's my home dir which is 
> 98% full, that's over 180GB used, normally its less than half of this.
> I just can't find anything that is very large.
> 
> Thanks in advance for any help
> Paul

Do you typically just move files to the trash instead of deleting them 
outright?  I had this problem and it turns out emptying the trash wasn't 
working on my system, I had to manually go into the .Trash directory and 
clear it out.


-- 
Libenter homines id quod volunt credunt -- Caius Julius Caesar


Mark Haney
Sr. Systems Administrator
ERC Broadband
(828) 350-2415

Call (866) ERC-7110 for after hours support

Re: [gentoo-amd64] Whats using all my disc space?

[Robert Cernansky]

On Wed, 15 Oct 2008 16:35:21 +0100 Paul Stear <gentoo@appjaws.plus.com> wrote:

> I need to find out the largest files on a disc. It's my home dir
> which is 98% full, that's over 180GB used, normally its less than
> half of this.  I just can't find anything that is very large.

Use gdmap (http://gdmap.sourceforge.net/). Unfortunately it is not in
portage. The old 0.7.5 version is in sunrise overlay.

Robert


-- 
Robert Cernansky
E-mail: hslists2@zoznam.sk
Jabber: hs@jabber.sk

Re: [gentoo-amd64] Whats using all my disc space?

[Pascal BERTIN]

Robert Cernansky a écrit :
> On Wed, 15 Oct 2008 16:35:21 +0100 Paul Stear <gentoo@appjaws.plus.com> wrote:
> 
>> I need to find out the largest files on a disc. It's my home dir
>> which is 98% full, that's over 180GB used, normally its less than
>> half of this.  I just can't find anything that is very large.
> 
> Use gdmap (http://gdmap.sourceforge.net/). Unfortunately it is not in
> portage. The old 0.7.5 version is in sunrise overlay.
> 
> Robert
> 
> 

Or if you use KDE swith to the "file size display" (not sure of the translation.

but if you prefer command line,
du --max-depth <X> |sort -n, replace <X> by a depth level should allow you to find where 
space is used.

Re: [gentoo-amd64] Whats using all my disc space?

[Justin]

[-- Attachment #1: Type: text/plain, Size: 582 bytes --]

Robert Cernansky schrieb:
> On Wed, 15 Oct 2008 16:35:21 +0100 Paul Stear <gentoo@appjaws.plus.com> wrote:
>
>   
>> I need to find out the largest files on a disc. It's my home dir
>> which is 98% full, that's over 180GB used, normally its less than
>> half of this.  I just can't find anything that is very large.
>>     
>
> Use gdmap (http://gdmap.sourceforge.net/). Unfortunately it is not in
> portage. The old 0.7.5 version is in sunrise overlay.
>
> Robert
>
>
>   
It is in my overlay:
http://gentoo.j-schmitz.net/portage/ebuilds/gnome-extra/gdmap/




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: [gentoo-amd64] Whats using all my disc space?

[Tonko Mulder]

I don't know if you use gnome, but if you do there's a tool called
Baobab in it.
You can easily check sizes with it

On Wed, Oct 15, 2008 at 5:52 PM, Robert Cernansky <hslists2@zoznam.sk> wrote:
> On Wed, 15 Oct 2008 16:35:21 +0100 Paul Stear <gentoo@appjaws.plus.com> wrote:
>
>> I need to find out the largest files on a disc. It's my home dir
>> which is 98% full, that's over 180GB used, normally its less than
>> half of this.  I just can't find anything that is very large.
>
> Use gdmap (http://gdmap.sourceforge.net/). Unfortunately it is not in
> portage. The old 0.7.5 version is in sunrise overlay.
>
> Robert
>
>
> --
> Robert Cernansky
> E-mail: hslists2@zoznam.sk
> Jabber: hs@jabber.sk
>
>
>



-- 
Tonko

Re: [gentoo-amd64] Whats using all my disc space?

[andi]

Try "ncdu".

It's a great console-app to find and delete garbage...!

On Wednesday 15 October 2008, Paul Stear wrote:
> Hello all,
> How do I list the contents of a disc in size order?
> I need to find out the largest files on a disc. It's my home dir which is 
> 98% full, that's over 180GB used, normally its less than half of this.
> I just can't find anything that is very large.
> 
> Thanks in advance for any help
> Paul
-- 
>++++++++++[<++++++++++>-]<---.
>+++[<++++>-]<+.
>-----[<++>-]<.
+++++.
>----[<++++++++++>-]<-.
>++++++++++[<++++>-]<-.
>+++[<+++>-]<++.
-----.
+++.
>---[<+++>-]<-.
>+++++[<---------->-]<------.
>++++++++++[<++++++>-]<--.
>+++[<++>-]<+.
--.
>--[<++++>-]<.
>++[<++++++++>-]<.
-------.
>--[<++>-]<-.
>+++++[<+++>-]<.
>----[<++++++++++++++++++++>-]<++++++.
>++++++++++[<+++++>-]<+++++++++++++++.
+++.
>---[<+++>-]<--.

Re: [gentoo-amd64] Whats using all my disc space?

[Fernando Boaglio]

[-- Attachment #1: Type: text/plain, Size: 120 bytes --]

Sometimes I run this graphic report:

http://www.jgoodies.com/freeware/jdiskreport/features.html

[]'s
Fernando Boaglio

[-- Attachment #2: Type: text/html, Size: 244 bytes --]

Re: [gentoo-amd64] Whats using all my disc space?

[Nicolas Sebrecht]

On Wed, Oct 15, 2008 at 06:02:55PM +0200, andi wrote:

> Try "ncdu".

Nice tool. Adopted.

-- 
Nicolas Sebrecht

Re: [gentoo-amd64] Whats using all my disc space?

[gentoo]

On Wednesday 15 October 2008 17:02:55 andi wrote:
> Try "ncdu".
>
> It's a great console-app to find and delete garbage...!
Thanks for all who responded.
I have now deleted a load of redundant files and have some space back.
Thanks again
Paul

Re: [gentoo-amd64] Whats using all my disc space?

[Mike Doty]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul Stear wrote:
> Hello all,
> How do I list the contents of a disc in size order?
> I need to find out the largest files on a disc. It's my home dir which is 
> 98% full, that's over 180GB used, normally its less than half of this.
> I just can't find anything that is very large.
> 
> Thanks in advance for any help
> Paul
for F in $(find . -type f); do ls -l ${F}; done | awk '{print $5, $9}' |
sort -nr | head -n 10

- --
=======================================================
Mike Doty                      kingtaco -at- gentoo.org
Gentoo Infrastructure
Gentoo/AMD64 Strategic Lead
GPG: E1A5 1C9C 93FE F430 C1D6  F2AF 806B A2E4 19F4 AE05
=======================================================
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iJwEAQECAAYFAkj2wf4ACgkQgGui5Bn0rgUkkAP8CDcREhu0nSKYJKoQcAw6WkTH
7Ze4dp9zRhWrEdsEqLpAVJrE2EcfJzHkpNm9M4JbcP/LF0l+v1imEcd2cRTS4BPX
W2UHhFvB5rkXRe8P9DLWs+1TCkg7dR4zxJQJ36Q+SNBZbNih4yeFWrvOrhnrF+UB
5OfC1/Wt6CDmU4frD+0=
=UMi4
-----END PGP SIGNATURE-----

Re: [gentoo-amd64] Whats using all my disc space?

[Pascal BERTIN]

Mike Doty a écrit :
> Paul Stear wrote:
>> Hello all,
>> How do I list the contents of a disc in size order?
>> I need to find out the largest files on a disc. It's my home dir which is 
>> 98% full, that's over 180GB used, normally its less than half of this.
>> I just can't find anything that is very large.
> 
>> Thanks in advance for any help
>> Paul
> for F in $(find . -type f); do ls -l ${F}; done | awk '{print $5, $9}' |
> sort -nr | head -n 10
> 

Well, this might simplify your life then :
find . -type f -printf "%s %p\n" |sort -nr |head -n 10

and if you expect the big files to be bigger than say 50M, use this one :
find . -type f -size +50M -printf "%s %p\n" |sort -nr |head -n 10

but this will only report files, while du --max-depth would also report big folders 
(containing tons of small files)

Re: [gentoo-amd64] Whats using all my disc space?

[Richard Freeman]

[-- Attachment #1: Type: text/plain, Size: 543 bytes --]

Paul Stear wrote:
> Hello all,
> How do I list the contents of a disc in size order?
> I need to find out the largest files on a disc. It's my home dir which is 
> 98% full, that's over 180GB used, normally its less than half of this.
> I just can't find anything that is very large.
> 

To add to the chorus of suggestions, may I offer "kdirstat"?  It is in 
portage and does a great job of mapping file use, as well as some 
administrative tools for cleanup.  Just be careful when deleting files 
that you don't just move them to the trash.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3670 bytes --]

Re: [gentoo-amd64] Whats using all my disc space?

[Paul Stear]

On Thursday 16 October 2008 15:52:59 Richard Freeman wrote:

> To add to the chorus of suggestions, may I offer "kdirstat"?  It is in
> portage and does a great job of mapping file use, as well as some
> administrative tools for cleanup.  Just be careful when deleting files
> that you don't just move them to the trash.

Well thanks again for all responses, kdirstat is now emerged and looks good at 
identifying all my rubbish.
Paul

-- 
This message has been sent using kmail with gentoo linux

Re: [gentoo-amd64] Whats using all my disc space?

[Michael Rock]

[-- Attachment #1: Type: text/plain, Size: 7964 bytes --]

Did anyone suggest fslint?  Apart from finding unnecessary files, duplicates

and broken links, there's an education to be had in clever scripting behind
it all:
*********************************************************************************************************
#!/bin/bash

# findup - find duplicate files
# Copyright (c) 2000-2006 by Pádraig Brady <P@draigBrady.com>.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# See the GNU General Public License for more details,
# which is available at www.gnu.org


# Description
#
#    will show duplicate files in the specified directories
#    (and their subdirectories), in the format:
#
#    2 * 2048    file1 file2
#    3 * 1024    file3 file4 file5
#    2 * 1024    file6 file7
#
#    Where the number is the disk usage in bytes of each of the
#    duplicate files on that line, and all duplicate files are
#    shown on the same line.
#        Output it ordered by largest disk usage first and
#    then by the number of duplicate files.
#
# Caveats/Notes:
#    I compared this to any equivalent utils I could find (as of Nov 2000)
#    and it's (by far) the fastest, has the most functionality (thanks to
#    find) and has no (known) bugs. In my opinion fdupes is the next best
but
#    is slower (even though written in C), and has a bug where hard links
#    in different directories are reported as duplicates sometimes.
#
#    This script requires uniq > V2.0.21 (part of GNU textutils|coreutils)
#    undefined operation if any dir/file names contain \n or \\
#    sparse files are not treated differently.
#    Don't specify params to find that affect output etc. (e.g -printf etc.)
#    zero length files are ignored.
#    symbolic links are ignored.
#    path1 & path2 can be files &/or directories

script_dir=`dirname $0`                #directory of this script
script_dir=`readlink -f "$script_dir"` #Make sure absolute path

. $script_dir/supprt/fslver

Usage() {
    ProgName=`basename "$0"`
    echo "find dUPlicate files.
Usage: $ProgName [[-t [-m|-d]] [-r] [-f] paths(s) ...]

If no path(s) specified then the currrent directory is assumed.

When -m is specified any found duplicates will be merged (using hardlinks).
When -d is specified any found duplicates will be deleted (only 1 left).
When -t is specfied, only report what -m or -d would do.

You can also pipe output to $script_dir/fstool/dupwaste to
get a total of the wastage due to duplicates.

Examples:

search for duplicates in current directory and below
    findup or findup .
search for duplicates in all linux source directories and merge using
hardlinks
    findup -m /usr/src/linux*
same as above but don't look in subdirectories
    findup -r .
search for duplicates in /usr/bin
    findup /usr/bin
search in multiple directories but not their subdirectories
    findup -r /usr/bin /bin /usr/sbin /sbin
search for duplicates in \$PATH
    findup \`$script_dir/supprt/getffp\`
search system for duplicate files over 100K in size
    findup / -size +100k
search only my files (that I own and are in my home dir)
    findup ~ -user \`id -u\`
search system for duplicate files belonging to roger
    findup / -user \`id -u roger\`"
    exit
}

for arg
do
    case "$arg" in
    -h|--help|-help)
        Usage ;;
    -v|--version)
        Version ;;
    --gui)
        mode="gui" ;;
    -m)
        mode="merge" ;;
    -d)
        mode="del" ;;
    -t)
        t="t" ;;
    *)
                argsToPassOn="$argsToPassOn '$arg'"
    esac
done
[ "$mode" = "merge" ] && argsToPassOn="$argsToPassOn -xdev"

if [ ! -z "$mode" ]; then
    forceFullPath="-f"
    sep_mode="prepend"
else
    sep_mode="none"
fi

if [ "$mode" = "gui" ] || [ "$mode" = "merge" ] || [ "$mode" = "del" ]; then
    merge_early=""    #process hardlinks
else
    merge_early="-u"  #ignore hardlinks
fi

. $script_dir/supprt/getfpf $forceFullPath "$argsToPassOn"

check_uniq

if [ `find . -maxdepth 0 -printf "%D" 2> /dev/null` = "D" ]
then
    devFmt="\060" #0
else
    devFmt=%D #This is new and will help find more duplicate files
fi
                     #print name, inode & size.
find "$@" -size +0c -type f -printf "$FPF\0$devFmt\0%i\0%s\n" |
tr ' \t\0' '\0\1 ' | #remove spaces, tabs in file names
sort -k2,2n -k4,4nr -k3,3 $merge_early |#group [and merge] dev,size & inodes
if [ -z "$merge_early" ]; then
    $script_dir/supprt/rmlint/merge_hardlinks
else
    uniq -3 -D       #pick just duplicate filesizes
fi |
sort -k3,3n |        #NB sort inodes so md5sum does less seeking all over
disk
cut -f1 -d' ' -s |   #get filenames to work on
tr '\0\1\n' ' \t\0' |#reset any space & tabs etc and delimit names with \0
xargs -r0 md5sum -- |#calculate md5sums for possible duplicates
sort |               #group duplicate files together
tr ' \t' '\1\2' |    #remove spaces & tabs again (sed can't match \0)
sed -e 's/\(^.\{32\}\)..\(.*\)/\2 \1/' | #switch sums and filenames

# The following optional block, checks duplicates again using sha1
# Note for data sets that don't totally fit in cache this will
# probably read duplicate files off the disk again.
uniq --all-repeated -1 | #pick just duplicates
cut -d' ' -f1 |          #get filenames
sort |                   #sort by paths to try to minimise disk seeks
tr '\1\2\n' ' \t\0' |    #reset any space & tabs etc and delimit names with
\0
xargs -r0 sha1sum -- |   #to be sure to be sure
sort |                   #group duplicate files together
tr ' \t' '\1\2' |        #remove spaces & tabs again (sed can't match \0)
sed -e 's/\(^.\{40\}\)..\(.*\)/\2 \1/' | #switch sums and filenames

uniq --all-repeated=$sep_mode -1 | #pick just duplicates
sed -e 's/\(^.*\) \(.*\)/\2 \1/' | #switch sums and filenames back
tr '\1\2' ' \t' |    #put spaces & tabs back

if [ ! -z "$mode" ]; then
  cut -d' ' -f2- |
  if [ ! $mode = "gui" ]; then # external call to python as this is faster
    if [ -f $script_dir/supprt/rmlint/fixdup.py ]; then
        $script_dir/supprt/rmlint/fixdup.py $t$mode
    elif [ -f $script_dir/supprt/rmlint/fixdup.sh ]; then
        $script_dir/supprt/rmlint/fixdup.sh $t$mode
    else
        echo "Error, couldn't find merge util" >&2
        exit 1
    fi
  else
    cat
  fi
else
(
psum='no match'
line=''
declare -i counter
while read sum file; do           #sum is delimited by first space
  if [ "$sum" != "$psum" ]; then
    if [ ! -z "$line" ]; then
       echo "$counter * $line"
    fi
    counter=1
    line="`du -b "$file"`"
    psum="$sum"
  else
    counter=counter+1             #Use bash arithmetic, not expr (for speed)
    line="$line $file"
  fi
done

if [ ! -z "$line" ]; then
  echo "$counter * $line"
fi
) |
sort -k3,3 -k1,1 -brn
fi
*************************************************************************************************************


On Fri, Oct 17, 2008 at 7:12 AM, Paul Stear <gentoo@appjaws.plus.com> wrote:

> On Thursday 16 October 2008 15:52:59 Richard Freeman wrote:
>
> > To add to the chorus of suggestions, may I offer "kdirstat"?  It is in
> > portage and does a great job of mapping file use, as well as some
> > administrative tools for cleanup.  Just be careful when deleting files
> > that you don't just move them to the trash.
>
> Well thanks again for all responses, kdirstat is now emerged and looks good
> at
> identifying all my rubbish.
> Paul
>
> --
> This message has been sent using kmail with gentoo linux
>
>

[-- Attachment #2: Type: text/html, Size: 11852 bytes --]

Re: [gentoo-amd64] Whats using all my disc space?

[Peter Humphrey]

On Thursday 16 October 2008 15:52:59 Richard Freeman wrote:
> Paul Stear wrote:
> > Hello all,
> > How do I list the contents of a disc in size order?
> > I need to find out the largest files on a disc. It's my home dir which
> > is 98% full, that's over 180GB used, normally its less than half of
> > this. I just can't find anything that is very large.
>
> To add to the chorus of suggestions, may I offer "kdirstat"?  It is in
> portage and does a great job of mapping file use, as well as some
> administrative tools for cleanup.  Just be careful when deleting files
> that you don't just move them to the trash.

When I "emerge -pv kdirstat" I get 

!!! The following installed packages are masked:
- cross-i686-pc-linux-gnu/linux-headers-2.6.23-r3 (masked by: ~amd64 
keyword)

Why on earth should kdirstat depend on a cross-compiler for a different 
architecture?

On the other hand, when I "ACCEPT_KEYWORDS=~amd64 emerge -pvt kdirstat" I 
get

[ebuild  N    ] kde-misc/kdirstat-2.5.3-r1  USE="-debug -xinerama" 662 kB

Total: 1 package (1 new), Size of downloads: 662 kB

Something's a bit out of kilter, methinks. "equery l cross" returns no 
entries.

-- 
Rgds
Peter

Re: [gentoo-amd64] Whats using all my disc space?

[Richard Freeman]

Paul Stear wrote:
> Hello all,
> How do I list the contents of a disc in size order?
> I need to find out the largest files on a disc. It's my home dir which is 
> 98% full, that's over 180GB used, normally its less than half of this.
> I just can't find anything that is very large.
> 

To add to the chorus of suggestions, may I offer "kdirstat"?  It is in
portage and does a great job of mapping file use, as well as some
administrative tools for cleanup.  Just be careful when deleting files
that you don't just move them to the trash.

Re: [gentoo-amd64] Whats using all my disc space?

[Florian D.]

Paul Stear wrote:
> Hello all,
> How do I list the contents of a disc in size order?
> I need to find out the largest files on a disc. It's my home dir which is 
> 98% full, that's over 180GB used, normally its less than half of this.
> I just can't find anything that is very large.
> 
> Thanks in advance for any help
> Paul
funny that nobody mentioned kde-misc/filelight so far. its stable,
compiles cleanly and integrates in konqueror

[gentoo-amd64] Re: Whats using all my disc space?

[Duncan]

"Florian D." <flockmock@gmx.at> posted 48FA29B6.3090208@gmx.at, excerpted
below, on  Sat, 18 Oct 2008 20:23:50 +0200:

> funny that nobody mentioned kde-misc/filelight so far. its stable,
> compiles cleanly and integrates in konqueror

I'm guessing the standard konqueror filesize view is sufficient for most 
folks using KDE (3.5 anyway, and I'd imagine 4.x has similar), so there's 
little reason to go looking for other solutions.  Similarly for those who 
know their CLI apps, du works well, so again, little reason to chase 
after other solutions.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman