[gentoo-user] OT: disk full but lot of space

[gentoo-user] OT: disk full but lot of space

[gentoo]

Hello,

This is OT, but I'm kind of worried and google hasn't been my friend...

* I'm doing backups of my server everynight and had the following errors in 
the mail from the cron output this morning. repeated a dozen of times.

bzip2: I/O or other error, bailing out.  Possible reason follows.
bzip2: No space left on device
        Input file = (stdin), output file = (stdout)
/local/sbin/mk-rs-backups.sh: line 26: 22516 Broken pipe             $TAR cjf 
$DSTDIR/site-$site.tar.gz $name

bzip2: I/O or other error, bailing out.  Possible reason follows.
bzip2: No space left on device
        Input file = (stdin), output file = (stdout)
tar: Child returned status 1
tar: Error exit delayed from previous errors

* checking my logs I found the following error repeating ~6000 times.

Aug  8 03:17:00 www kernel: EXT3-fs error (device ide0(3,2)): ext3_new_block: 
block(14366126) >= blocks count(14366126) - block_group = 438, es == c2510400
[...]
Aug  8 08:46:52 www kernel: EXT3-fs error (device ide0(3,2)): ext3_new_block: 
block(14372149) >= blocks count(14366126) - block_group = 438, es == c2510400

"df" is telling that my disk is only half full: ~28Gb free.
"df -i" is reporting only ~2% inodes used.

"tune2ds -l" follows

So... is my disk broken? my fs corrupted? While is ext3 trying to allocate a 
block with a higher count than the max?

Any help welcomed, thanks in advance.

[root@www log]# /sbin/tune2fs -l /dev/hda2
tune2fs 1.26 (3-Feb-2002)
Filesystem volume name:   /
Last mounted on:          <not available>
Filesystem UUID:          <none>
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal filetype needs_recovery sparse_super
Filesystem state:         clean with errors
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              7192576
Block count:              14366126
Reserved block count:     718306
Free blocks:              11154684
Free inodes:              7060644
First block:              0
Block size:               4096
Fragment size:            4096
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         16384
Inode blocks per group:   512
Last mount time:          Wed Feb 23 20:49:57 2005
Last write time:          Mon Aug  8 08:46:52 2005
Mount count:              22
Maximum mount count:      -1
Last checked:             Wed Nov 20 03:44:33 2002
Check interval:           0 (<none>)
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal UUID:             <none>
Journal inode:            8
Journal device:           0x0000
First orphan inode:       6488266

--mat
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Christoph Gysin]

gentoo@depardo.ch wrote:
> Aug  8 03:17:00 www kernel: EXT3-fs error (device ide0(3,2)): ext3_new_block: 
> block(14366126) >= blocks count(14366126) - block_group = 438, es == c2510400
> [...]
> Aug  8 08:46:52 www kernel: EXT3-fs error (device ide0(3,2)): ext3_new_block: 
> block(14372149) >= blocks count(14366126) - block_group = 438, es == c2510400

Did you try fsck?

Christoph
-- 
echo mailto: NOSPAM !#$.'<*>'|sed 's. ..'|tr "<*> !#:2" org@fr33z3
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[gentoo]

> > Aug  8 03:17:00 www kernel: EXT3-fs error (device ide0(3,2)):
> > ext3_new_block:  
> > block(14366126) >= blocks count(14366126) - block_group = 438, es ==
> > c2510400 
> > [...]
> > Aug  8 08:46:52 www kernel: EXT3-fs error (device ide0(3,2)):
> > ext3_new_block:  
> > block(14372149) >= blocks count(14366126) - block_group = 438, es ==
> > c2510400 

> Did you try fsck?

nope :-(
I'm a bit too afraid to try it. It's a production server with only one 
partition and it's located about 6000 miles away. If something goes wrong, 
I'll be really in a bad situation.

I think I'll have to do it anyway, but I'm checking for lighter solutions 
first...

--mat
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Norberto Bensa]

[-- Attachment #1: Type: text/plain, Size: 408 bytes --]

gentoo@depardo.ch wrote:
> bzip2: I/O or other error, bailing out.  Possible reason follows.
> bzip2: No space left on device
>         Input file = (stdin), output file = (stdout)
> tar: Child returned status 1
> tar: Error exit delayed from previous errors
>


Check your temp partitions...


-- 
Norberto Bensa
informática BeNSA
4544-9692 / 15-4190-6344
Ciudad de Buenos Aires, Argentina

[-- Attachment #2: Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-user] OT: disk full but lot of space

[gentoo]

Le Lundi, 8 Août 2005 13.57, vous avez ecrit :
> > bzip2: I/O or other error, bailing out.  Possible reason follows.
> > bzip2: No space left on device
> >         Input file = (stdin), output file = (stdout)
> > tar: Child returned status 1
> > tar: Error exit delayed from previous errors
>
> Check your temp partitions...

No temp partitions. There is only one partition: /
It's a rented server, so I can't do anything about it.

Beside the last command of my script is "date > DATE", to keep a timestamp in 
a file. Even this has failed: "date: write error: No space left on device"

--mat

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Christoph Gysin]

Norberto Bensa wrote:
> gentoo@depardo.ch wrote:
>>bzip2: I/O or other error, bailing out.  Possible reason follows.
>>bzip2: No space left on device
>>        Input file = (stdin), output file = (stdout)
>>tar: Child returned status 1
>>tar: Error exit delayed from previous errors
> 
> Check your temp partitions...

What makes you believe this has something to do with /tmp?

Just curious...

Christoph
-- 
echo mailto: NOSPAM !#$.'<*>'|sed 's. ..'|tr "<*> !#:2" org@fr33z3
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Christoph Gysin]

gentoo@depardo.ch wrote:
>>Did you try fsck?
> 
> nope :-(
> I'm a bit too afraid to try it. It's a production server with only one 
> partition and it's located about 6000 miles away. If something goes wrong, 
> I'll be really in a bad situation.
> 
> I think I'll have to do it anyway, but I'm checking for lighter solutions 
> first...

fsck -n ?

Christoph
-- 
echo mailto: NOSPAM !#$.'<*>'|sed 's. ..'|tr "<*> !#:2" org@fr33z3
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[gentoo]

Le Lundi, 8 Août 2005 14.10, Christoph Gysin a ecrit :
> >>Did you try fsck?
> >
> > nope :-(
> > I'm a bit too afraid to try it. It's a production server with only one
> > partition and it's located about 6000 miles away. If something goes
> > wrong, I'll be really in a bad situation.
> >
> > I think I'll have to do it anyway, but I'm checking for lighter solutions
> > first...
>
> fsck -n ?

with this, fsck won't change anything to my partition at all?
and what about this line from the output of tune2fs?
Filesystem state:         clean with errors
Is it like "critical" or like "informative"?

Since e2fsck _will_ find errors, I'll have to do an e2fsck without -n anyway. 
Or should I light a candle, buy some horseshoes and hope that my / will stay 
the longest possible "clean with errors", before going to HD paradise?

--mat

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Christoph Gysin]

gentoo@depardo.ch wrote:
> with this, fsck won't change anything to my partition at all?

nope.

> and what about this line from the output of tune2fs?
> Filesystem state:         clean with errors
> Is it like "critical" or like "informative"?

only fsck can tell...

> Since e2fsck _will_ find errors, I'll have to do an e2fsck without -n anyway. 
> Or should I light a candle, buy some horseshoes and hope that my / will stay 
> the longest possible "clean with errors", before going to HD paradise?

How useful is this system to you, when you can't even write to the disk?

I would take to system down, check the filesystems, repair them (if needed), 
recover lost data from backups and get up and running again.

Things are a little more complicated for you, since you don't seem to have full 
access to the machine. Talk to your vendor/sysadmin who is responsible for the 
(physical) machines. I'm not familiar with your particular situation...

Christoph
-- 
echo mailto: NOSPAM !#$.'<*>'|sed 's. ..'|tr "<*> !#:2" org@fr33z3
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[gentoo]

> How useful is this system to you, when you can't even write to the disk?

I can write to the system again... The system couldn't write from 3 AM to 8 
AM, after that, It was ok again. That's why I'm not really enthousiast to 
tune my fs. "If it's not broken, don't fix it.

> I would take to system down, check the filesystems, repair them (if
> needed), recover lost data from backups and get up and running again.

Yes, I would do that too, if I could :-(

> Things are a little more complicated for you, since you don't seem to have
> full access to the machine. Talk to your vendor/sysadmin who is responsible
> for the (physical) machines. I'm not familiar with your particular
> situation...

I've attached the output of e2fsck -n. Could you please tell me how bad it 
looks? Are there questions to which answering yes is dangerous?

e2fsck 1.26 (3-Feb-2002)
Warning!  /dev/hda2 is mounted.
Warning: skipping journal recovery because doing a read-only filesystem check.
/ contains a file system with errors, check forced.

Pass 1: Checking inodes, blocks, and sizes
Deleted inode 229438 has zero dtime.  Fix? no
Inodes that were part of a corrupted orphan linked list found.  Fix? no
Inode 229528 was part of the orphaned inode list.  IGNORED.
Inode 4145432, i_blocks is 16, should be 8.  Fix? no
Inode 4538621, i_blocks is 120, should be 88.  Fix? no
Inode 4538633, i_blocks is 104, should be 88.  Fix? no
Inode 4538779, i_blocks is 248, should be 224.  Fix? no
Inode 4997298, i_blocks is 64, should be 56.  Fix? no
Inode 5161059, i_blocks is 86304, should be 41288.  Fix? no
Inode 5161060, i_blocks is 2888, should be 1016.  Fix? no
Inode 5652717, i_blocks is 40, should be 0.  Fix? no
Inode 5701868, i_blocks is 24, should be 8.  Fix? no
Inode 6488261, i_blocks is 48, should be 40.  Fix? no
Inode 6488266 was part of the orphaned inode list.  IGNORED.
Inode 6488293, i_blocks is 24, should be 8.  Fix? no
Inode 6488299, i_blocks is 40, should be 8.  Fix? no
Inode 6488300, i_blocks is 24, should be 8.  Fix? no
Inode 6488253, i_blocks is 168, should be 152.  Fix? no
Inode 6488259, i_blocks is 5680, should be 5664.  Fix? no
Inode 6488242, i_blocks is 2032, should be 2016.  Fix? no
Inode 6504594, i_blocks is 56, should be 48.  Fix? no
Inode 6504639, i_blocks is 72, should be 56.  Fix? no
Inode 6750248, i_blocks is 64, should be 56.  Fix? no

Pass 2: Checking directory structure
Entry 'xfj78Dhpf14819' in /var/spool/mqueue (2277384) has deleted/unused inode 
2277431.  Clear? no
Entry 'dfj78Dhpf14819' in /var/spool/mqueue (2277384) has deleted/unused inode 
2277432.  Clear? no
Entry 'qfj78Dhpf14819' in /var/spool/mqueue (2277384) has deleted/unused inode 
2277447.  Clear? no

Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Unattached zero-length inode 3244154.  Clear? no
Unattached inode 3244154
Connect to /lost+found? no
Unattached zero-length inode 3244183.  Clear? no
Unattached inode 3244183
Connect to /lost+found? no
Unattached zero-length inode 3244186.  Clear? no
Unattached inode 3244186
Connect to /lost+found? no
Unattached zero-length inode 3244187.  Clear? no
Unattached inode 3244187
Connect to /lost+found? no
Unattached zero-length inode 3244188.  Clear? no
Unattached inode 3244188
Connect to /lost+found? no
Unattached zero-length inode 3244189.  Clear? no
Unattached inode 3244189
Connect to /lost+found? no
Unattached zero-length inode 3244204.  Clear? no
Unattached inode 3244204
Connect to /lost+found? no
Unattached zero-length inode 3244206.  Clear? no
Unattached inode 3244206
Connect to /lost+found? no
Unattached zero-length inode 3244208.  Clear? no
Unattached inode 3244208
Connect to /lost+found? no
Unattached zero-length inode 3244210.  Clear? no
Unattached inode 3244210
Connect to /lost+found? no
Unattached zero-length inode 3244211.  Clear? no
Unattached inode 3244211
Connect to /lost+found? no
Unattached zero-length inode 3244236.  Clear? no
Unattached inode 3244236
Connect to /lost+found? no
Unattached zero-length inode 3244243.  Clear? no
Unattached inode 3244243
Connect to /lost+found? no
Unattached zero-length inode 3244263.  Clear? no
Unattached inode 3244263
Connect to /lost+found? no
Unattached zero-length inode 3244264.  Clear? no
Unattached inode 3244264
Connect to /lost+found? no
Unattached zero-length inode 3244265.  Clear? no
Unattached inode 3244265
Connect to /lost+found? no
Unattached zero-length inode 3244266.  Clear? no
Unattached inode 3244266
Connect to /lost+found? no
Unattached zero-length inode 3244267.  Clear? no
Unattached inode 3244267
Connect to /lost+found? no
Unattached zero-length inode 3244268.  Clear? no
Unattached inode 3244268
Connect to /lost+found? no
Unattached zero-length inode 3244269.  Clear? no
Unattached inode 3244269
Connect to /lost+found? no
Unattached zero-length inode 3244270.  Clear? no
Unattached inode 3244270
Connect to /lost+found? no
Unattached zero-length inode 3244271.  Clear? no
Unattached inode 3244271
Connect to /lost+found? no
Unattached zero-length inode 3244272.  Clear? no
Unattached inode 3244272
Connect to /lost+found? no
Unattached zero-length inode 3244273.  Clear? no
Unattached inode 3244273
Connect to /lost+found? no
Unattached zero-length inode 3244274.  Clear? no
Unattached inode 3244274
Connect to /lost+found? no
Unattached zero-length inode 3244275.  Clear? no
Unattached inode 3244275
Connect to /lost+found? no
Unattached zero-length inode 3244276.  Clear? no
Unattached inode 3244276
Connect to /lost+found? no
Unattached zero-length inode 3244277.  Clear? no
Unattached inode 3244277
Connect to /lost+found? no
Unattached zero-length inode 3244278.  Clear? no
Unattached inode 3244278
Connect to /lost+found? no
Unattached zero-length inode 3244279.  Clear? no
Unattached inode 3244279
Connect to /lost+found? no
Unattached zero-length inode 3244280.  Clear? no
Unattached inode 3244280
Connect to /lost+found? no
Unattached zero-length inode 3244281.  Clear? no
Unattached inode 3244281
Connect to /lost+found? no
Unattached zero-length inode 3244288.  Clear? no
Unattached inode 3244288
Connect to /lost+found? no
Unattached zero-length inode 3244289.  Clear? no
Unattached inode 3244289
Connect to /lost+found? no
Unattached zero-length inode 3244292.  Clear? no
Unattached inode 3244292
Connect to /lost+found? no
Unattached zero-length inode 3244293.  Clear? no
Unattached inode 3244293
Connect to /lost+found? no
Unattached zero-length inode 3244294.  Clear? no
Unattached inode 3244294
Connect to /lost+found? no
Unattached zero-length inode 3244298.  Clear? no
Unattached inode 3244298
Connect to /lost+found? no
Unattached zero-length inode 3244299.  Clear? no
Unattached inode 3244299
Connect to /lost+found? no
Unattached zero-length inode 3244300.  Clear? no
Unattached inode 3244300
Connect to /lost+found? no
Unattached zero-length inode 3244302.  Clear? no
Unattached inode 3244302
Connect to /lost+found? no
Unattached zero-length inode 3244303.  Clear? no
Unattached inode 3244303
Connect to /lost+found? no
Unattached zero-length inode 3244304.  Clear? no
Unattached inode 3244304
Connect to /lost+found? no
Unattached zero-length inode 3244305.  Clear? no
Unattached inode 3244305
Connect to /lost+found? no
Unattached zero-length inode 3244306.  Clear? no
Unattached inode 3244306
Connect to /lost+found? no
Unattached zero-length inode 3244307.  Clear? no
Unattached inode 3244307
Connect to /lost+found? no
Unattached zero-length inode 3244308.  Clear? no
Unattached inode 3244308
Connect to /lost+found? no
Unattached zero-length inode 3244309.  Clear? no
Unattached inode 3244309
Connect to /lost+found? no
Unattached zero-length inode 3244310.  Clear? no
Unattached inode 3244310
Connect to /lost+found? no

Pass 5: Checking group summary information
Block bitmap differences:  -39633 -39634 -39635 -47071 -47072 -49478 -49479 
-49480 -49481 -49482 -49483 -49484 -49485 -49498 -49499 -49500 -49501 -49502 
-49503 -49504 -49505 -49506 -49507 -49508 -49509 -49510 -61764 -61765 -66382 
-66451 -66456 -68693 -81309 -82076 -82077 -82078 -82079 -82080 -82081 -82082 
-82083 -82084 -82085 -82086 -82087 -82088 -82089 -82090 -82091 -82092 -82093 
-82094 -82095 -82096 -82097 -82098 -82099 -82100 -82101 -82102 -82103 -82104 
-82105 -88924 -88925 -88926 -89060 -90516 -90696 -90733 -90738 -91120 -91363 
-91364 -91366 -91368 -91377 -91640 -91701 -140264 -168838 -168839 -270423 
-270424 -270425 -270935 -270936 -270937 -270938 -270939 -270940 -427571 
-427572 -427573 -427574 -427575 -427576 -427577 -427578 -427579 -427580 
-427581 -427582 -427583 -427600 -427608 -458756 -460119 -460120 -460121 
-460122 -460123 -460124 -460125 -4556768 -4556769 -4556770 -4556771 -4556772 
-4556773 -4556774 -4556775 -4556776 -4556788 -4556790 -4556791 -4556792 
-4556793 -4556796 +6489000 +6489073 +6489074 +6489076 +6489083 +6489084 
+6489085 +6489107 +6489108 +6489109 +6489110 +6489111 +6489112 +6489125 
+6489126 +6489127 +6489129 +6489130 +6489131 +6489133 +6489134 +6489135 
+6489137 +6489138 +6489162 +6489179 +6489180 +6489185 +6489186 +6489187 
+6489188 +6489189 +6489190 +6489191 +6489192 +6489200 +6489201 +6489214 
+6489215 +6489216 +6489231 +6489368 +6489369 +6489370 +6489371 +6489372 
+6489373 +6489374 -9078064 -9078065 -9078066 -9078067 -9078068 -9078069 
-9078070 -9078071 -9078072 -9078073 -9078074 -9078075 -9078076 -9078077 
-9078240 -9078327 -9078328 -9078329 -9078330 -9078331 -9078332 -9078333 
-9078334 -9078335 -9078336 -9078337 -9078338 -9078339 -9078340 -9078341 
-9078342 -9078343 -9078344 -9078345 -9078346 -9078347 -9078348 -9078349 
-9078350 -9078351 -9078360 -9078361 -9078362 -9078363 -9078364 -9078365 
-9078366 -9078367 -9078368 -9078369 -9078370 -9078371 -9078372 -9078373 
-9078374 -9078375 -9078376 -9078377 -9078387 -9078388 -9078389 -9078390 
-9078391 -9078392 -9078393 -9078394 -9078395 -9078396 -9078397 -9078398 
-9078399 -9078400 -9078401 -9078402 -9078403 -11308393 -11308394 -11308395 
-11308396 -11308397 -11308398 -11308399 -11308400 -11308401 -11308402 
-11308403 -11308404 -11308405 -11308406 -11308407 -11308409 -11308410 
-11308411 -11308412 -11308413 -11308414 -11308415 -11308416 -11308417 
-11308418 -11308419 -11308420 -11308421 -11308422 -11308423 -12978808
Fix? no
Free blocks count wrong for group #1 (28, counted=0).
Fix? no
Free blocks count wrong for group #2 (62, counted=11).
Fix? no
Free blocks count wrong for group #4 (10, counted=9).
Fix? no
Free blocks count wrong for group #5 (841, counted=836).
Fix? no
Free blocks count wrong for group #8 (6360, counted=6344).
Fix? no
Free blocks count wrong for group #13 (30819, counted=30804).
Fix? no
Free blocks count wrong for group #139 (30511, counted=30496).
Fix? no
Free blocks count wrong for group #198 (138, counted=184).
Fix? no
Free blocks count wrong for group #277 (25407, counted=25332).
Fix? no
Free blocks count wrong for group #345 (27614, counted=27583).
Fix? no
Free blocks count wrong for group #438 (19246, counted=13222).
Fix? no
Free blocks count wrong (11147972, counted=11096727).
Fix? no

Inode bitmap differences:  -213129 -213131 -229438 -229528 -2277431 -2277432 
-2277435 +3244154 +3244183 +3244186 +3244187 +3244188 +3244189 +3244204 
+3244206 +3244208 +3244210 +3244211 +3244236 +3244243 +3244263 +3244264 
+3244265 +3244266 +3244267 +3244268 +3244269 +3244270 +3244271 +3244272 
+3244273 +3244274 +3244275 +3244276 +3244277 +3244278 +3244279 +3244280 
+3244281 +3244288 +3244289 +3244292 +3244293 +3244294 +3244298 +3244299 
+3244300 +3244302 +3244303 +3244304 +3244305 +3244306 +3244307 +3244308 
+3244309 +3244310 -6488266
Fix? no
Free inodes count wrong for group #13 (16094, counted=16092).
Fix? no
Free inodes count wrong for group #139 (16312, counted=16309).
Fix? no
Free inodes count wrong for group #198 (16066, counted=16113).
Fix? no
Free inodes count wrong (7061883, counted=7061925).
Fix? no

/: ********** WARNING: Filesystem still has errors **********

/: 130693/7192576 files (3.7% non-contiguous), 3218154/14366126 blocks


--mat
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Christopher Fisk]

On Mon, 8 Aug 2005, Christoph Gysin wrote:

>>  Check your temp partitions...
>
> What makes you believe this has something to do with /tmp?
>
> Just curious...

He didn't read the whole message, saw disk full and said oh, his temp 
partition is overflowing.

The correct answer here is that you have to do an fsck.  The best solution 
is to just be at the console of the machine, where you can boot into 
single user mode and run the fsck.

If that is not an option you'll need to make sure everyone is off the 
machine (Maintenance window?), stop all the services except the network 
services and sshd then remount the partition readd only and run the fsck.

quick rundown of commands:

rc-update show
/etc/init.d/service stop #for each non-repair essential service
ps auwx #to verify that all the services are stopped that need to be
mount -o remount,ro /
fsck.ext3 /dev/hdxy
mount -o remount,rw /
/etc/init.d/services start #for each service you stopped

At the end of this I really prefer doing a reboot (And fsck may tell you a 
reboot is required).

Might want to test the procedure a few times on a non-critical machine, 
but that should get you through the process.


Christopher Fisk
-- 
Adelai: A package is just a box until it's delivered.
 	cBlog: http://chris.uasoft.com/
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Christopher Fisk]

On Mon, 8 Aug 2005, gentoo@depardo.ch wrote:

>> How useful is this system to you, when you can't even write to the disk?
>
> I can write to the system again... The system couldn't write from 3 AM to 8
> AM, after that, It was ok again. That's why I'm not really enthousiast to
> tune my fs. "If it's not broken, don't fix it.

But it is broken...


>> I would take to system down, check the filesystems, repair them (if
>> needed), recover lost data from backups and get up and running again.
>
> Yes, I would do that too, if I could :-(

Read my other post about how to do this remotely.


>> Things are a little more complicated for you, since you don't seem to have
>> full access to the machine. Talk to your vendor/sysadmin who is responsible
>> for the (physical) machines. I'm not familiar with your particular
>> situation...
>
> I've attached the output of e2fsck -n. Could you please tell me how bad it
> looks? Are there questions to which answering yes is dangerous?

Disclaimer: You should have backups of the system!
I've found that fsck is actually pretty damn good at being safe.  I've 
never had fsck fail to repair a non-hardware filesystem corruption.  The 
most likely thing you'll need to do after a repair here is to reboot the 
machine, which is the sweating time.

If you can possibly get someone onsite from that office to watch the 
reboot you should be fine.


Christopher Fisk
-- 
"Listen, Bender, where's your bathroom?" -Fry 
"Bath what?" -Bender "Bathroom." -Fry 
"What room?" -Bender "Bathroom!" -Fry 
"What what?" Bender "Ah, nevermind." -Fry
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Hans-Werner Hilse]

Hi,

On Mon, 8 Aug 2005 15:53:54 +0200
gentoo@depardo.ch wrote:

> I've attached the output of e2fsck -n. Could you please tell me how bad it 
> looks? Are there questions to which answering yes is dangerous?
> 
> e2fsck 1.26 (3-Feb-2002)
> Warning!  /dev/hda2 is mounted.

That is done in a mounted situation. I guess, it is even read/write-
mounted? 

Well, I'll cite the man page:
---snip
Note that in general it is not safe to run e2fsck on mounted
filesystems.  The only exception is if the -n option is specified, and -
c, -l, or -L options are not specified. However, even if it is safe
to do so, the results printed by e2fsck are not valid if the filesystem
is mounted.
---snip

This is very harsh, I'd say, remounting it ro (read-only), sync'ing it
and then doing "e2fsck -n" will give a lot more valid information.

In mounted state, esp. read-write, it is pretty normal on a busy system
to have these inconsistencies the e2fsck dump showed...

-hwh
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Jonathan Nichols]

> I can write to the system again... The system couldn't write from 3 AM to 8 
> AM, after that, It was ok again. That's why I'm not really enthousiast to 
> tune my fs. "If it's not broken, don't fix it.
> 

Yikes.

Y'know, what city is this machine in? One of us might be nearby and 
willing to talk the data center folks through the issue. *shrug* Just an 
idea. :)

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Raymond Lillard]

Christopher Fisk wrote:
> On Mon, 8 Aug 2005, gentoo@depardo.ch wrote:
> 
>>> How useful is this system to you, when you can't even write to the disk?
>>
>>
>> I can write to the system again... The system couldn't write from 3 AM 
>> to 8
>> AM, after that, It was ok again. That's why I'm not really enthousiast to
>> tune my fs. "If it's not broken, don't fix it.
> 
> 
> But it is broken...

Mat,

It sure sounds broken to me.  Have you looked at your
log files for hardware errors?  Maybe you've lost a
superblock due to a hard disk error?

Ray
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Norberto Bensa]

[-- Attachment #1: Type: text/plain, Size: 276 bytes --]

Christoph Gysin wrote:
> What makes you believe this has something to do with /tmp?

he's using a pipe (stdin | stdout) 

and nope... I didn't read the whole message.

-- 
Norberto Bensa
informática BeNSA
4544-9692 / 15-4190-6344
Ciudad de Buenos Aires, Argentina

[-- Attachment #2: Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-user] OT: disk full but lot of space

[Christoph Gysin]

Norberto Bensa wrote:
> Christoph Gysin wrote:
> 
>>What makes you believe this has something to do with /tmp?
> 
> he's using a pipe (stdin | stdout) 

What have pipes to do with /tmp? A pipe is nothing more than a buffer (in 
memory) between two processes. If there's not enough memory, the buffer could be 
swapped out to the swap partition (if any). Nothing gets written to /tmp.

> and nope... I didn't read the whole message.

Please read the posts before answering. If the post is really long, it's okay to 
skip some parts if you mention that in your answer.

Christoph
-- 
echo mailto: NOSPAM !#$.'<*>'|sed 's. ..'|tr "<*> !#:2" org@fr33z3
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[mat]

> Y'know, what city is this machine in? One of us might be nearby and 
> willing to talk the data center folks through the issue. *shrug* Just an 
> idea.

Such a nice proposition. Thanks a lot. The machine is somewhere in texas... 
I don't know where exactly.  Anyway, the housing company is offering KVM and 
remote console access to the box, so I won't need anyone to go ring at the 
housing company to check on a server rented by some remote swiss guy. But 
thanks anyway :-)

> >>> How useful is this system to you, when you can't even write to the
> >>> disk?
> >> I can write to the system again... The system couldn't write from 3 AM
> >> to 8  AM, after that, It was ok again. That's why I'm not really
> >> enthousiast  to tune my fs. "If it's not broken, don't fix it.
> >
> > But it is broken...
> 
> It sure sounds broken to me.  Have you looked at your
> log files for hardware errors?  Maybe you've lost a
> superblock due to a hard disk error?

I couldn't see any hardware errors in the logs and had to wait until now to 
e2fsck the partition, but everything went fine, only:

18 times "Inode 4145432, i_blocks is 16, should be 8.  Fix<y>? yes"

and

Free blocks count wrong for group #438 (19246, counted=13222).
Fix<y>? yes
Free blocks count wrong (10346330, counted=10295276).
Fix<y>? yes

Nothing I should worry to much about... or should I?

And thanks to every other people helping me on this matter.

--mat
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] OT: disk full but lot of space

[Norberto Bensa]

[-- Attachment #1: Type: text/plain, Size: 427 bytes --]

Christoph Gysin wrote:
> Norberto Bensa wrote:
> > Christoph Gysin wrote:
> >>What makes you believe this has something to do with /tmp?
> >
> > he's using a pipe (stdin | stdout)
>
> What have pipes to do with /tmp? 

I don't know. It was just the case sometime ago on one of my boxes. Perhaps, 
as you say, I was short of swap.


-- 
Norberto Bensa
4544-9692 / 15-4190-6344
Ciudad de Buenos Aires, Argentina

[-- Attachment #2: Type: application/pgp-signature, Size: 190 bytes --]