So I ran into a problem which involved an out of space error. However, upon issuing the df -h command I found that there was plenty of disk space available on the partition (in this case /var).
[[email protected] /]# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda3 5.9G 1.9G 3.7G 34% /var
Peculiar enough I thought I'd try to 'touch' a file to the /var directory and wound up with the following error:
[[email protected] var]# touch /var/test touch: cannot touch `/var/test': No space left on device
Naturally this seemed pretty odd at first until I issued the next command to check inodes df -i
[[email protected] var]# df -i Filesystem Inodes IUsed IFree IUse% Mounted on /dev/sda3 385024 385024 0 100% /var
Ah HAH! Well there we have it, all the available inodes for this partition have been used. That's certainly going to cause some problems. It's amazing that the server itself was still running. This means that there are probably a whole slew of empty files taking up inodes that don't need to be there. To find these issue the following command:
for i in *; do echo $i; find $i |wc -l; done
You may have to cd into sub-directories and run the above command over again until you find the location of the directory that contains the files you're looking for. Once you find them. You'll need to evaluate them to make sure you can remove them. Once you remove some of these garbage files you should have plenty of inodes available and be back up and running.