Linux file system recovery - undelete
Posted by davidnewcomb on 12 Aug 2009 in Techie
If you are a systems UNIX administrator, sooner or later you are going to have a catastrophic disaster which will for ever alter your way of working. UNIX is unforgiving when it comes to deleting files.
The UNIX file system works faster and more efficiently that a windows file system. Notably, the disk never needs de-fragmenting and the disk caching is better. The bad point about it being better is that if you delete anything it is almost impossible to get it back!
There are several common mistakes that you have to watch out for: accidentally deleting something important, accidentally deleting system tools or accidentally writing into a device file which doesn’t point to what you thought it did.
Accidentally deleting something important
Is the file still in use by any other program? If yes then read on, if no then it’s either back to those backup tapes or you could have a go at Carlo Wood’s HOWTO recover deleted files on an ext3 file system which is not for the faint hearted!
One thing worth noting is that when a file is open the space can not be reclaimed by the system. Unix allows many pointers to a file. They may or may not exist in the file system. Under windows you can’t delete a file that another user is writing to, but under Unix you can. …and believe it or not it’s really handy. No more of those famous “can’t delete folder - file in use” messages from Windows.
In fact the best way to create a temporary file that only you can write to, is to open a new file then delete it from the file system. There is no longer a global reference (file system entry) to your file so no one can read it. When you close the file the OS reclaims the space. If your process dies then the file is closed for you by the OS and is therefore automatically tidied away. Cool!
The first method of getting a deleted file back is the easiest and will work in most cases.
- Find the process id of the program that has your file open. You can use one of the following help programs to help find a process that has your deleted file open (locked):
lsof
orls -l /proc/*/fd | grep myfilename
- Change directory into the /proc filesystem for that process.
cd /proc/12345/fd
- In the directory lists there will be a symbolic link that points to your deleted file. eg
total 4 lrwx—— 1 root root 64 Aug 11 17:33 0 -> /dev/pts/0 lrwx—— 1 root root 64 Aug 11 17:33 1 -> /dev/pts/0 lrwx—— 1 root root 64 Aug 11 17:33 2 -> /dev/pts/0 lr-x—— 1 root root 64 Aug 11 17:33 3 -> /tmp/hello.txt (deleted)
- Simply copy the file somewhere else using the copy command:
cp 3 /tmp/hello.txt.undeleted
- Run the program
lsof
and search the output for the line containing the name of the file you have just deleted.# lsof | grep -e hello.txt -e COMMAND COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME cat 1002 root 3r REG 3,3 1500015 3866793 /home/hello.txt (deleted)
- The number next to the file name is the inode number and represents the location of the first part of the file in the file system structure (3866793).
- Now that we have the inode of the file we want to restore, we can re-link it to the file system using the e2fsprogs suite.
- Firstly you need the device name of the file system your deleted file is on. Change to the directory you want to restore the file to and use the
df
command with a dot for the current directory. Note you can only restore files on the same file system. Once it has been restored, you can copy it anywhere you like.# df . Filesystem 1k-blocks Used Available Use% Mounted on /dev/hda3 38535468 9966368 26611580 28% /
- Next load in the file system debugger in write mode:
# debugfs -w /dev/hda3 debugfs 1.26 (3-Feb-2002) debugfs:
- Change to the folder you want to restore the file to:
debugfs: pwd [pwd] INODE: 2 PATH: / [root] INODE: 2 PATH: / debugfs: cd /home debugfs: pwd [pwd] INODE: 3866727 PATH: /home [root] INODE: 2 PATH: / debugfs: ln <3866793> hello.txt
- Now link the first inode of the delete file back to the file system.
debugfs: ls -l 3866727 40755 0 0 4096 12-Aug-2009 10:13 . 2 40755 0 0 4096 1-Apr-2009 00:04 .. debugfs: ln <3866793> hello.txt debugfs: ls -l 3866727 40755 0 0 4096 12-Aug-2009 10:13 . 2 40755 0 0 4096 1-Apr-2009 00:04 .. 3866793 100644 0 0 1500015 12-Aug-2009 10:13 hello.txt
The ‘<’ and ‘>’ tellsln
that the source file is actually an inode and not a regular file. - Changes are made to the file system straight away, so all you have to do is exit:
debugfs: q #
ls -Rail / > /ls-Rail.txtThis gives a recursive listing of all files (including hidden files) with their inodes from the root. If you delete a lot of files then you may want to set the file system to read-only mode, so that the system can’t overwrite any of the deleted blocks. This will give you a bit of time to repair the file system.
mount -n -o remount -o ro /dev/hda1 /homewhere
- -n : don’t write
/etc/fstab
. - -o remount : lets you change options without unmounting the file system. Often you can’t unmount anyway it because it is in use.
- -o ro : sets the file system to read-only mode.
- /dev/hda1 : device file to be mounted.
- /home : folder to place the file system.
mount -n -o remount /dev/hda1 /homeCatastrophic disaster I once tried to create a tar file on a USB stick, but accidentally used the root file system device file instead of the USB device file. This overwrote all the superblocks in the file allocation table. Now the only copy was in memory! The filesystem degraded over time as the superblocks were read from disk during the normal running of the system. Most of
/bin
was gone, so the plan became “save as many as you can". My other partitions were unaffected because they had separate superblock structures. The file lister ls
was gone along with half the other file system tools, so not only was I buggered, I was also blind!
Krishna Kunchith said that the echo
command is a shell command as well as an actual command program, so when my echo command program died, the shell echo
gave me eyes.
echo *prints all the files which match the while card, you can also use the wild card system to do file list filtering:
echo h*
cd
is also a shell command. So now I had eyes and legs, and for the most part that’s all you need!
Oh dear!
If your here, then you really ought to have a long think about your disaster and recovery backup procedure! Good luck!No feedback yet
Form is loading...