Linux open files deleted

recovering deleted file held open by apache?

This is what I am doing to recover the file and put it back where it was. Is there any simpler way of doing this because above code doesn’t look good. Furthermore how do I know from where the file was deleted(directorytobecopied) so that I don’t have to manually ask somebody where the file was located originialy and put it back there.

1 Answer 1

If a file has been deleted but is still open, that means the file still exists in the filesystem (it has an inode) but has a hard link count of 0. Since there is no link to the file, you cannot open it by name. There is no facility to open a file by inode either.

There is no way to discover the file through its filesystem, and especially no way to look for the file in the directory where it last was. The directory entry is gone. All that remains is the file itself. You can get to the file with a filesystem debugger, but that requires root permissions and is hard to use and error-prone.

Linux exposes open files through special symbolic links under /proc . These links are called /proc/12345/fd/42 where 12345 is the PID of a process and 42 is the number of a file descriptor in that process. A program running as the same user as that process can access the file (the read/write/execute permissions are the same you had as when the file was deleted).

The name under which the file was opened is still visible in the target of the symbolic link: if the file was /var/log/apache/foo.log , then the target of the link is /var/log/apache/foo.log (deleted) . (If the file was renamed after it was opened, the target of the symlink may reflect the renaming.)

Thus you can recover the content of an open deleted file given the PID of a process that has it open and the descriptor that it’s opened on like this:

recover_open_deleted_file () < old_name=$(readlink "$1") case "$old_name" in *' (deleted)') old_name=$if [ -e "$old_name" ]; then new_name=$(TMPDIR=$ mktemp) echo "$oldname has been replaced, recovering content to $new_name" else new_name="$old_name" fi cat "$new_name";; *) echo "File is not deleted, doing nothing";; esac > recover_open_deleted_file "/proc/$pid/fd/$fd" 

If you only know the process ID but not the descriptor, you can recover all files with

for x in /proc/$pid/fd/*; do recover_open_deleted_file "$x" done 

If you don’t know the process ID either, you can search among all processes:

for x in /proc/2*/fd/*; do case $(readlink "$x") in /var/log/apache/*) recover_open_deleted_file "$x";; esac done 

You can also obtain this list by parsing the output of lsof , but it isn’t simpler nor more reliable nor more portable (this is Linux-specific anyhow).

Источник

Восстановление открытых файлов но удаленных c файловой системы linux

Всех с прошедшим новым годом!
В этой заметке я бы хотел поделиться как можно восстановить открытый файл в linux.

Предыстория

Зашел человек на канал посвященный debian в jabber и сказал что взломали его jabber-bot и выполнили команду:

так как это было выполнено не под рутом, особых проблем быть не должно, но конфигурационные файлы бота удалены. Бот остался запущен и задача была восстановить открытые им файлы и попробовать максимально быстро поднять всё с теми же настройками.

Восстанавливаем файл

Первым делом нам нужно убедиться что у нас стоит приложение lsof и примонтирован procfs в /proc.
В этой заметке я буду считать что в системе где будут восстанавливаться открытые файлы все нужные приложения стоят, root доступ есть, всё примонтировано как нужно.

Первым делом нам нужно найти открытый файл с помощью программы lsof:

$ sudo lsof | grep /home/anton/.xsession-errors kwin 2031 4002 anton 2w REG 253,3 4486557 1835028 /home/anton/.xsession-errors

kwin 2031 4002 anton 2w REG 253,3 4486557 1835028 /home/anton/.xsession-errors

$ sudo cp /proc/2031/fd/2 /home/anton/.xsession-error

На этом всё, так можно восстановить открытый файл, но который по какой-то причине был удален.

UPD1: Меня спросили как найти и восстановить все открытые файлы конкретным приложением.
Предположим мы знаем 1 файл который нужно восстановить, мы его нашли с помощью

$ sudo lsof | grep /home/anton/.xsession-errors kwin 2031 anton 2w REG 253,3 4486557 1835028 /home/anton/.xsession-errors

Мы знаем что 2031 — это pid процесса который держит ваш файл. Нам нужно найти все файлы которые держать открытыми данный процесс:

$ sudo lsof -p 2031 | grep deleted

После чего просто восстанавливаем все файлы как описано выше.

UPD2: Почему я использую grep для поиска файлов вместо параметра, который работает быстрей?
Я использую grep так как там видно удален файл или нет, я считаю это более удобным (ИМХО)

UPD3: Также можно посмотреть все открытые файлы процесса через комманду ls, пометки deleted будут, пример:

Источник

Find and remove large files that are open but have been deleted

How does one find large files that have been deleted but are still open in an application? How can one remove such a file, even though a process has it open? The situation is that we are running a process that is filling up a log file at a terrific rate. I know the reason, and I can fix it. Until then, I would like to rm or empty the log file without shutting down the process. Simply doing rm output.log removes only references to the file, but it continues to occupy space on disk until the process is terminated. Worse: after rm ing I now have no way to find where the file is or how big it is! Is there any way to find the file, and possibly empty it, even though it is still open in another process? I specifically refer to Linux-based operating systems such as Debian or RHEL.

If you know the pid then you can use lsof -p to list its open files and their sizes. The deleted file will have a (deleted) next to it. The deleted file will be linked at /proc//fd/1 probably. I don’t know how to make a process stop writing to its file descriptor without terminating it. I would think that would depend on the process.

@donothingsuccessfully The «deleted» tag reported by lsof is Solaris specific, in fact Solaris 10 or later only. The OP did not specify what operating system he is using. @dotancohen On Solaris you can pipe the output of lsof to search for deleted, eg lsof | grep «(deleted)» . When there are no more processes holding a deleted file open, the kernel will free up the inode and disk blocks. Processes do not have «handlers» by which they can be notified that an open, essentially locked file, have been removed from disk.

@Johan, the lsof | grep ‘(deleted)’ works on Linux as well. On Linux, you can be notified of file deletion (even files that already don’t have any entry in any directory other than /proc/some-pid/fd anymore) with the inotify mechanism (IN_DELETE_SELF event)

I created somefile and opened it in VIM, then rm ed it in another bash process. I then run lsof | grep somefile and it is not in there, even though the file is open in VIM.

4 Answers 4

If you can’t kill your application, you can truncate instead of deleting the log file to reclaim the space. If the file was not open in append mode (with O_APPEND ), then the file will appear as big as before the next time the application writes to it (though with the leading part sparse and looking as if it contained NUL bytes), but the space will have been reclaimed (that does not apply to HFS+ file systems on Apple OS/X that don’t support sparse files though).

If it was already deleted, on Linux, you can still truncate it by doing:

Where $pid is the process id of the process that has the file opened, and $fd one file descriptor it has it opened under (which you can check with lsof -p «$pid» .

If you don’t know the pid, and are looking for deleted files, you can do:

lsof -nP +L1 , as mentioned by @user75021 is an even better (more reliable and more portable) option (list files that have fewer than 1 link).

find /proc/*/fd -ls | grep '(deleted)' 

Or to find the large ones with zsh :

An alternative, if the application is dynamically linked is to attach a debugger to it and make it call close(fd) followed by a new open(«the-file», . ) .

@OlivierDulac, lsof is probably going to be the closest to a portable solution you can get to list open files. the debugger approach to close the fd under the application feet should be quite portable as well.

@StephaneChazelas: thanks. I found a way to list all PIDs which have a file open on each partitions : df -k | awk ‘NR>1 < print $NF >‘ | xargs fuser -Vud (and then easy to send signals to the offenders to force them to release the fd)

You can also use lsof +L1 . From the lsof man page: «A specification of the form +L1 will select open files that have been unlinked. A specification of the form +aL1 will select unlinked open files on the specified file system.». That should be a bit more reliable than grepping.

Check out the quickstart here: lsof Quickstart

I’m surprised no one mentioned the lsof quickstart file (included with lsof). Section «3.a» shows how to find open, unlinked files:

[root@enterprise ~]# lsof -a +L1 /tmp COMMAND PID USER FD TYPE DEVICE SIZE NLINK NODE NAME httpd 2357 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) mysqld 2588 mysql 4u REG 253,17 52 0 1495 /tmp/ibY0cXCd (deleted) mysqld 2588 mysql 5u REG 253,17 1048 0 1496 /tmp/ibOrELhG (deleted) mysqld 2588 mysql 6u REG 253,17 0 0 1497 /tmp/ibmDFAW8 (deleted) mysqld 2588 mysql 7u REG 253,17 0 0 11387 /tmp/ib2CSACB (deleted) mysqld 2588 mysql 11u REG 253,17 0 0 11388 /tmp/ibQpoZ94 (deleted) httpd 3457 root 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 8437 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 8438 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 8439 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 8440 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 8441 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 8442 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 8443 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 8444 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 16990 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 19595 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 27495 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 28142 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) httpd 31478 apache 29u REG 253,17 3926560 0 1499 /tmp/.NSPR-AFM-3457-9820130.0 (deleted) 

On Red Hat systems to find the local copy of the quick-start file, I usually do this:

[root@enterprise ~]# locate -i quickstart |grep lsof /usr/share/doc/lsof-4.78/00QUICKSTART 

[root@enterprise ~]# rpm -qd lsof /usr/share/doc/lsof-4.78/00.README.FIRST /usr/share/doc/lsof-4.78/00CREDITS /usr/share/doc/lsof-4.78/00DCACHE /usr/share/doc/lsof-4.78/00DIALECTS /usr/share/doc/lsof-4.78/00DIST /usr/share/doc/lsof-4.78/00FAQ /usr/share/doc/lsof-4.78/00LSOF-L /usr/share/doc/lsof-4.78/00MANIFEST /usr/share/doc/lsof-4.78/00PORTING /usr/share/doc/lsof-4.78/00QUICKSTART /usr/share/doc/lsof-4.78/00README /usr/share/doc/lsof-4.78/00TEST /usr/share/doc/lsof-4.78/00XCONFIG /usr/share/man/man8/lsof.8.gz 

Источник

What happens to an open file handle on Linux if the pointed file gets moved or deleted

Why I’m asking such questions: I’m using hot-plugged hardware (such as USB devices etc.). It can happen, that the device (and also its /dev/file) gets reattached by the user or another Gremlin.

What’s the best practice dealing with this?

7 Answers 7

If the file is moved (in the same filesystem) or renamed, then the file handle remains open and can still be used to read and write the file.

If the file is deleted, the file handle remains open and can still be used (This is not what some people expect). The file will not really be deleted until the last handle is closed.

If the file is replaced by a new file, it depends exactly how. If the file’s contents are overwritten, the file handle will still be valid and access the new content. If the existing file is unlinked and a new one created with the same name or, if a new file is moved onto the existing file using rename() , it’s the same as deletion (see above) — that is, the file handle will continue to refer to the original version of the file.

In general, once the file is open, the file is open, and nobody changing the directory structure can change that — they can move, rename the file, or put something else in its place, it simply remains open.

In Unix there is no delete, only unlink() , which makes sense as it doesn’t necessarily delete the file — just removes the link from the directory.

If on the other hand the underlying device disappears (e.g. USB unplug) then the file handle won’t be valid any more and is likely to give IO/error on any operation. You still have to close it though. This is going to be true even if the device is plugged back in, as it’s not sensible to keep a file open in this case.

Источник