How to clean up disk space if after deleting a file disk space is not released - interactive bash script

1) You can view the list of all deleted files with open file descriptors using the command

lsof | grep deleted

In the output, we get strings type of

nginx      4403  4430   nginx   29w      REG                8,1   5779197     165960 /tmp/access.log (deleted)

The eighth column in the eighth column we can see the file size in bytes, but we are interested in the location of the file descriptor, so that the first part of the path to it is the second output column, in this case 4403

2) Find the full path to the handle

Run the command below by substituting 4403 after /proc/ and the file name in the grep pattern

ls -l /proc/4403/fd | grep "/tmp/access.log"

The output from the command should look something like this:

l-wx------ 1 nginx nginx 64 Oct 30 15:21 29 -> /tmp/access.log (deleted)

Here we see the second part of the path, this is the 9th column, number 29

As a result, the full path to the file descriptor of the remote file looks like this:

/proc/4403/fd/29

Also, the path to the file for cleaning can be obtained using the following command (several awk iterations are used for additional validation)

find /proc/*/fd -ls 2> /dev/null | grep '(deleted)' | sed 's#\ (deleted)##g' | awk '{print $11" "$13}' | sort -u -k 2 | grep "/tmp/access.log" | awk '{print $1}'

3) Truncate the file

Run the command and to clear the disk space that still takes up the deleted file

truncate -s 0 /proc/4403/fd/29

Disk space released

In order not to perform these actions manually, you can use the interactive script as a function that will do everything for you

clean-deleted-space()
{
DELETED_FILES=$(cat << EOL $(lsof 2>/dev/null | grep -s deleted | grep -Po "\w+\s+\w+\s+/.*" | awk '{print $1" "$3}' | sort -u | grep -v '^0')
EOL
)
cat << EOL | awk '{sum+=$1} END {print "Will released: "sum/1024/1024" Mb"}'
${DELETED_FILES}
EOL
ONLY_FILES=$(cat << EOL | awk '{print $2}' | grep '^/'
${DELETED_FILES}
EOL
)
cat << EOL
${ONLY_FILES}
EOL
echo "Are you sure to release space of deleted files above?"
echo
select CLEAN in Yes No
do
if [ "$CLEAN" = "Yes" ]
then
for DFILE in ${ONLY_FILES}
do
echo
echo File: $DFILE
FILE_DESC_ADDR=$(find /proc/*/fd -ls 2> /dev/null | grep '(deleted)' | sed 's#\ (deleted)##g' | awk '{print $11" "$13}' | sort -u -k 2 | grep $DFILE | awk '{print $1}')
if [ "$FILE_DESC_ADDR" != "" ] ; then 
echo "executing: truncate -s 0 ${FILE_DESC_ADDR}"
truncate -s 0 ${FILE_DESC_ADDR}
echo done
else
echo "Truncate deleted file $DFILE is impossible, skip"
fi
echo
done
else
echo Nothing to do
fi
break
done
}
clean-deleted-space

Example script usage:

root@cloudmaker ~ $  clean-deleted-space
Will released: 7.37246 Mb
/var/log/zabbix/zabbix_agentd.log-20180916
/var/log/salt/minion-20180923
Are you sure to release space of deleted files above?
1) Yes
2) No
? 1
File: /var/log/zabbix/zabbix_agentd.log-20180916
executing: truncate -s 0 /proc/4186/fd/1
done
File: /var/log/salt/minion-20180923
executing: truncate -s 0 /proc/4213/fd/6
done