Linux truncate opened file

How to remove X bytes from the end of a large file without reading the whole file?

In Linux, I have a rather large file with some extraneous information tacked on to the end of it. Let’s say for example I know there are 314 bytes of extraneous data at the end of a 1.6GB file. Of course it is very easy and efficient to add more data to the end of a file, but what can I do to remove it without having to copy the first portion of that file into another (or overwrite said file)? Edit I’m seeing some good advice on doing this in C. I was hoping to script it from the commandline, but failing that I would be more inclined to doing it in python than C. I see that python has a truncate method on its file object but it seems to be demolishing my file no matter how i use it—I should be able to figure this one out, but of course answers are more than welcome still.

4 Answers 4

use the function truncate

int truncate(const char *path, off_t length); int ftruncate(int fd, off_t length); 

truncate takes the file name
ftruncate takes an open file descriptor

both of these set the file length to length so it either truncates or elongates (in the latter case, the rest of the file will be filled with NULL/ZERO)

[edit]
truncate (linux shell command) will work also

**SYNTAX** truncate -s integer **OPTIONS** -s number specify the new file length. If the new length is smaller than the current filelength data is lost. If the new length is greater the file is padded with 0. You can specify a magnitude character to ease large numbers: b or B size is bytes. k size is 1000 bytes. K size is 1024 bytes. m size is 10^6 bytes. M size is 1024^2 bytes. g size is 10^9 bytes. G size is 1024^3 bytes. **EXAMPLES** To shrink a file to 10 bytes: truncate -s 10 /tmp/foo To enlarge or shrink a file to 345 Megabytes: truncate -s 345M /tmp/foo 

Источник

Linux truncate opened file

NAME

truncate, ftruncate - truncate a file to a specified length

SYNOPSIS

#include unistd.h> #include  int truncate(const char *path, off_t length); int ftruncate(int fd, off_t length); Feature Test Macro Requirements for glibc (see feature_test_macros(7)): truncate(): _BSD_SOURCE || _XOPEN_SOURCE >= 500 || _XOPEN_SOURCE && _XOPEN_SOURCE_EXTENDED || /* Since glibc 2.12: */ _POSIX_C_SOURCE >= 200809L ftruncate(): _BSD_SOURCE || _XOPEN_SOURCE >= 500 || _XOPEN_SOURCE && _XOPEN_SOURCE_EXTENDED || /* Since glibc 2.3.5: */ _POSIX_C_SOURCE >= 200112L

DESCRIPTION

The truncate() and ftruncate() functions cause the regular file named by path or referenced by fd to be truncated to a size of precisely length bytes. If the file previously was larger than this size, the extra data is lost. If the file previously was shorter, it is extended, and the extended part reads as null bytes ('\0'). The file offset is not changed. If the size changed, then the st_ctime and st_mtime fields (respectively, time of last status change and time of last modification; see stat(2)) for the file are updated, and the set-user-ID and set-group-ID mode bits may be cleared. With ftruncate(), the file must be open for writing; with truncate(), the file must be writable.

RETURN VALUE

On success, zero is returned. On error, -1 is returned, and errno is set appropriately.

ERRORS

For truncate(): EACCES Search permission is denied for a component of the path prefix, or the named file is not writable by the user. (See also path_resolution(7).) EFAULT The argument path points outside the process's allocated address space. EFBIG The argument length is larger than the maximum file size. (XSI) EINTR While blocked waiting to complete, the call was interrupted by a signal handler; see fcntl(2) and signal(7). EINVAL The argument length is negative or larger than the maximum file size. EIO An I/O error occurred updating the inode. EISDIR The named file is a directory. ELOOP Too many symbolic links were encountered in translating the pathname. ENAMETOOLONG A component of a pathname exceeded 255 characters, or an entire pathname exceeded 1023 characters. ENOENT The named file does not exist. ENOTDIR A component of the path prefix is not a directory. EPERM The underlying filesystem does not support extending a file beyond its current size. EPERM The operation was prevented by a file seal; see fcntl(2). EROFS The named file resides on a read-only filesystem. ETXTBSY The file is a pure procedure (shared text) file that is being executed. For ftruncate() the same errors apply, but instead of things that can be wrong with path, we now have things that can be wrong with the file descriptor, fd: EBADF fd is not a valid descriptor. EBADF or EINVAL fd is not open for writing. EINVAL fd does not reference a regular file. EINVAL or EBADF The file descriptor fd is not open for writing. POSIX permits, and portable applications should handle, either error for this case. (Linux produces EINVAL.)

CONFORMING TO

POSIX.1-2001, POSIX.1-2008, 4.4BSD, SVr4 (these calls first appeared in 4.2BSD).

NOTES

BUGS

A header file bug in glibc 2.12 meant that the minimum value of _POSIX_C_SOURCE required to expose the declaration of ftruncate() was 200809L instead of 200112L. This has been fixed in later glibc versions.

SEE ALSO

open(2), stat(2), path_resolution(7)

Источник

Linux truncate opened file

NAME

truncate, ftruncate - truncate a file to a specified length

SYNOPSIS

#include unistd.h> #include  int truncate(const char *path, off_t length); int ftruncate(int fd, off_t length); Feature Test Macro Requirements for glibc (see feature_test_macros(7)): truncate(): _XOPEN_SOURCE >= 500 || /* Since glibc 2.12: */ _POSIX_C_SOURCE >= 200809L || /* Glibc versions ftruncate(): _XOPEN_SOURCE >= 500 || /* Since glibc 2.3.5: */ _POSIX_C_SOURCE >= 200112L || /* Glibc versions 
DESCRIPTION
The truncate() and ftruncate() functions cause the regular file named by path or referenced by fd to be truncated to a size of precisely length bytes. If the file previously was larger than this size, the extra data is lost. If the file previously was shorter, it is extended, and the extended part reads as null bytes ('\0'). The file offset is not changed. If the size changed, then the st_ctime and st_mtime fields (respectively, time of last status change and time of last modification; see inode(7)) for the file are updated, and the set-user-ID and set-group-ID mode bits may be cleared. With ftruncate(), the file must be open for writing; with truncate(), the file must be writable.
 
RETURN VALUE
 
On success, zero is returned. On error, -1 is returned, and errno is set appropriately.
 
ERRORS
 
For truncate(): EACCES Search permission is denied for a component of the path prefix, or the named file is not writable by the user. (See also path_resolution(7).) EFAULT The argument path points outside the process's allocated address space. EFBIG The argument length is larger than the maximum file size. (XSI) EINTR While blocked waiting to complete, the call was interrupted by a signal handler; see fcntl(2) and signal(7). EINVAL The argument length is negative or larger than the maximum file size. EIO An I/O error occurred updating the inode. EISDIR The named file is a directory. ELOOP Too many symbolic links were encountered in translating the pathname. ENAMETOOLONG A component of a pathname exceeded 255 characters, or an entire pathname exceeded 1023 characters. ENOENT The named file does not exist. ENOTDIR A component of the path prefix is not a directory. EPERM The underlying filesystem does not support extending a file beyond its current size. EPERM The operation was prevented by a file seal; see fcntl(2). EROFS The named file resides on a read-only filesystem. ETXTBSY The file is an executable file that is being executed. For ftruncate() the same errors apply, but instead of things that can be wrong with path, we now have things that can be wrong with the file descriptor, fd: EBADF fd is not a valid file descriptor. EBADF or EINVAL fd is not open for writing. EINVAL fd does not reference a regular file or a POSIX shared memory object. EINVAL or EBADF The file descriptor fd is not open for writing. POSIX permits, and portable applications should handle, either error for this case. (Linux produces EINVAL.)
 
CONFORMING TO
 
POSIX.1-2001, POSIX.1-2008, 4.4BSD, SVr4 (these calls first appeared in 4.2BSD).
 
NOTES
 
ftruncate() can also be used to set the size of a POSIX shared memory object; see shm_open(3). The details in DESCRIPTION are for XSI-compliant systems. For non-XSI-compliant systems, the POSIX standard allows two behaviors for ftruncate() when length exceeds the file length (note that truncate() is not specified at all in such an environment): either returning an error, or extending the file. Like most UNIX implementations, Linux follows the XSI requirement when dealing with native filesystems. However, some nonnative filesystems do not permit truncate() and ftruncate() to be used to extend a file beyond its current length: a notable example on Linux is VFAT. The original Linux truncate() and ftruncate() system calls were not designed to handle large file offsets. Consequently, Linux 2.4 added truncate64() and ftruncate64() system calls that handle large files. However, these details can be ignored by applications using glibc, whose wrapper functions transparently employ the more recent system calls where they are available. On some 32-bit architectures, the calling signature for these system calls differ, for the reasons described in syscall(2).
 
BUGS
 
A header file bug in glibc 2.12 meant that the minimum value of _POSIX_C_SOURCE required to expose the declaration of ftruncate() was 200809L instead of 200112L. This has been fixed in later glibc versions.
 
SEE ALSO
 
truncate(1), open(2), stat(2), path_resolution(7)
 
COLOPHON
 
This page is part of release 5.10 of the Linux man-pages project. A description of the project, information about reporting bugs, and the latest version of this page, can be found at https://www.kernel.org/doc/man-pages/.
 
© 2019 Canonical Ltd. Ubuntu and Canonical are registered trademarks of Canonical Ltd.
 
Источник
 
How to Empty a Log File in Linux
 
In this tutorial, you'll learn various ways to truncate a log file and delete its content without deleting the file itself.
 
You’ll find yourself in situations where you need to empty a file. This often happens when you have huge log files and How would you do that?
 
One not so clean way would be to remove the file and then create a new file. But this is not a good idea. It won’t be the same file; the timestamp (atime, mtime etc) will be different along with other file permissions.
 
Instead of creating a new empty file, you can delete its content. So, how do you empty a file in Linux? How to clear a file from all of its content without deleting the actual file?
 
4 ways to empty a file in Linux
 
There are several ways you can empty a file without actually deleting the file. Let me show you some of these methods.
 
Method 1: Truncate a file using truncate command
 
The safest way to truncate a log file is using the truncate command.
 
In the above command, -s is used to set/adjust the size (in bytes) of the file. When you use -s 0, it means you adjusted the file size to 0 bytes.
 
Method 2: Empty file using :> or >
 
The simplest way to empty a file is to use the command below. If the file is not in use, it will work in Bash:
 
While the above works only in Bash Shell, you can use a similar command for other shells:
 
You can also use this command to clear a file:
 
Method 3: Using echo command to empty file in Linux
 
Another way to empty a file is via echo command in Linux:
 
You can also use the echo command in this way:
 
Method 4: Use /dev/null to clear a file
 
You can also use the famous /dev/null and combine it with the cat command to clear a log file:
 
In the end…
 
And if you don’t have enough permissions for any of the above commands, this is the sure shot but a little dirty way to achieve it:
 
touch newfile mv newfile filename
 
If your aim was to clear log files and make some free space, you should learn about cleaning journald logs.
 
I hope this quick tip helped you to clear a file in Linux. Do bookmark us for more Linux quick tips.
 
Источник