Monitor access to file linux

Linux — How to track all files accessed by a process?

Is there a way to track all file I/O for a given process? All I really need is the locations of files being read from/written to from a given process (and ideally if it was a read or write operation although that’s not as important). I can run the process and track it rather than needing to attach to an existing process which I would assume is significantly simpler. Is there any kind of wrapper utility I can run a process though that will monitor file access?

4 Answers 4

lsof :

Try doing this as a starter :

this command will list all currently open files, fd, sockets for the process with the passed process ID.

For your special needs, see what I can offer as a solution to monitor a php script :

php foo.php & _pid=$! lsof -r1 -p $_pid kill %1 # if you want to kill php script 

strace :

I recommend the use of strace . Unlike lsof , it stays running for as long as the process is running. It will print out which syscalls are being called when they are called. -e trace=file filters only for syscalls that access the filesystem:

sudo strace -f -t -e trace=file php foo.php 

or for an already running process :

sudo strace -f -t -e trace=file -p

Thanks that’s a good starting point! It works for processes already running at the moment it’s run. I’m trying to do this for a PHP script for its entire execution, tracking the files from the start of the process until it exists. Looking at the help, There’s a -r repeat option but this seems to periodically scan the files that are open by the process rather than have been opened. Essentially I want to do this: lsof -p $$ && exec php foo.php This doesn’t seem to list files that are opened by foo.php

thanks, that’s certainly providing more relevant information and showing all the php extensions being loaded, the script contains unfortunately, file.txt is not listed in the output. I can verify the file is being opened by amending the script to print the contents of file.txt but I still don’t see file.txt in the output of lsof.

To properly trace an AppImage, I needed to run strace as root but the command using my own user. This got the job done: sudo strace -fte trace=%file -u $(id -un)

Mixing your two solutions together becomes perfect: php foo.php & sudo strace -f -t -e trace=file -p $! especially for short running tasks.

Besides strace there is another option which does not substantially slow down the monitored process. Using the Liunx kernel’s fanotify (not to be confused with the more popular inotify) it is possible to monitor whole mount-points for IO-activity. With unshared mountnamespaces the mounts of a given process can be isolated fromt the rest of the system (a key technology behind docker).

An implementation of this concept can be found in shournal, which I am the author of.

$ shournal -e sh -c 'cat foo > bar' $ shournal --query --history 1 . 1 written file(s): /home/user/bar 1 read file(s): /home/user/foo 

External links are always highly appreciated as sources, but imagine this one was to become invalid — your solution would be unsalvageable for future SO users. Please consider posting code here and explaining your solution so we all can learn.

@harmonica141: That’s always the problem: what to write and what to omit. A complete, minimal example would be not much shorter than the example at the bottom at man7.org/linux/man-pages/man7/fanotify.7.html . In fact, it could be almost the same with a leading unshare( CLONE_NEWNS); . Do you think it would be helpful to include the full source here?

strace is an amazing tool but its output is a bit verbose.
If you want you can use a tool I’ve written which processes strace output and provide a CSV report of all files accessed (TCP sockets too) with the following data:
1. Filename
2. Read/Written bytes
3. Number of read/write operations
4. Number of time the file was opened

It can be run on new processes or processes already running (using /proc/fd data).
I found it useful for debugging scenarios and performance analysis.
You can find it here: iotrace

Filename, Read bytes, Written bytes, Opened, Read op, Write op /dev/pts/1,1,526512,0,1,8904 socket_127.0.0.1:47948->127.0.0.1:22,1781764,396,0,8905,11 myfile.txt,65,0,9,10,0 pipe:[3339],0,0,0,1,0 

Afterward, you can process the CSV data in Excel or other tools for sorting or other analysis required.
The downside is you need to download & compile and it isn’t always 100% accurate.

Источник

Monitoring File access, Changes and Data Modifications

Linux has several solutions to monitor what happens with your data. From changing contents to who accessed particular information, and at what time.

For our auditing toolkit Lynis, we researched and tested several solutions over the last few years. In this article we have a look at these solutions to monitor file access, changes and modifications to the data and beyond.

What is Data?

Data is a collection of bits, ordered in such a way it gives meaning to humans. The related information stored in data blocks, can be as simple as text, or become a visible representation like an image. Data is usually the most important part on a system, which means it has to be properly safeguarded.

Data versus Meta-data

Besides the information stored for us, the system needs to store a little bit of information as well. For example a data block on disk, might need some supporting information to know where it is stored. This data is usually not useful for us, but certainly for the system to retrieve the information, especially when we ask for it. This “data about data” is called meta-data. So besides protecting data, we have to take the protection of meta-data in mind as well.

Monitoring File Access

The first level of monitoring is who is accessing specific files. This helps us understand what particular files are being accessed, by what process and by whom. To accomplish this task, we can use the Linux audit framework. The framework is written by Red Hat and uses “watches” on files and directories to determine what should be monitored. Additionally it can monitor processes, including the underlying system calls which are performed by them.

Adding watches

To protect our kernel configuration, we can determine who accesses the sysctl.conf file. This file stores kernel settings, so it interesting to start with this file. To have this file monitored, we need to add a watch on the file.

auditctl -w /etc/sysctl.conf -p a -k kernel

The parameter -w sets the watch, followed by the file name. The -p defines the related permission action (a = attribute change, r = read, w = write, x = execute). It looks similar to file permissions, but actually it is slightly different. With the -k we define a custom key, which simplifies searching at a later moment. It is also helpful to categorize events.

Reporting watches

Now we have defined our watch, we can search for it with the earlier defined key.

Running this command gives us the following output:

File access monitoring with Linux audit framework

When looking at this output, you might be overwhelmed by all the fields available. Additionally some fields actually have rather strange values, like an architecture of c000003e (which actually equals x86_64).

The most important fields are the purple box, showing what object was hit and the green box revealing the process (or binary), followed by the defined key. In this case both the cat command and vim editor have opened the file

In this screenshot we can also see a failed syscall in the yellow box, with the value 89. To determine what syscall this is, we first have to look it up:

This will show all available syscalls for our particular system architecture. So in this case a call to “getrusage”, to retrieve process statistics from the kernel.

Monitoring specific functions

We can use the Linux audit framework also for monitoring specific system calls, or functions. We have to use the -S followed by the system call.

auditctl -a always,exit -S openat -F success=1

The -a always,exit defines to write out an event at exit time of the related system call.

For example when you want to monitor all successful “openat” calls, add this system call and tell auditctl only to log successful requests. In this case you might get a message that the system call is unclear, as it is found on multiple architectures. Find the related system call ID with ausyscall openat and add the ID instead. Even better is specifying the architecture together with the system call, as it is easier to read (example: -F arch=b64 -S openat).

For more tips regarding the Linux audit framework, have a look at our other article Configuring & Auditing Linux Systems with the Audit Daemon

File Integrity Monitoring

Another interesting level to monitor file changes, is by implementing file integrity tooling. Linux has several options for this, varying from simple tools up to kernel modules.

File Integrity Tools

The easiest way to verify if a file has been changed, is using tools. Simple tools like md5sum or shasum can help with detecting changes. Also specialized tools like AIDE and Samhain are a great help to set-up automatic monitoring and alerting.

Since setting up these tools are worth a blog post of their own, it will be covered in a separate post.

Integrity Measurement Architecture (IMA)

The most extensive option is monitoring files with IMA. This security module allows the system to create and monitor hashes for files and block unauthorized changes.

IMA has a few modes it can operate in, like fix and appraise. In “fix mode” the system allows the administrator to set hash values along each file. These hashes are small strings of text to help the system detect changes and are stored in extended attributes (xattrs) of the file system.

Digital signatures

Additionally IMA supports digital signing. This ensures you that the contents of the file is correct (or unaltered). Additionally because it is signed, you can validate the signature. So if a file is to be changed, it also needs proper signing.

Since IMA is a very extensive way of monitoring, we will cover more in other blogs posts. It’s a very exciting subject and a great help to protect your data.

Extended Verification Method (EVM)

Where IMA monitors the file contents, EVM performs monitoring of the file attributes. It also allows hashing and digital signing. It’s a great extension to IMA, to ensure that both contents and the attributes of a file are being unaltered.

Monitoring File Attributes

To monitor file permissions, we can also use the audit framework. File permissions and ownership are part of the file attributes. The file attributes can be monitored with “-p a“.

Additionally, we can use the earlier covered EVM to ensure attributes are not changed by an unauthorized process or person.

Conclusion

Now we have looked at some of the tools, it should be clear that a lot of areas can be monitored on Linux systems. It is up to the administrator to define what files should be monitored and to which extent. From simply logging changes to attributes with the Linux audit framework, up to fully blocking altered files with IMA and EVM.

One more thing.

Keep learning

So you are interested in Linux security? Join the Linux Security Expert training program, a practical and lab-based training ground. For those who want to become (or stay) a Linux security expert.

Security scanning with Lynis and Lynis Enterprise

Run automated security scans and increase your defenses. Lynis is an open source security tool to perform in-depth audits. It helps with system hardening, vulnerability discovery, and compliance.

Continue reading

Linux System Integrity Explained: Ensure Data, Logging and Kernel Integrity

About Linux Audit

This blog is part of our mission: help individuals and companies, to scan and secure their systems. We simply love Linux security, system hardening, and questions regarding compliance.

Besides the blog, we have our security auditing tool Lynis. Open source, GPL, and free to use.

For those with enterprise needs, or want to audit multiple systems, there is an Enterprise version.

«One security solution to audit, harden, and secure your Linux/UNIX systems.»

• Perform audits within a few minutes
• Central management
• Powerful reporting
• Compliance checks (e.g. PCI DSS)
• Additional plugins and more tests

Linux and UNIX security automation

Lynis is a free and open source security scanner. It helps with testing the defenses of your Linux, macOS, and Unix systems. Typical use-cases for this software include system hardening, vulnerability scanning, and checking compliance with security standards (PCI-DSS, ISO27001, etc).

Recent Posts

Contact

This blog is part of our mission to share valuable tips about Linux security. We are reachable via @linuxaudit

Company details

CISOfy
De Klok 28,
5251 DN, Vlijmen, The Netherlands
+31-20-2260055

Источник