How do I sort grep output by time+date of file created?
But the above output is sorted alphabetically. If we sorted them by time+date created, the output should look like this:
run_it000_model.star:_rlnAveragePmax 0.000000 run_it001_model.star:_rlnAveragePmax 0.008995 run_it002_model.star:_rlnAveragePmax 2.517429e-04 run_it003_model.star:_rlnAveragePmax 0.003727 run_it004_model.star:_rlnAveragePmax 0.056681 run_it005_model.star:_rlnAveragePmax 0.109754 run_it006_model.star:_rlnAveragePmax 0.153500 run_ct6_it006_model.star:_rlnAveragePmax 0.153500 run_ct6_it007_model.star:_rlnAveragePmax 0.096772 2 Answers 2
Unless you’re running macOS, the date a file is created is not stored. That concept is poorly defined anyway, since editing a file often creates a new file that replaces the previous version. What is recorded is the date a file was last modified (which includes both initial creation and any subsequent edit).
The easiest way to sort files by their modification time is to run zsh (as opposed to other, less powerful shells such as bash, ksh or fish). In zsh, you can use glob qualifiers such as Om to sort files by modification time (oldest first, make that Om to get youngest first).
grep _rlnAveragePmax *model*(Om) That gives the desired result since grep traverses the files in the order given on the command line.
First do a time sorted ls, and then pipe the stdout to grep through xargs, using the -H flag to include the filename (since it will do it for only one file, default does not include the filename)
In a first grep after ls, directories are excluded (think this is not a recursive search from the example you provided). A second grep after ls, filters for the filename (without xargs)
ls -tpQ --quoting-style=shell | grep -v / | grep model | xargs -i grep -H _rlnAveragePmax <> Update after Wildcard’s valuable comments: In order to take into account all cases in which filenames may have strange characters, inode numbers are extracted from ls, and piped into «find» for inode number matching and executing grep command. xargs is replaced with «while read line» since there would be two <>‘s that the xargs command couldn’t differentiate
Symlinks are followed, only files are included, excluding directories etc. The maxdepth argument is redundant in this context (only files are left) however I included it so that a recursive search could also be possible, changing the arguments:
ls -ti | cut -d " " -f1 | while read line; do find -L . -maxdepth 1 -type f -inum $line -exec grep -H xxx <> \; ; done Grep inside all files created within date range
I am on the Ubuntu OS. I want to grep a word (say XYZ) inside all log files which are created within date range 28-may-2012 to 30-may-2012. How do I do that?
3 Answers 3
This is a little different from Banthar’s solution, but it will work with versions of find that don’t support -newermt and it shows how to use the xargs command, which is a very useful tool.
You can use the find command to locate files «of a certain age». This will find all files modified between 5 and 10 days ago:
find /directory -type f -mtime -10 -mtime +5 To then search those files for a string:
find /directory -type f -mtime -10 -mtime +5 -print0 | xargs -0 grep -l expression You can also use the -exec switch, but I find xargs more readable (and it will often perform better, too, but possibly not in this case).
(Note that the -0 flag is there to let this command operate on files with embedded spaces, such as this is my filename .)
Update for question in comments
When you provide multiple expressions to find , they are ANDed together. E.g., if you ask for:
. find will only return files that are both (a) named foo and (b) larger than 10 kbytes. Similarly, if you specify:
. find will only return files that are (a) newer than 10 days ago and (b) older than 5 days ago.
For example, on my system it is currently:
$ date Fri Aug 19 12:55:21 EDT 2016 I have the following files:
$ ls -l total 0 -rw-rw-r--. 1 lars lars 0 Aug 15 00:00 file1 -rw-rw-r--. 1 lars lars 0 Aug 10 00:00 file2 -rw-rw-r--. 1 lars lars 0 Aug 5 00:00 file3 If I ask for «files modified more than 5 days ago ( -mtime +5 ) I get:
$ find . -mtime +5 ./file3 ./file2 But if I ask for «files modified more than 5 days ago but less than 10 days ago» ( -mtime +5 -mtime -10 ), I get:
$ find . -mtime +5 -mtime -10 ./file2 How to search for a specific string in files from a specific date?
Now I want to search for a certain string like ‘JMS111′ in all files where date = ’21 Feb’. I tried below commands but wasn’t successful:
ls -lt |grep 'JMS111' -ctime 2019/02/21 find -name '*.xml' -type f -printf '%Tw:%h/%f\0' | grep 'JMS111' note that putty is just your interface to some other system (presumably Linux, given the tag); I’ll remove the putty tag since your question is more directly about find & grep
If any of the answers solved your problem, please accept it by clicking the checkmark next to it. Thank you!
4 Answers 4
One way to approach this is to set up timestamp files to bracket the dates you’re looking for:
touch -t 201902210000 /tmp/start-time touch -t 201902212359.59 /tmp/end-time . and then ask find for files that are newer than the start-time but not newer than the end-time:
find . -type f -newer /tmp/start-time ! -newer /tmp/end-time touch -t 201902210000 /tmp/start-time touch -t 201902212359.59 /tmp/end-time find . -type f -newer /tmp/start-time ! -newer /tmp/end-time -name '*.xml' -exec grep JMS111 /dev/null <> + rm /tmp/start-time /tmp/end-time This will look for files in the right timeframe with names ending in .xml and then pass those filenames to the grep , fitting in as many as it can per pass. I’ve added /dev/null as a «file» for grep to search from along with the filenames from find , if any. That way, if there is only one matching file, grep is «forced» to report the matching filename (as /dev/null will never match, it won’t be reported).
Your output will be the filenames and matching lines from those files.
. or with GNU find , you could use date strings directly I think e.g. -newermt ’21 Feb’ ! -newermt ’22 Feb’
I recommend grep -H , in case there’s only one matching file. Or, more portably, add /dev/null to the grep command to ensure there’s always at least two.
Thanks, Toby! I’ve incorporated the /dev/null suggestion as a more-general solution. Feel free to edit in such excellent suggestions directly next time!
find . -name ‘a[123]’ -ctime 6 | xargs grep -l ‘JMS111’
The -ctime 6 looks for any file that’s 6 days old. It actually compares the creation date of the file to the 24 hour period that’s between 6*24 hours an 7*24 hours prior to your running the command. If the creation date is in that period, and the filename is ‘a1’, ‘a2’, or ‘a3’, then find prints the path of that file. Since the period specified by -ctime is based on when you ran the find command, you will have to use a different value for if you run find tomorrow, than if you run it today.
Note: If you use -ctime -6 , find will print filenames that are 6 or fewer days old, -ctime +6 will print file names that are 6 days or older. Also, some versions of find will allow you to specify units of measurement, e.g 6h, for 6 hours old.
You could instead use -mtime if you want to search on modification time instead of creation time.
Using xargs makes it easy to check if you are getting the correct list of file from find before doing the grep . Also, if you put grep in a -exec primary, find invokes it once for each file it finds, whereas, xargs receives a list of files that it passes to a single invocation of grep , which is less resource intensive.
One disadvantage of -ctime is you have to calculate the number of days since the file was created, but you can figure that out by running
find . -ctime n | xargs ls -l until you see files with the date you’re interested in.
. And, you don’t have to remember to delete the bracketing files like you do with -newer .
How to find data between range of time using grep?
I am a new bash script student. I am learning about grep! I have a file that contains information about computer or activity. I am trying to use grep command to find information about my computer during a specific time range. I have tried following commands, but some work and some don’t:
grep -E "Jan 09 22:00:00 && Jan 09 22:05:00" /etc/active #doesn't work grep -E "Jan 09 22:00:00" && "Jan 09 22:05:00" /etc/active #doesn't work grep -E "Jan 09 22:00:00" | grep -E "Jan 09 22:05:00" /etc/active #doesn't work If I try grep -E «Jan 09 22:00» /etc/active this work, but it is only one time. I want between two different times I know there is a command awk , but we haven’t started with awk!
1 Answer 1
I think what you are asking for simply doesn’t work too good with grep. You see, grep is about matching text against patterns.
grep has no understanding of «dates»; or the semantics of dates. grep doesn’t understand that «Jan 09 22:00:00» denotes a point in time; and that you are looking for the entries up to the next 5 minutes.
The only thing that you can do: use a pattern that matches that input.
should match any timestamp from 22.00.00 to 22.09.59
In other words: you can study how to create more sophisticated patterns (see here for some ideas); and then try to find patterns that match exactly what you need. But please understand that this might be hard, depending on your requirements.
Coming back to my intial point: grep is a tool to do «stupid» text matching. You are looking for something that works on a «higher level of abstraction».
EDIT: I actually really told you what you need to know, but maybe you need an example. So I cretead a file forgrep and put this in
Jan 09 22:00:00 zero Jan 09 22:01:00 one Jan 09 22:02:00 two Jan 09 22:03:00 three See what is possible using different regular expressions:
grep -E "Jan 09 22:01:00" forgrep Jan 09 22:01:00 one The above matches the exact string provided, but
grep -E "Jan 09 22:0.:00" forgrep Jan 09 22:00:00 zero Jan 09 22:01:00 one Jan 09 22:02:00 two Jan 09 22:03:00 three simply matches any number there (because of the .)
grep -E "Jan 09 22:0[12]:00" forgrep Jan 09 22:01:00 one Jan 09 22:02:00 two matches numbers 1 and 2 in that place!
It doesn’t get «better than that». You simply have study the link I provided to you and start making experiments.
As you can see: you can use patterns to