How to get execution time of a script effectively?
This just show’s the time of start and end of the script. Would it be possible to display a fine grained output like processor time/ io time , etc?
@CiroSantilli烏坎事件2016六四事件法轮功 I think your proposed thread is still not sufficient to eliminate the effect of temporal events.
There is a command that you can run and then it times all unix commands automatically, forgot howto enable?
19 Answers 19
Just use time when you call the script:
and «real» is probably what people want to know — «Real is wall clock time — time from start to finish of the call»
start=`date +%s` stuff end=`date +%s` runtime=$((end-start)) or, if you need sub-second precision and have bc installed,
start=`date +%s.%N` stuff end=`date +%s.%N` runtime=$( echo "$end - $start" | bc -l ) Note that this only works if you don’t need sub-second precision. For some uses that might be acceptable, for others not. For slightly better precision (you’re still invoking date twice, for example, so you might at best get millisecond precision in practice, and probably less), try using date +%s.%N . ( %N is nanoseconds since the whole second.)
@ChrisH Oh. Good pointing it out; bash arithmetic expansion is integer-only. I see two obvious options; either ditch the period in the date format string (and treat the resultant value as nanoseconds since epoch), so use date +%s%N , or use something more capable like bc to calculate the actual runtime from the two values like jwchew suggests. Still, I feel this approach is a suboptimal way of doing it; time is considerably better if available, for reasons outlined above.
Further to @rubo77 comment, if your script takes several hours use hours=$((runtime / 3600)); minutes=$(( (runtime % 3600) / 60 )); seconds=$(( (runtime % 3600) % 60 )); echo «Runtime: $hours:$minutes:$seconds (hh:mm:ss)»
Just call times without arguments upon exiting your script.
With ksh or zsh , you can also use time instead. With zsh , time will also give you the wall clock time in addition to the user and system CPU time.
To preserve the exit status of your script, you can make it:
Or you can also add a trap on EXIT :
That way, times will be called whenever the shell exits and the exit status will be preserved.
$ bash -c 'trap times EXIT; : ' 0m0.932s 0m0.028s 0m0.000s 0m0.000s $ zsh -c 'trap time EXIT; : ' shell 0.67s user 0.01s system 100% cpu 0.677 total children 0.00s user 0.00s system 0% cpu 0.677 total Also note that all of bash , ksh and zsh have a $SECONDS special variable that automatically gets incremented every second. In both zsh and ksh93 , that variable can also be made floating point (with typeset -F SECONDS ) to get more precision. This is only wall clock time, not CPU time.
Does your approach eliminate the effect of temporal events? — — I think your approach is near time approach presented earlier. — — I think the timeit approach presented in MATLAB code can be useful here.
Hi, @Stéphane Chazelas. I found times doesn’t give wall time, right? for example, bash -c ‘trap times EXIT;sleep 2’
@Binarus, yes, that feature comes from ksh. Contrary to ksh93/zsh, beside bash not supporting floating point, it’s also broken in that after you set SECONDS to 0, it will change to 1 from 0 to 1 second later (as it only considers the result of time() which has only full second granularity).
Thank you very much for the hint. So in bash it takes at least one second and at maximum two seconds until it changes to 1 after having set it to 0. However, I guess that this won’t break anyone’s scripts, because no reasonable person will use a measurement facility with 1s resolution to measure time spans of one or two seconds. I usually use it to check time spans of 30 seconds or more, and where it isn’t important whether this is actually 31s or 29s . But once again, thank you very much for bringing this subtle and very interesting difference between the various shells to our attention.
# Reset BASH time counter SECONDS=0 # # do stuff # ELAPSED="Elapsed: $(($SECONDS / 3600))hrs $((($SECONDS / 60) % 60))min $(($SECONDS % 60))sec" This does show elapsed time, but it doesn’t show «fine grained output like processor time, I/O time» as requested in the question.
Those searching for an answer to the question as posed are likely to find this solution useful. Your comment would be better addressed to the OPs question.
Why do you need to set SECONDS to 0 at the start? Reading up on this it sounds like it starts at 0 when the script starts. If you want to time the whole script could you safely skip that line?
@sync Yes, if you are sure the script is actually run within its own subshell and does not inherit environment variables. It all depends. The safest way is local start=$SECONDS; … echo $((SECONDS-start)) .
Today I learned about $SECONDS : see askubuntu.com/questions/1028924/…. Could be useful to another bash newbie wondering why this would work
I’m a bit late to the bandwagon, but wanted to post my solution (for sub-second precision) in case others happen to stumble upon this thread through searching. The output is in format of days, hours, minutes, and finally seconds:
res1=$(date +%s.%N) # do stuff in here res2=$(date +%s.%N) dt=$(echo "$res2 - $res1" | bc) dd=$(echo "$dt/86400" | bc) dt2=$(echo "$dt-86400*$dd" | bc) dh=$(echo "$dt2/3600" | bc) dt3=$(echo "$dt2-3600*$dh" | bc) dm=$(echo "$dt3/60" | bc) ds=$(echo "$dt3-60*$dm" | bc) LC_NUMERIC=C printf "Total runtime: %d:%02d:%02d:%02.4f\n" $dd $dh $dm $ds Hope someone out there finds this useful!
[edit] You need to count all characters in field definition in bash printf, if you want pad seconds to 2 digits before dot you have to define it as %07.4f (all digits and dot count too in to filed length) so the line should look like: LC_NUMERIC=C printf «Total runtime: %d:%02d:%02d:%07.4f\n» $dd $dh $dm $ds
I guess there is no other (only bash) way except using ‘bc’ to do the calculations. BTW really good script 😉
Beware that FreeBSD’s date does not support sub-second precision and will just append literal “ N ” to the timestamp.
Very nice script, but my bash doesn’t handle the subsecond part. I also learned, that /1 in bc effectively strips that, so I added @/1@ to the $ds calculation, and it displays stuff very nice now!
bc supports modulo: dt2=$(echo «$dt%86400» | bc) . Just saying. BTW I prefer the form dt2=$(bc <<< "$
I don’t know anything about bc yet, but the division by 86400 makes me mistrustful for the following reason: There are days which do not have 86400 seconds, for example days where time is switched from DST to normal time and vice versa, or days with leap seconds. If such days are part of the time span you want to calculate, i.e. are between res1 and res2 (including these values), your script will probably fail.
Personally, I like to wrap all my script code in some «main» function like so:
main () < echo running . ># stuff . # calling the function at the very end of the script time main Notice how easy is to use the time command in this scenario. Obviously you’re not measuring the precise time including script parse time, but I find it accurate enough in most situations.
This is the best solution. only problem is that you can’t access script arguments from within main . To pass the arguments to the function you need to call time main «$@» in the last line
This question is quite old but in trying to find my favorite way of doing it this thread came up high. and I’m surprised no one mentioned it:
‘perf’ is a performance analyzing tool included in the kernel under ‘tools/perf’ and often available to install as a separate package (‘perf’ in CentOS and ‘linux-tools’ on Debian/Ubuntu). The Linux Kernal perf Wiki has much more information about it.
Running ‘perf stat’ gives quite a bit of details including average execution time right at the end:
1.002248382 seconds time elapsed ( +- 0.01% ) perf stat now complains about «You may not have permission to collect stats.» unless you run some commands with sudo , which makes it useless in all scenarios where you don’t completely own target machine.
Amazing, thanks! Way more fine-grained solution than time others suggesting. In particular, time is not applicable to measuring code competition solutions (as it doesn’t print time in milliseconds and less), whereas your solution does.
#!/bin/bash start=$(date +%s.%N) # HERE BE CODE end=$(date +%s.%N) runtime=$(python -c "print($ - $)") echo "Runtime was $runtime" Yes, this calls Python, but if you can live with that then this is quite a nice, terse solution.
Beware that running that python command in a subshell and reading its output will take several millions of nanoseconds on most current systems. Same for running date .
«Several millions of nanoseconds» is several milliseconds. People timing bash scripts are usually not very concerned about that (unless they run several billions of scripts per megasecond).
Notice also that, even if the calculation is done inside a python process, the values were entirely collected before python was invoked. The execution time of the script will be measured correctly, without the «millions of nanoseconds» overhead.
time main above is so much more clean. yes, we can live with lots of degradation but it’s way better without so let’s stay engineers!
A small shell function that can be added before commands to measure their time:
tm() < local start=$(date +%s) $@ local exit_code=$? echo >&2 "took ~$(($(date +%s)-$)) seconds. exited with $" return $exit_code > Then use it in your script, or on your command line like so:
tm the_original_command with all its parameters The accepted solution using time writes to stderr .
The solution using times writes to stdout .
The solutions using $SECONDS are missing sub-second precision.
The other solutions involve calling external programs like date or perf which is not efficient.
If you are fine with any of these, use it.
But if you need an efficient solution to get the times with millisecond precision and need them into variables such that the original output remains undisturbed you may combine process substitution with some redirections around time which is much faster than calling external programs and allows redirections around the timing wrapper script as on the original command/script.
# Preparations: Cmd=vgs # example of a program to be timed which is writing to stdout and stderr Cmd="eval < echo stdout; echo stderr >&2; sleep 0.1; >" # other example; replace with your own TIMEFORMAT="%3R %3U %3S" # make time output easy to parse Select one of the following variants parsing the output of time appropriate to you needs:
Shortest variant where stdout of $Cmd is written to stderr and nothing to stdout :
read Elapsed User System < <(< time $Cmd 2>&3; > 3>&2 2>&1 >&3) Longer variant that keeps original stdout and stderr separate of each other:
Most complicated variant includes closing the extra file descriptors such that $Cmd is called as if without this timing wrapper around it and lvm commands like vgs do not complain about leaked file descriptors:
You can even fake a floating point addition in bash without calling bc which would be much slower:
CPU=`printf %04d $((10#$+10#$))` # replace with your own postprocessing echo CPU $.$ s, Elapsed $Elapsed s >&2 # redirected independent of $Cmd Possible outputs with the two examples of $Cmd on a slow CPU:
File descriptor 3 (/dev/pts/1) leaked on vgs invocation. Parent PID 10756: bash File descriptor 4 (/dev/pts/1) leaked on vgs invocation. Parent PID 10756: bash VG #PV #LV #SN Attr VSize VFree b3 3 24 0 wz--n- 1.31t 1.19t CPU 0.052 s, Elapsed 0.056 s stdout stderr CPU 0.008 s, Elapsed 0.109 s