Python linux find command

How do I execute a complex «find» linux shell command in python

@Barmar Well the script is supposed to list all the files in a certain directory. But it listed some files that it was not supposed to list. But I followed your advice and removed shell=True and now it works as expected!

2 Answers 2

You should only use shell=True when the first argument to Popen() is a string that should be parsed by the shell. If it’s an array, you’ve already done the necessary parsing, and shouldn’t use shell=True .

import commands commands = r'''find PATH -type f -exec du -h --all <> +''' result = commands.getstatusoutput(command)[0] print("<>".format(result)) 

Thank you for this code snippet, which may provide some immediate help. A proper explanation would greatly improve its educational value by showing why this is a good solution to the problem, and would make it more useful to future readers with similar, but not identical, questions. Please edit your answer to add explanation, and give an indication of what limitations and assumptions apply.

commands module is build over os.open(), this takes system command as a normal string and return a tuple containing any output generated by the command and, optionally, the exit status. First item in a tuple is always a status code of the commands run and item at index 1 is always an out generated by the command. However, if you’re using python 2.6 above or 3.x, then this module doesn’t work More details docs.python.org/2/library/commands.html

Источник

Python function that similar to bash find command

You can query os.environ[‘PYTHONPATH’] for its value in Python as shown below: IF defined in shell as THEN result => ELSE result => To set PYTHONPATH to multiple paths, see here. As mentioned in the comments another option, which is probably easier to read, is to use triple double quotes: While this answers the question, for ease of reading and maintainability I suggest to replace it instead completely with Python, as suggested in another answer.

Python function that similar to bash find command

I have a dir structure like the following:

[me@mypc]$ tree . . ├── set01 │ ├── 01 │ │ ├── p1-001a.png │ │ ├── p1-001b.png │ │ ├── p1-001c.png │ │ ├── p1-001d.png │ │ └── p1-001e.png │ ├── 02 │ │ ├── p2-001a.png │ │ ├── p2-001b.png │ │ ├── p2-001c.png │ │ ├── p2-001d.png │ │ └── p2-001e.png 

I would like to write a Python script to rename all *a.png to 01.png, *b.png to 02.png, and so on. Frist I guess I have to use something similar to find . -name ‘*.png’ , and the most similar thing I found in python was os.walk . However, in os.walk I have to check every file, if it’s png, then I’ll concatenate it with it’s root, somehow not that elegant. I was wondering if there is a better way to do this? Thanks in advance.

For a search pattern like that, you can probably get away with glob .

from glob import glob paths = glob('set01/*/*.png') 

You can use os.walk to traverse the directory tree . Maybe this works?

import os for dpath, dnames, fnames in os.walk("."): for i, fname in enumerate([os.path.join(dpath, fname) for fname in fnames]): if fname.endswith(".png"): #os.rename(fname, os.path.join(dpath, "%04d.png" % i)) print "mv %s %s" % (fname, os.path.join(dpath, "%04d.png" % i)) 

For Python 3.4+ you may want to use pathlib.glob() with a recursive pattern (e.g., **/*.png ) instead:

• recursively iterate through all subdirectories using pathlib
• https://docs.python.org/3/library/pathlib.html#pathlib.Path.glob
• https://docs.python.org/3/library/pathlib.html#pathlib.Path.rglob

Check out genfind.py from David M. Beazley.

# genfind.py # # A function that generates files that match a given filename pattern import os import fnmatch def gen_find(filepat,top): for path, dirlist, filelist in os.walk(top): for name in fnmatch.filter(filelist,filepat): yield os.path.join(path,name) # Example use if __name__ == '__main__': lognames = gen_find("access-log*","www") for name in lognames: print name 

These days, pathlib is a convenient option.

Python MongoDB Find, The find () method returns all occurrences in the selection. The first parameter of the find () method is a query object. In this example we use an empty query object, which selects all documents in the collection. No parameters in the find () method gives you the same result as SELECT * in MySQL. Example

Executing bash’s complex find command in python shell

I am new to python. I am trying to execute a bash script in python to extract the count of different file extensions . I tried the following command

import subprocess output = subprocess.check_output("sudo find . -type f -name '*.*' -exec sh -c 'echo $' <> \; | sort | uniq -c | sort -nr | awk ''", shell=True) 

But it throws a syntax error. On executing find command in bash shell

sudo find . -type f -name '*.*' -exec sh -c 'echo $' <> \; | sort | uniq -c | sort -nr | awk '' 

output will be as follows

png:3156 json:333 c:282 svg:241 zsh:233 js:192 gz:169 zsh-theme:143 ttf:107 cache:103 md:93 

So how can i get the same output in python code? what is the correction required in my current approach? Thanks in advance

As mentioned in the comments any double quote in a string quoted with double quotes needs to be escaped with a backslash:

import subprocess output = subprocess.check_output("sudo find . -type f -name '*.*' -exec sh -c 'echo $' <> \; | sort | uniq -c | sort -nr | awk ''", shell=True) 

Single quotes inside a double quoted string do not have any special meaning (except directly at the beginning), so that doesn’t allow you to avoid escaping.

The fine details are explained under the header String and Bytes literals from the Python language reference.

As mentioned in the comments another option, which is probably easier to read, is to use triple double quotes:

import subprocess output = subprocess.check_output("""sudo find . -type f -name '*.*' -exec sh -c 'echo $' <> \; | sort | uniq -c | sort -nr | awk ''""", shell=True) 

While this answers the question, for ease of reading and maintainability I suggest to replace it instead completely with Python, as suggested in another answer.

By the way, you could try to do the same thing in pure Python. Here is a minimal code that does it:

import os def count_all_ext ( path ): res = <> for root,dirs,files in os.walk( path ): for f in files : if '.' in f : e = f.rsplit('.',1)[1] res[e] = res.setdefault(e,0)+1 return res.items() print '\n'.join( '%s:%d'%i for i in count_all_ext('.')) 

OK, it’s very long compared to the Bash snippet , but it’s Python.

Python function that similar to bash find command, I would like to write a python script to rename all *a.png to 01.png, *b.png to 02.png, and so on. Frist I guess I have to use something similar to find . -name ‘*.png’, and the most similar thing I found in python was os.walk.

Implementation of the Linux find command in python

Can someone point me to a set of instructions of how to actually implement and utilize the find command within a Python script?

I have looked at: https://docs.python.org/2/library/subprocess.html I’m not exactly sure how to utilize the command, even if implementation goes well with either subprocess.call or subprocess.Popen . From all of the SO threads that I’ve read concerning this topic, it seems like these are the two best options. However, I am not sure what kind of arguments I would need to use, as well as how to specifically implement the find command within them.

Can someone please demonstrate how to use find in the context of subprocess.call or subprocess.Popen so that I can later call find directly in my Python script?

You probably want to use the check_output() helper function.

find_output = subprocess.check_output('find ~', shell = True) 

In the above example, find_output will contain a bytes instance of the find commands stdout . If you wish to capture the stderr as well, add stderr=subprocess.STDOUT as a keyword argument.

found = subprocess.Popen(['find', '.'],stdout=subprocess.PIPE) for line in iter(found.stdout.readline, ''): print line, 

The which Command in Python, This command can identify the path for a given executable. In this tutorial, we will emulate this command in Python. Use the shutil.which () Function to Emulate the which Command in Python We can emulate this command in Python using the shutil.which () function. This function is a recent addition in Python 3.3.

How do I find out my PYTHONPATH using Python?

How do I find out which directories are listed in my system’s PYTHONPATH variable, from within a Python script (or the interactive shell)?

You would probably also want this:

Or as a one liner from the terminal:

python -c "import sys; print('\n'.join(sys.path))" 

Caveat : If you have multiple versions of Python installed you should use a corresponding command python2 or python3 .

sys.path might include items that aren’t specifically in your PYTHONPATH environment variable . To query the variable directly, use:

import os try: user_paths = os.environ['PYTHONPATH'].split(os.pathsep) except KeyError: user_paths = [] 

Can’t seem to edit the other answer. Has a minor error in that it is Windows-only. The more generic solution is to use os.pathsep as below:

sys.path might include items that aren’t specifically in your PYTHONPATH environment variable . To query the variable directly, use:

import os os.environ.get('PYTHONPATH', '').split(os.pathsep) 

PYTHONPATH is an environment variable whose value is a list of directories. Once set, it is used by Python to search for imported modules , along with other std. and 3rd-party library directories listed in Python’s «sys.path».

As any other environment variables, you can either export it in shell or in ~/.bashrc, see here. You can query os.environ [‘PYTHONPATH’] for its value in Python as shown below:

$ python3 -c "import os, sys; print(os.environ['PYTHONPATH']); print(sys.path) if 'PYTHONPATH' in sorted(os.environ) else print('PYTHONPATH is not defined')" 

$ export PYTHONPATH=$HOME/Documents/DjangoTutorial/mysite 

/home/Documents/DjangoTutorial/mysite ['', '/home/Documents/DjangoTutorial/mysite', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7', '/usr/local/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/site-packages'] 

PYTHONPATH is not defined 

To set PYTHONPATH to multiple paths, see here.

Note that one can add or delete a search path via sys.path.insert(), del or remove() at run-time, but NOT through os.environ[]. Example:

>>> os.environ['PYTHONPATH']="$HOME/Documents/DjangoTutorial/mysite" >>> 'PYTHONPATH' in sorted(os.environ) True >>> sys.path // but Not there ['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7', '/usr/local/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/site-packages'] >>> sys.path.insert(0,os.environ['PYTHONPATH']) >>> sys.path // It's there ['$HOME/Documents/DjangoTutorial/mysite', '', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7', '/usr/local/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/site-packages'] >>> 

In summary, PYTHONPATH is one way of specifying the Python search path(s) for imported modules in sys.path. You can also apply list operations directly to sys.path without the aid of PYTHONPATH.

Learn List of Basic To Advanced Python Commands, Comments: # symbol is being used for comments in python.For multiline comments, you have to use “”” symbols or enclosing the comment in the “”” symbol.; Example: print “Hello World” # this is the comment section. Example: “”” This is Hello world project.””” Type function: These Python Commands are used …

Источник