cat, grep and cut — translated to python
maybe there are enough questions and/or solutions for this, but I just can’t help myself with this one question: I’ve got the following command I’m using in a bash-script:
var=$(cat "$filename" | grep "something" | cut -d'"' -f2) Now, because of some issues I have to translate all the code to python. I never used python before and I have absolutely no idea how I can do what the postet commands do. Any ideas how to solve that with python?
You asked for feedback, so I provided it. Your question is more or less «write me some python to do this» without showing any effort yourself or where you got stuck. If you tried and failed, then people will be more likely to want to help you and less likely to downvote if you include your efforts and show where you got stuck.
Maybe lose the bad attitude as well, people are trying to help you. Dumping a load of failed attempts in a question is no more useful than simply asking for the answer. You need to make it clear what you don’t understand and ask a clear, focused question, then you will rightfully be upvoted.
5 Answers 5
You need to have better understanding of the python language and its standard library to translate the expression
cat «$filename»: Reads the file cat «$filename» and dumps the content to stdout
| : pipe redirects the stdout from previous command and feeds it to the stdin of the next command
grep «something»: Searches the regular expression something plain text data file (if specified) or in the stdin and returns the matching lines.
cut -d'»‘ -f2: Splits the string with the specific delimiter and indexes/splices particular fields from the resultant list
cat "$filename" | with open("$filename",'r') as fin: | Read the file Sequentially | for line in fin: | ----------------------------------------------------------------------------------- grep 'something' | import re | The python version returns | line = re.findall(r'something', line)[0] | a list of matches. We are only | | interested in the zero group ----------------------------------------------------------------------------------- cut -d'"' -f2 | line = line.split('"')[1] | Splits the string and selects | | the second field (which is | | index 1 in python) Combining
import re with open("filename") as origin_file: for line in origin_file: line = re.findall(r'something', line) if line: line = line[0].split('"')[1] print line How to remove the last character from a bash grep output
What I want to do is I want to remove the trailing «;» as well. How can i do that? I am a beginner to bash. Any thoughts or suggestions would be helpful.
For the problem at hand, it could have been solve with just grep: COMPANY_NAME=$(grep -Po ‘(?<=company_name=)"[^"]*"' file.txt)
14 Answers 14
This will remove the last character contained in your COMPANY_NAME var regardless if it is or not a semicolon:
echo "$COMPANY_NAME" | rev | cut -c 2- | rev COMPANY_NAME=`cat file.txt | grep "company_name" | cut -d '=' -f 2 | sed 's/;$//'` @Anony-Mousse Yes, I know there are at least two ways to avoid cat here. I left in the cat in order to avoid changing the command line from the question beyond what was actually necessary to make it work.
@Anony-Mousse Not really in all cases, simply grep without cat -v will hide invisible (e.g. malicious) characters, unix.stackexchange.com/questions/202198/…
foo="hello world" echo $ hello worl This just made my day because I was trying to make a quick and dirty list of websites we had TLS keys for. for L in `ls *key` ; do echo $
I’d use head —bytes -1 , or head -c-1 for short.
COMPANY_NAME=`cat file.txt | grep "company_name" | cut -d '=' -f 2 | head --bytes -1` head outputs only the beginning of a stream or file. Typically it counts lines, but it can be made to count characters/bytes instead. head —bytes 10 will output the first ten characters, but head —bytes -10 will output everything except the last ten.
NB: you may have issues if the final character is multi-byte, but a semi-colon isn’t
I’d recommend this solution over sed or cut because
- It’s exactly what head was designed to do, thus less command-line options and an easier-to-read command
- It saves you having to think about regular expressions, which are cool/powerful but often overkill
- It saves your machine having to think about regular expressions, so will be imperceptibly faster
I’ve tested the other suggested solution and this seems as the best one (and a simple one!) for my use case. Thanks!
I believe the cleanest way to strip a single character from a string with bash is:
but I haven’t been able to embed the grep piece within the curly braces, so your particular task becomes a two-liner:
COMPANY_NAME=$(grep "company_name" file.txt); COMPANY_NAME=$
This will strip any character, semicolon or not, but can get rid of the semicolon specifically, too. To remove ALL semicolons, wherever they may fall:
To remove only a semicolon at the end:
Or, to remove multiple semicolons from the end:
For great detail and more on this approach, The Linux Documentation Project covers a lot of ground at http://tldp.org/LDP/abs/html/string-manipulation.html
Using sed , if you don’t know what the last character actually is:
$ grep company_name file.txt | cut -d '=' -f2 | sed 's/.$//' "Abc Inc" Don’t abuse cat s. Did you know that grep can read files, too?
The canonical approach would be this:
grep "company_name" file.txt | cut -d '=' -f 2 | sed -e 's/;$//' the smarter approach would use a single perl or awk statement, which can do filter and different transformations at once. For example something like this:
COMPANY_NAME=$( perl -ne '/company_name=(.*);/ && print $1' file.txt ) don’t have to chain so many tools. Just one awk command does the job
COMPANY_NAME=$(awk -F"=" '/company_name/' file.txt) you can strip the beginnings and ends of a string by N characters using this bash construct, as someone said already
$ fred=abcdefg.rpm $ echo $ bcdefg HOWEVER, this is not supported in older versions of bash.. as I discovered just now writing a script for a Red hat EL6 install process. This is the sole reason for posting here. A hacky way to achieve this is to use sed with extended regex like this:
$ fred=abcdefg.rpm $ echo $fred | sed -re 's/^.(.*). $/\1/g' bcdefg In Bash using only one external utility:
IFS='= ' read -r discard COMPANY_NAME Assuming the quotation marks are actually part of the output, couldn’t you just use the -o switch to return everything between the quote marks?
COMPANY_NAME="\"ABC Inc\";" | echo $COMPANY_NAME | grep -o "\"*.*\"" Some refinements to answer above. To remove more than one char you add multiple question marks. For example, to remove last two chars from variable $SRC_IP_MSG, you can use:
cat file.txt | grep "company_name" | cut -d '=' -f 2 | cut -d ';' -f 1 as linus torvalds says, one of the most difficult things to find in a programmer is «good taste», which is difficult to define. but since you would benefit from it i will do so in this case: because of the initial cat every element of the pipeline that does real work operates on its standard input and its standard output. the benefit is that you can replace the cat with some other pipeline that produces output similar to file.txt and you don’t have to change even a single character in the functional part of the pipeline. this allows drop-in pipeline reusability.
@Det fyb above (forgot to tag you, just like somebody might forget to remove the input argument when composing pipelines and wonder why the output isn’t as expected)
@randomstring, while I don’t really see this as you giving me a «piece of your mind» (I lol’d by the way) it’s rather extreme reusability to preserve the use of cat just to give you the possibility of changing it to something else later on. The only thing I could see it worth for is simplicity. If a user types grep pattern file , then I’m pretty damn sure he understands it’s that first part which reads the file and where he starts piping it.