Passing variables to docker run from within a bash shell, getting «unterminated quoted string» error
I have a bash shell with multiple variables that form the command options of a docker run command. I suspect a < character is breaking the config line that I am passing. This is the command that I am trying to pass:
docker run -it --entrypoint /bin/sh alpine -c \ "apk update && apk add application && application -f < /dir/file.log" When I run it on the command line it runs fine. If I even run it in the script as that line it runs fine. But when it runs in the script with the variable substitution, it gives the following error:
update: line 1: syntax error: unterminated quoted string This is my actual script: (it is probably NOT best practice, but I was struggling to pass multiple commands to the entrypoint command, so hacked it so:
DOCKER_SHELL=/bin/sh DOCKER_IMAGE=alpine TEMP=/tmp FILE=file.log DOCKER_ENTRY="apk update && apk add application && application -f < "$TEMP"/"$FILE TEST=\"$DOCKER_ENTRY\" docker run -it --entrypoint $DOCKER_SHELL $DOCKER_IMAGE -c $TEST 1 Answer 1
General information
The quoting in your code is very wrong. Important information:
- An unquoted variable undergoes word splitting and filename generation.
- Quotes that appear from variable expansion are not special to the shell that expanded the variable.
- There are two shells: the shell that interprets the script and the shell running in a container. Each parses code on its own.
Additionally consider lowercase variable names. If you get used to using uppercase names then sooner or later you will inadvertently change PATH , IFS , USER , TERM , HOME or anything like this and something will go wrong.
Specific analysis
DOCKER_ENTRY="apk update && apk add application && application -f < "$TEMP"/"$FILE $TEMP and $FILE are not quoted. Their particular values in the script are "safe", so the lack of quoting does not bite you in this case (unless IFS from the environment is "unfortunate"). But what if you change them to "non-safe" in the future? Will you remember to fix the code? The right thing is to double-quote regardless of the value.
the variable is not quoted. It's a case where not-quoting is not a bug (still IMO it's easier to always quote than to remember all edge cases where you can safely omit quotes). The escaped double-quote characters are not special and get stored into the TEST variable.
docker run -it --entrypoint $DOCKER_SHELL $DOCKER_IMAGE -c $TEST no variable is quoted. Similarly as it was with $TEMP and $FILE , the values of $DOCKER_SHELL and $DOCKER_IMAGE are "safe" (unless IFS …). What you're doing with $TEST is nonetheless fatal. After variable expansion the command will be equivalent to:
docker run -it --entrypoint /bin/sh alpine -c \"apk update && apk add application && application -f < /tmp/file.log\" where double-quotes from the expansion of $TEST are not special and therefore I showed them as \" as if I was typing non-special double-quotes in a shell. Word splitting (because of unquoted $TEST ) makes "apk a separate word, update a separate word, and so on.
Inside the container /bin/sh will run with the following arguments: -c , "apk , update , && , apk , add , application , && , application , -f , < , /tmp/file.log" .
The option-argument to -c is "apk . This is the only code /bin/sh inside the container will try to run. The code contains unterminated quoted string. update works like the second sh in this question: What is the second sh in sh -c 'some shell code' sh ? That's why you got:
update: line 1: syntax error: unterminated quoted string Solutions
This snippet fixes quoting, introduces lowercase variable names and a shebang:
#!/bin/sh docker_shell=/bin/sh docker_image=alpine temp=/tmp file=file.log docker_entry="apk update && apk add application && application -f < $temp/$file" docker run -it --entrypoint "$docker_shell" "$docker_image" -c "$docker_entry" It can be improved and we will improve it in a moment. For now the most important thing is the double-quoted $docker_entry expands to a single word, so after variable expansion the last line will be equivalent to:
docker run -it --entrypoint /bin/sh alpine -c 'apk update && apk add application && application -f < /tmp/file.log' where I used single-quotes to indicate the single word from the expansion of $docker_entry , as if I was trying to pass a single word while typing in a shell.
Now inside the container /bin/sh will run with the following arguments: -c , apk update && apk add application && application -f < /tmp/file.log .
Note file'.log and file.log; do_unwanted_things are legitimate names. You may want to use any of them or whatever as the name. To use them in the above script you need additional escaping or quoting in the shell code passed to sh . Embedding quotes in the code like this:
docker_entry="apk update && apk add application && application -f < '$temp'/'$file'" (see quotes within quotes) is not a robust way because a single-quote (like in file'.log ) will interfere, break things and open a possibility to inject code (e.g. do_unwanted_things ).
You can manually alter the original variable so it behaves well later. Example:
file='file.log\; do_unwanted_things' If your $temp and $file are known in advance and static then you can always find such quoting/escaping (inside temp , file and/or docker_entry ) so it behaves well and unwanted things are not treated as code. It's not that simple if you want to reliably support any value without manually adjusting the code each time (so maybe a value not known in advance).
In general it will be better if you are able to use any name without worrying some part of it will be interpreted as code (e.g. a meaningful quote or actual command to be run).
In Bash you can try with $ and $ . The Q operator causes the value to be quoted in a format that can be reused as shell code to be parsed, so after parsing you get the original value back. This is quite helpful if you must reuse a variable inside shell code (e.g. with ssh user@server code where there is no other simple way to pass variables than inside code ). We will get back to this approach.
In case of sh -c we don't need to pass $temp and $file as code. Shells provide a robust way to pass data as parameters:
#!/bin/sh docker_shell=/bin/sh docker_image=alpine temp=/tmp file=file.log docker_entry='apk update && apk add application && application -f < "$1/$2"' docker run -it --entrypoint "$docker_shell" "$docker_image" -c "$docker_entry" sh "$temp" "$file" Now the locally expanded value of $temp will be known to the shell in the container as $1 ; and the expanded value of $file will be known as $2 . docker_entry will contain "$1/$2" as a literal string. $1 and $2 will be expanded in the container. $1/$2 is properly double-quoted inside the shell code run in the container. All this ensures the value of $temp (or $file ) is never treated as shell code.
Instead of concatenating $1 and $2 in the container ( $1/$2 ) you can concatenate and form a single argument in the script: $temp/$file ; and then refer to it as $1 . Like this:
… docker_entry='apk update && apk add application && application -f < "$1"' docker run -it --entrypoint "$docker_shell" "$docker_image" -c "$docker_entry" sh "$temp/$file" Regardless if you choose to pass $temp and $file as two arguments or $temp/$file as one, the code should be totally safe. The only downside I see is docker_entry contains $1/$2 or $1 which are not descriptive. Where the variable is defined it's not clear what these positional parameters (will) mean. For this reason consider the following example that is somewhat convoluted, yet after you get its point you may actually like it, especially in pure sh :
#!/bin/sh docker_shell=/bin/sh docker_image=alpine temp=/tmp file=file.log docker_entry='apk update && apk add application && application -f < "$temp/$file"' docker run -it --entrypoint "$docker_shell" "$docker_image" -c \ 'temp="$1"; file="$2"; '"$docker_entry" sh "$temp" "$file" Now docker_entry contains meaningful variable names (note: it actually contains names, not values). The docker … line prepends shell code ( temp="$1"; file="$2"; ) that works in the container and creates variables from the local variables supplied as later arguments. This way when the shell in the container gets to the code from the expansion of $docker_entry , the variables will be there.
With (local) Bash you can use meaningful names without additional shell code. Use the already mentioned Q operator:
#!/bin/bash docker_shell=/bin/sh docker_image=alpine temp=/tmp file=file.log docker_entry="apk update && apk add application && application -f < $/$" docker run -it --entrypoint "$docker_shell" "$docker_image" -c "$docker_entry" Now docker_entry contains the values of $temp and $file , as opposed to literal strings $temp and $file in the previous example (however in the script itself you see meaningful names in both cases, not $1 and such). The values are not necessarily the original values; if needed they are modified by the Q operator, so they are always safe when embedded in shell code like this.
docker_entry containing the values is similar to what you were trying to do; and to what the first fixed script in this answer does. The improvement is: with @Q any value will be handled firmly and safely.
Final note
I don't really know Docker. I don't know "if there is a better way of running the --entrypoint command". My answer does not address this concern at all.
"Syntax error: Unterminated quoted string"
I am trying to use a backup script that will backup my sql database and website files. I am running Ubuntu 12.04 32 bit version and zPanel. Also what are these variables: MYSQL="$()"
MYSQLDUMP="$()"
GZIP="$()" I assume the first one is DB name. Here is the script:
#!/bin/sh #---------------------------------------------------------------- # Daily Backup Routine - Backup and Sync to Dropbox # This script creates a backup using today's date, then deleted # any backups made 3 days ago. If run every day it will ensure # you have a week's worth of backups of your MySQL databases and # zPanel web directories. # # Uses whatever Dropbox account is running on the server. # # Written by Richard Ferreira for the backup of zPanel websites. # Contact me - richard[at]beetle001.com #---------------------------------------------------------------- # # Before we get started, we should set some parameters. We'll need these for later. # The webserver's datafiles: WEBDIR="/var/zpanel/hostdata/" # Where do we want the backup to go? (SET THIS - IT'S A TEMP FOLDER) BACKUP="/root/backup-temp" # Where is our dropbox folder? (SET THIS TO YOUR ABSOLUTE BACKUP PATH) DROPBOX="/root/Dropbox/Backups # What do we want our date to look like? NOW=$(date +"%d-%m-%Y") # We need to know the date 3 days ago to purge any backups that were made 3 days ago. # This ensures we don't keep unnecessarily old backups. # It doesn't matter if it skips one every now and then - we'll just have to check manually from time to time. # If you want to keep more days backups, change the "3 days ago" to "x days ago" DAYSAGO=$(date --date="3 days ago" +"%d-%m-%Y") # What should our file backup look like? WEBFILE="webdirs-full-$NOW.tar.gz" # Our MySQL Login information and some paths (we'll use 'which' to make sure we get them): SQLUSER="root" # Don't forget to change the root password here! SQLPASS="xxxxxxxxxxxx" SQLHOST="localhost" MYSQL="$(db-name)" MYSQLDUMP="$(db-name)" GZIP="$(.gz)" # # Let's just, for sanity's sake, make sure our temp backup folder exists. mkdir $BACKUP # DON'T EDIT ANYTHING BELOW THIS LINE #---------------------------------------------------------------- # Now let's start! # Let's get the databases that we want to backup. DBS="$($MYSQL -u $SQLUSER -h $SQLHOST -p$SQLPASS -Bse 'show databases')" # Now let's dump them to .sql files and put them in our backup directory. for db in $DBS do FILE=$BACKUP/mysql-$db.$NOW.gz $MYSQLDUMP -u $SQLUSER -h $SQLHOST -p$SQLPASS $db | $GZIP -9 > $FILE done # # Let's shove the whole webserver directory into a tarball and throw that in with the sql files: tar -zcvf /root/backup/$WEBFILE $WEBDIR # That's all done - we should put the backups on Copy by putting them into our Copy folder. # First let's make a folder for today. mkdir $DROPBOX/$NOW # Copy our backups into it. cp -R $BACKUP/* $DROPBOX/Asterix/$NOW # We can delete the backup we made 3 days ago from Copy now. rm -rf $DROPBOX/$DAYSAGO # And clear out the temporary director for next time. rm $BACKUP/* # Job well done! # Have a beer and relax! Here is my problem: When I try to run the script I get this error: ./backup.sh: 66: ./backup.sh: Syntax error: Unterminated quoted string If anyone could help me with this would appreciate it a lot!