How do I execute a command and get the output of the command within C++ using POSIX?
I am looking for a way to get the output of a command when it is run from within a C++ program. I have looked at using the system() function, but that will just execute a command. Here’s an example of what I’m looking for:
std::string result = system("./some_command"); I need to run an arbitrary command and get its output. I’ve looked at boost.org, but I have not found anything that will give me what I need.
Also see answers in this question: https://stackoverflow.com/questions/52164723/how-to-execute-a-command-and-get-return-code-stdout-and-stderr-of-command-in-c for an extension of the great answer below that provides methods to get the return code and stderr as well as stdout that this answer already explains
12 Answers 12
#include #include #include #include #include #include std::string exec(const char* cmd) < std::arraybuffer; std::string result; std::unique_ptr pipe(popen(cmd, "r"), pclose); if (!pipe) < throw std::runtime_error("popen() failed!"); >while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr) < result += buffer.data(); >return result; > #include #include #include #include std::string exec(const char* cmd) < char buffer[128]; std::string result = ""; FILE* pipe = popen(cmd, "r"); if (!pipe) throw std::runtime_error("popen() failed!"); try < while (fgets(buffer, sizeof buffer, pipe) != NULL) < result += buffer; >> catch (. ) < pclose(pipe); throw; >pclose(pipe); return result; > Replace popen and pclose with _popen and _pclose for Windows.
Also be aware that an exception can occur in result += buffer , so the pipe might not be properly closed.
Getting both stdout and stderr (and also writing to stdin, not shown here) is easy peasy with my pstreams header, which defines iostream classes that work like popen :
#include #include #include int main() < // run a process and create a streambuf that reads its stdout and stderr redi::ipstream proc("./some_command", redi::pstreams::pstdout | redi::pstreams::pstderr); std::string line; // read child's stdout while (std::getline(proc.out(), line)) std::cout
I disagree. popen requires you to use the C stdio API, I prefer the iostreams API. popen requires you to manually clean up the FILE handle, pstreams do that automatically. popen only accepts a const char* for the argument, which requires care to avoid shell injection attacks, pstreams allows you to pass a vector of strings similar to execv , which is safer. popen gives you nothing but a pipe, pstreams tells you the child's PID allowing you to send signals e.g. to kill it if it's blocked or not exiting. All of those are advantages even if you only want unidirectional IO.
Another issue with this solution is if the child writes to stderr enough to fill the buffers and block before it starts writing to stdout. The parent will block reading stdout, while the child is blocked waiting for stderr to be read. resource deadlock! At least one of those loops would be better as asynchronous (i.e., threaded).
@JesseChisholm, yes, that could be a problem. But you don't need to use threads because pstreams allows an approximation of non-blocking I/O using the iostream interface, specifically using the readsome function, which checks for readiness using pstreambuf::in_avail() , so won't block. That allows demultiplexing on the process' stdout and stderr as each has data available. pstreambuf::in_avail() only works 100% reliably if the OS supports the non-standard FIONREAD ioctl, but that is supported on (at least) GNU/Linux and Solaris.
For Windows, popen also works, but it opens up a console window - which quickly flashes over your UI application. If you want to be a professional, it's better to disable this "flashing" (especially if the end-user can cancel it).
So here is my own version for Windows:
(This code is partially recombined from ideas written in The Code Project and MSDN samples.)
#include #include // // Execute a command and get the results. (Only standard output) // CStringA ExecCmd( const wchar_t* cmd // [in] command to execute ) < CStringA strResult; HANDLE hPipeRead, hPipeWrite; SECURITY_ATTRIBUTES saAttr = ; saAttr.bInheritHandle = TRUE; // Pipe handles are inherited by child process. saAttr.lpSecurityDescriptor = NULL; // Create a pipe to get results from child's stdout. if (!CreatePipe(&hPipeRead, &hPipeWrite, &saAttr, 0)) return strResult; STARTUPINFOW si = ; si.dwFlags = STARTF_USESHOWWINDOW | STARTF_USESTDHANDLES; si.hStdOutput = hPipeWrite; si.hStdError = hPipeWrite; si.wShowWindow = SW_HIDE; // Prevents cmd window from flashing. // Requires STARTF_USESHOWWINDOW in dwFlags. PROCESS_INFORMATION pi = < 0 >; BOOL fSuccess = CreateProcessW(NULL, (LPWSTR)cmd, NULL, NULL, TRUE, CREATE_NEW_CONSOLE, NULL, NULL, &si, &pi); if (! fSuccess) < CloseHandle(hPipeWrite); CloseHandle(hPipeRead); return strResult; >bool bProcessEnded = false; for (; !bProcessEnded ;) < // Give some timeslice (50 ms), so we won't waste 100% CPU. bProcessEnded = WaitForSingleObject( pi.hProcess, 50) == WAIT_OBJECT_0; // Even if process exited - we continue reading, if // there is some data available over pipe. for (;;) < char buf[1024]; DWORD dwRead = 0; DWORD dwAvail = 0; if (. PeekNamedPipe(hPipeRead, NULL, 0, NULL, &dwAvail, NULL)) break; if (!dwAvail) // No data available, return break; if (. ReadFile(hPipeRead, buf, min(sizeof(buf) - 1, dwAvail), &dwRead, NULL) || !dwRead) // Error, the child process might ended break; buf[dwRead] = 0; strResult += buf; >> //for CloseHandle(hPipeWrite); CloseHandle(hPipeRead); CloseHandle(pi.hProcess); CloseHandle(pi.hThread); return strResult; > //ExecCmd
This is my favourite solution for Windows, I hope you forgive my changes. I'd suggest to make the const-cast more explicit, whereas I consider the explicit usage of wchar_t and CreateProcessW as an unnecessary restriction.
Do you see any problem or potential problem with this cast ? I prefer to keep code at minimum and don't write it without need.
After reading CreateProcess function (Windows), I see a real danger in doing this: The Unicode version of this function, CreateProcessW, can modify the contents of this string. Therefore, this parameter cannot be a pointer to read-only memory (such as a const variable or a literal string). If this parameter is a constant string, the function may cause an access violation. So it's maybe better to copy the command line into a separate buffer first, to prevent the caller from getting its original input changed.
I'd use popen() (++waqas).
But sometimes you need reading and writing.
It seems like nobody does things the hard way any more.
(Assuming a Unix/Linux/Mac environment, or perhaps Windows with a POSIX compatibility layer. )
enum PIPE_FILE_DESCRIPTERS < READ_FD = 0, WRITE_FD = 1 >; enum CONSTANTS < BUFFER_SIZE = 100 >; int main() < int parentToChild[2]; int childToParent[2]; pid_t pid; string dataReadFromChild; char buffer[BUFFER_SIZE + 1]; ssize_t readResult; int status; ASSERT_IS(0, pipe(parentToChild)); ASSERT_IS(0, pipe(childToParent)); switch (pid = fork()) < case -1: FAIL("Fork failed"); exit(-1); case 0: /* Child */ ASSERT_NOT(-1, dup2(parentToChild[READ_FD], STDIN_FILENO)); ASSERT_NOT(-1, dup2(childToParent[WRITE_FD], STDOUT_FILENO)); ASSERT_NOT(-1, dup2(childToParent[WRITE_FD], STDERR_FILENO)); ASSERT_IS(0, close(parentToChild [WRITE_FD])); ASSERT_IS(0, close(childToParent [READ_FD])); /* file, arg0, arg1, arg2 */ execlp("ls", "ls", "-al", "--color"); FAIL("This line should never be reached. "); exit(-1); default: /* Parent */ cout else < FAIL("read() failed"); exit(-1); >default: dataReadFromChild . append(buffer, readResult); break; > > /* while (true) */ > /* switch (pid = fork())*/ >
You also might want to play around with select() and non-blocking reads.
fd_set readfds; struct timeval timeout; timeout.tv_sec = 0; /* Seconds */ timeout.tv_usec = 1000; /* Microseconds */ FD_ZERO(&readfds); FD_SET(childToParent[READ_FD], &readfds); switch (select (1 + childToParent[READ_FD], &readfds, (fd_set*)NULL, (fd_set*)NULL, & timeout)) < case 0: /* Timeout expired */ break; case -1: if ((errno == EINTR) || (errno == EAGAIN)) < errno = 0; break; >else < FAIL("Select() Failed"); exit(-1); >case 1: /* We have input */ readResult = read(childToParent[READ_FD], buffer, BUFFER_SIZE); // However you want to handle it. break; default: FAIL("How did we see input on more than one file descriptor?"); exit(-1); >
The hard way is right 🙂 I like the idea with select() call, though in this case, I actually need to wait until the task completes. I'll keep this code for another project I have 🙂
Your execlp call has a bug: the last arg pointer passed must be (char *) NULL to properly terminate the variadic argument list (see execlp(3) for reference).
- I don't think popen() is part of the C++ standard (it's part of POSIX from memory), but it's available on every UNIX I've worked with (and you seem to be targeting UNIX since your command is ./some_command ).
- On the off-chance that there is no popen() , you can use system("./some_command >/tmp/some_command.out"); , then use the normal I/O functions to process the output file.
I couldn't figure out why popen / pclose is missing from Code::Blocks/MinGW. So I worked around the problem by using CreateProcess() and CreatePipe() instead.
Here's the solution that worked for me:
//C++11 #include #include #include #include #include #include #include using namespace std; int SystemCapture( string CmdLine, //Command Line string CmdRunDir, //set to '.' for current directory string& ListStdOut, //Return List of StdOut string& ListStdErr, //Return List of StdErr uint32_t& RetCode) //Return Exit Code < int Success; SECURITY_ATTRIBUTES security_attributes; HANDLE stdout_rd = INVALID_HANDLE_VALUE; HANDLE stdout_wr = INVALID_HANDLE_VALUE; HANDLE stderr_rd = INVALID_HANDLE_VALUE; HANDLE stderr_wr = INVALID_HANDLE_VALUE; PROCESS_INFORMATION process_info; STARTUPINFO startup_info; thread stdout_thread; thread stderr_thread; security_attributes.nLength = sizeof(SECURITY_ATTRIBUTES); security_attributes.bInheritHandle = TRUE; security_attributes.lpSecurityDescriptor = nullptr; if (!CreatePipe(&stdout_rd, &stdout_wr, &security_attributes, 0) || !SetHandleInformation(stdout_rd, HANDLE_FLAG_INHERIT, 0)) < return -1; >if (!CreatePipe(&stderr_rd, &stderr_wr, &security_attributes, 0) || !SetHandleInformation(stderr_rd, HANDLE_FLAG_INHERIT, 0)) < if (stdout_rd != INVALID_HANDLE_VALUE) CloseHandle(stdout_rd); if (stdout_wr != INVALID_HANDLE_VALUE) CloseHandle(stdout_wr); return -2; >ZeroMemory(&process_info, sizeof(PROCESS_INFORMATION)); ZeroMemory(&startup_info, sizeof(STARTUPINFO)); startup_info.cb = sizeof(STARTUPINFO); startup_info.hStdInput = 0; startup_info.hStdOutput = stdout_wr; startup_info.hStdError = stderr_wr; if(stdout_rd || stderr_rd) startup_info.dwFlags |= STARTF_USESTDHANDLES; // Make a copy because CreateProcess needs to modify string buffer char CmdLineStr[MAX_PATH]; strncpy(CmdLineStr, CmdLine.c_str(), MAX_PATH); CmdLineStr[MAX_PATH-1] = 0; Success = CreateProcess( nullptr, CmdLineStr, nullptr, nullptr, TRUE, 0, nullptr, CmdRunDir.c_str(), &startup_info, &process_info ); CloseHandle(stdout_wr); CloseHandle(stderr_wr); if(!Success) < CloseHandle(process_info.hProcess); CloseHandle(process_info.hThread); CloseHandle(stdout_rd); CloseHandle(stderr_rd); return -4; >else < CloseHandle(process_info.hThread); >if(stdout_rd) < stdout_thread=thread([&]() < DWORD n; const size_t bufsize = 1000; char buffer [bufsize]; for(;;) < n = 0; int Success = ReadFile( stdout_rd, buffer, (DWORD)bufsize, &n, nullptr ); printf("STDERR: Success:%d n:%d\n", Success, (int)n); if(!Success || n == 0) break; string s(buffer, n); printf("STDOUT:(%s)\n", s.c_str()); ListStdOut += s; >printf("STDOUT:BREAK!\n"); >); > if(stderr_rd) < stderr_thread=thread([&]() < DWORD n; const size_t bufsize = 1000; char buffer [bufsize]; for(;;) < n = 0; int Success = ReadFile( stderr_rd, buffer, (DWORD)bufsize, &n, nullptr ); printf("STDERR: Success:%d n:%d\n", Success, (int)n); if(!Success || n == 0) break; string s(buffer, n); printf("STDERR:(%s)\n", s.c_str()); ListStdErr += s; >printf("STDERR:BREAK!\n"); >); > WaitForSingleObject(process_info.hProcess, INFINITE); if(!GetExitCodeProcess(process_info.hProcess, (DWORD*) &RetCode)) RetCode = -1; CloseHandle(process_info.hProcess); if(stdout_thread.joinable()) stdout_thread.join(); if(stderr_thread.joinable()) stderr_thread.join(); CloseHandle(stdout_rd); CloseHandle(stderr_rd); return 0; > int main() < int rc; uint32_t RetCode; string ListStdOut; string ListStdErr; cout cout How to get output of a bash command in a variable
I have been following through some of the similar questions (such as How to set a variable to the output from a command in Bash?), however the accepted answers seem to be non-working for me. I wasn't sure whether I ought to derail someone else's question or post my own duplicate, so apologies if I chose wrong here. I wish to get the output and exit status of a number of commands in a script I am putting together. Here is an example of what I have been using:
cmd_output=$(rm $file) exit_status=$? if [ "$" -eq 0 ] then log "Successfully removed the original" $ else fail "Failed to remove the original, the output was: \n $" fi
# Usage: fail "Failure message" function fail < echo "FATAL ERROR: $1" >> "$/$" exit 1 > # Usage: log "Log message" 3 Where the tab-level is 3. function log < if (("$" > 0)) then eval "printf ' %.0s' " >> "$/$" fi echo "$1" >> "$/$" return 0 >
In the example above I use the $(cmd) format, but I have also tried using backticks. In my log file, all I see when there is a failure is:
Also, the output of the failed commands ends up on screen as per usual. Is there a common reason that my cmd_output variables would be remaining empty?
3 Answers 3
You have to include the output of the special standard error output stream:
There are three default streams on every program (that are numbered file descriptors):
0. Standard input (where the program normally reads from) 1. Standard output (where the program normally writes to) 2. Standard error (where the program normally writes error messages)
So to capture the error messages, we must redirect standard error output (stderr) into normal standard output (stdout) which will then be captured by the $(. ) expression.
The syntax for redirection is through the > "operator". Immediately before it you tell which file descriptor to redirect (the default is 1, which is stdout). And you can specify it to redirect to a file. If you write an ampersand ( & ) after it, you force it to redirect into another file descriptor. Therefore, in this example, we redirect file descriptor 2 (stderr) into file descriptor 1 (stdout).
Another observation is that it is probably good practice to place your $file variable between double quotes, in case it has white space characters.
get command output in pipe, C for Linux
I need to run a Linux CLI command and get its stdout output from C. I can use pipe() to create a pipe, then fork/exec, redirecting child's stdout descriptor into the pipe before calling exec(), and reading from the pipe in parent. Plus I'll need to wait on the child. Is there a simple call to do fork + redirect + exec + wait, like system() does fork + exec + wait, only system() doesn't do the redirect. There's popen(), which does fork + redirect + exec, but doesn't do wait, so I can't get exit status.
4 Answers 4
NAME popen, pclose - process I/O SYNOPSIS #include FILE *popen(const char *command, const char *type); int pclose(FILE *stream); DESCRIPTION The popen() function opens a process by creating a pipe, forking, and invoking the shell. Since a pipe is by definition unidirectional, the type argument may specify only reading or writing, not both; the resulting stream is correspondingly read-only or write-only. The command argument is a pointer to a null-terminated string containing a shell command line. This command is passed to /bin/sh using the -c flag; interpretation, if any, is performed by the shell. The type argument is a pointer to a null-terminated string which must be either ‘r’ for reading or ‘w’ for writing. The return value from popen() is a normal standard I/O stream in all respects save that it must be closed with pclose() rather than fclose(). Writing to such a stream writes to the standard input of the command; the command’s standard output is the same as that of the process that called popen(), unless this is altered by the command itself. Conversely, reading from a ‘‘popened’’ stream reads the command’s standard output, and the command’s standard input is the same as that of the process that called popen(). Note that output popen() streams are fully buffered by default. The pclose() function waits for the associated process to terminate and returns the exit status of the command as returned by wait4().
pclose() returns it with a caveat: Failure to execute the shell is indistinguishable from the shell’s failure to execute command, or an immediate exit of the command. The only hint is an exit status of 127. So if this is not enough, you have to write your own, as dmckee says