How to use write() or fwrite() for writing data to terminal (stdout)?
I am trying to speed up my C program to spit out data faster.
Currently I am using printf() to give some data to the outside world. It is a continuous stream of data, therefore I am unable to use return(data). How can I use write() or fwrite() to give the data out to the console instead of file? Overall my setup consist of program written in C and its output goes to the python script, where the data is processed further. I form a pipe:
./program_in_c | script_in_python 3 Answers 3
#include ssize_t write(int fd, const void *buf, size_t count); write() writes up to count bytes from the buffer starting at buf to the file referred to by the file descriptor fd.
the standard output file descriptor is: 1 in linux at least! concern using flush the stdoutput buffer as well, before calling to write system call to ensure that all previous garabge was cleaned
fflush(stdout); // Will now print everything in the stdout buffer write(1, buf, count); size_t fwrite(const void *ptr, size_t size, size_t nmemb, FILE *stream); The function fwrite() writes nmemb items of data, each size bytes long, to the stream pointed to by stream, obtaining them from the location given by ptr.
fflush(stdout); int buf[8]; fwrite(buf, sizeof(int), sizeof(buf), stdout); Please refare to man pages for further reading, in the links below:
@ryyker- sure! fixing the answer.i have changed it to int cos i want to show the use of sizeof in this example. thanks
Calling write and then fflush(stdout) is the wrong way around. write always writes immediately without buffering. And stdout doesn’t know about it. So you might want to write whatever’s in the buffer (by calling fflush) before you call write , if you’ve been using stdout as well.
@LuisColorado — thanks you i have moved the fflush before for ensuring that all the previous garbage data in the buffer was cleaned before the write
Well, there’s little or no win in trying to overcome the already used buffering system of the stdio.h package. If you try to use fwrite() with larger buffers, you’ll probably win no more time, and use more memory than is necessary, as stdio.h selects the best buffer size appropiate to the filesystem where the data is to be written.
A simple program like the following will show that speed is of no concern, as stdio is already buffering output.
#include int main() < int c; while((c = getchar()) >= 0) putchar(c); > If you try the above and below programs:
#include #include #include int main() < char buffer[512]; int n; while((n = read(0, buffer, sizeof buffer)) >0) write(1, buffer, n); if (n < 0) < perror("read"); return EXIT_FAILURE; >return EXIT_SUCCESS; > You will see that there’s no significative difference or, even, the first program will be faster, despite it is doing I/O on a per character basis. (as B. Kernighan & Dennis Ritchie wrote it in her first edition of «The C programming language») Most probably the first program will win.
The calls to read() and write() involve a system call each, with a buffer size decided by you. The individual getchar() and putchar() calls don’t. They just store the received chars in a memory buffer, as you print them, whose size has been decided by the stdio.h library implementation, based on the filesystem, and it flushes the buffer, once it is full of data. If you grow the buffer size in the second program, you’ll see that you get better results increasing it up to a point, but after that you’ll see no more increment in speed. The number of calls made to the library is insignificant with respect to the time involved in doing the actual I/O, and selecting a very large buffer, will eat much memory from your system (and a Raspberry Pi memory is limited in this sense, to 1Gb or ram) If you end making swap due to a so large buffer, you’ll lose the battle completely.
Most filesystems have a preferred buffer size, because the kernel does write ahead (the kernel reads more than what you asked for, on sequential reads, in prevision that you’ll continue reading more after you consumed the data) and this affects the optimum buffer size. For that, the stat(2) system call tells you what is the optimum buffer size, and stdio uses that when it selects the actual buffer size.
Don’t think you are going to get better (or much better) than the program listed first above. Even if you use large enough buffers.
What is not correct (or valid) is to intermix calls that do buffering (like all the stdio package’s) with basic system calls (like read(2) or write(2) —as I’ve seen recommending you to use fflush(3) after write(2) , which is totally incoherent— that do not buffer the data) there’s no earn (and probably you’ll get your output incorrectly ordered, if you do part of the calls using printf(3) and part using write(2) (this happens more in pipelines like you plan to do, because the buffers are not line oriented —another characteristic of buffered output in stdio—)
Finally, I recomend you to read «The Unix programming environment» by Dennis Ritchie and Rob Pike. It will teach you a lot of unix, but one very good thing is that it will teach you to use perfectly the stdio package and the unix filesystem calls for reading and writing. With a little of luck you’ll find it in .pdf on internet.
The next program shows you the effect of buffering:
#include #include #include int main() < int i; char *sep = ""; for (i = 0; i < 10; i++) < printf("%s%d", sep, i); sep = ", "; sleep(1); >printf("\n"); > One would assume you are going to see (on the terminal) the program, writing the numbers 0 to 9 , separated by , and paced on one second intervals.
But due to the buffering, what you observe is quite different, you’ll see how your program waits for 10 seconds without writing anything at all on the terminal, and at the end, writes everything in one shot, including the final line end, when the program terminates, and the shell shows you the prompt again.
If you change the program to this:
#include #include #include int main() < int i; char *sep = ""; for (i = 0; i < 10; i++) < printf("%s%d", sep, i); fflush(stdout); sep = ", "; sleep(1); >printf("\n"); > You’ll see the expected output, because you have told stdio to flush the buffer at each loop pass. In both programs you did 10 calls to printf(3) , but there was only one write(2) at the end to write the full buffer. In the second version you forced stdio to do one such write(2) after each printf, and that showed the data out as the program passed through the loop.
Be careful, because another characteristic of stdio can be confounding you, as printf(3) , when you print to a terminal device, flushes the output at each \n , but when you run it through a pipe, it does it only when the buffer fills up completely. This saves system calls (in FreeBSD, for example, the buffer size selected by stdio is around 32kb, large enough to force two blocks to write(2) and optimum (you’ll not get better going above that size)
How can I make the system call write() print to the screen?
For my OS class I’m supposed to implement Linux’s cat using only system calls (no printf) Reading this reference I found it being used to print to a file. I guess I should manipulate ofstream. In the example appears: ofstream outfile («new.txt»,ofstream::binary); How can I make it write to the screen? EDIT: I realized this write() is part of iostream library, is this the same as the int write (int fd, char *buf , int size) system call?
And don’t forget that Linux cat is GNU cat and sports 11 non-ignored command line options ;-).linux.die.net/man/1/cat
You’ve tagged this as C but you’re talking about and have linked to C++ classes. Is this supposed to be C, or C++? Re your edit — I wouldn’t consider iostream to be allowed, given «only system calls», only things like man 2 write
5 Answers 5
A system call is a service provided by Linux kernel. In C programming, functions are defined in libc which provide a wrapper for many system calls. The function call write() is one of these system calls.
The first argument passed to write() is the file descriptor to write to. The symbolic constants STDERR_FILENO , STDIN_FILENO , and STDOUT_FILENO are respectively defined to 2, 0, and 1 in unidtd.h. You want to write to either STDOUT_FILENO or STDERR_FILENO.
const char msg[] = "Hello World!"; write(STDOUT_FILENO, msg, sizeof(msg)-1);You can alternatively use the syscall() function to perform an indirrect system call by specifying the function number defined in syscall.h or unistd.h. Using this method, you can guarantee that you are only using system calls. You may find The Linux System Call Quick Refernence (PDF Link) to be helpful.
/* 4 is the system call number for write() */ const char msg[] = "Hello World!"; syscall(4, STDOUT_FILENO, msg, sizeof(msg)-1);What is the simplest way to write to stdout in binary mode?
I’ve been trying to figure out the best way to write binary data to stdout from a C program. It works fine on Linux, but I’m having issues when I compile on Windows because «\n» gets converted to «\r\n». Is there a standard way to write to stdout in some sort of binary mode which avoids newline conversion? If not, what is the simplest way to get Windows to stop doing this? I’m using GCC and MinGW, and writing to stdout using fwrite .
2 Answers 2
You can use setmode(fileno(stdout), O_BINARY)
Wrap it in an ifdef if you want to keep it compatible with Linux.
freopen(NULL, «wb», stdout) can also be used, and is portable C. However, it may have unwanted truncation effects; see the Application Usage section of pubs.opengroup.org/onlinepubs/9699919799/functions/freopen.html. For what it’s worth, I think this text is wrong; when the new filename is NULL, there is no underlying open operation and thus the text about O_TRUNC does not apply.
@R.. you cant use freopen with a NULL filename in Windows, it will call the invalid parameter handler according to this: msdn.microsoft.com/en-us/library/wk2h68td.aspx and the default behavior of the invalid parameter handler is to crash the application.
The link you posted states that is implementation-defined to allow streams to be reopened. My original point is that this is not a portable way to set the stream mode to binary.
What does it mean to write to stdout in C?
Does a program that writes to «stdout» write to a file? the screen? I don’t understand what it means to write to stdout.
It means it writes to file descriptor 1. stdout is like a constant variable name, something that everyone uses so you don’t have to remember that the actual FD number is 1. For example, when I first wrote this, I thought it was FD 0. 🙂
7 Answers 7
That means that you are printing output on the main output device for the session. whatever that may be. The user’s console, a tty session, a file or who knows what. What that device may be varies depending on how the program is being run and from where.
The following command will write to the standard output device (stdout).
Which is just another way, in essence, of doing this.
fprintf( stdout, "hello world\n" ); In which case stdout is a pointer to a FILE stream that represents the default output device for the application. You could also use
fprintf( stderr, "that didn't go well\n" ); in which case you would be sending the output to the standard error output device for the application which may, or may not, be the same as stdout — as with stdout , stderr is a pointer to a FILE stream representing the default output device for error messages.
Is there an example in C of the command to write to stdout? Is a simple printf statement write to stdout? What about writing to a file with write()?
printf by default writes on stdout , if you want to write to a specific stream you should use fprintf which accepts a FILE* as the destination stream.
Also it’s «std» out because it’s called «standard» output. As opposed to stdin or «standard input», stderr for «standard» error.
I wrote an addendum on how to force a flush of the buffer with fflush(stdout) here: stackoverflow.com/a/48551850/4561887
When you commit to sending output to stdout , you’re basically leaving it up to the user to decide where that output should go.
If you use printf(. ) (or the equivalent fprintf(stdout, . ) ), you’re sending the output to stdout , but where that actually ends up can depend on how I invoke your program.
If I launch your program from my console like this, I’ll see output on my console:
However, I might launch the program like this, producing no output on the console:
but I would now have a file «hello.txt» with the text «Hello, World!» inside, thanks to the shell’s redirection feature.
Who knows – I might even hook up some other device and the output could go there. The point is that when you decide to print to stdout (e.g. by using printf() ), then you won’t exactly know where it will go until you see how the process is launched or used.