how does linux kernel implement shared memory between 2 processes
How does the Linux kernel implement the shared memory mechanism between different processes? To elaborate further, each process has its own address space. For example, an address of 0x1000 in Process A is a different location when compared to an address of 0x1000 in Process B. So how does the kernel ensure that a piece of memory is shared between different process, having different address spaces? Thanks in advance.
2 Answers 2
Interprocess Communication Mechanisms
Processes communicate with each other and with the kernel to coordinate their activities. Linux supports a number of Inter-Process Communication (IPC) mechanisms. Signals and pipes are two of them but Linux also supports the System V IPC mechanisms named after the Unix TM release in which they first appeared.
Signals are one of the oldest inter-process communication methods used by Unix TM systems. They are used to signal asynchronous events to one or more processes. A signal could be generated by a keyboard interrupt or an error condition such as the process attempting to access a non-existent location in its virtual memory. Signals are also used by the shells to signal job control commands to their child processes. There are a set of defined signals that the kernel can generate or that can be generated by other processes in the system, provided that they have the correct privileges. You can list a system’s set of signals using the kill command (kill -l).
The common Linux shells all allow redirection. For example
pipes the output from the ls command listing the directory’s files into the standard input of the pr command which paginates them. Finally the standard output from the pr command is piped into the standard input of the lpr command which prints the results on the default printer. Pipes then are unidirectional byte streams which connect the standard output from one process into the standard input of another process. Neither process is aware of this redirection and behaves just as it would normally. It is the shell which sets up these temporary pipes between the processes.
In Linux, a pipe is implemented using two file data structures which both point at the same temporary VFS inode which itself points at a physical page within memory. Figure shows that each file data structure contains pointers to different file operation routine vectors; one for writing to the pipe, the other for reading from the pipe.
- Message Queues: Message queues allow one or more processes to write messages, which will be read by one or more reading processes. Linux maintains a list of message queues, the msgque vector; each element of which points to a msqid_ds data structure that fully describes the message queue. When message queues are created a new msqid_ds data structure is allocated from system memory and inserted into the vector.
- System V IPC Mechanisms: Linux supports three types of interprocess communication mechanisms that first appeared in Unix TM System V (1983). These are message queues, semaphores and shared memory. These System V IPC mechanisms all share common authentication methods. Processes may access these resources only by passing a unique reference identifier to the kernel via system calls. Access to these System V IPC objects is checked using access permissions, much like accesses to files are checked. The access rights to the System V IPC object is set by the creator of the object via system calls. The object’s reference identifier is used by each mechanism as an index into a table of resources. It is not a straight forward index but requires some manipulation to generate the index.
- Semaphores: In its simplest form a semaphore is a location in memory whose value can be tested and set by more than one process. The test and set operation is, so far as each process is concerned, uninterruptible or atomic; once started nothing can stop it. The result of the test and set operation is the addition of the current value of the semaphore and the set value, which can be positive or negative. Depending on the result of the test and set operation one process may have to sleep until the semphore’s value is changed by another process. Semaphores can be used to implement critical regions, areas of critical code that only one process at a time should be executing.
Say you had many cooperating processes reading records from and writing records to a single data file. You would want that file access to be strictly coordinated. You could use a semaphore with an initial value of 1 and, around the file operating code, put two semaphore operations, the first to test and decrement the semaphore’s value and the second to test and increment it. The first process to access the file would try to decrement the semaphore’s value and it would succeed, the semaphore’s value now being 0. This process can now go ahead and use the data file but if another process wishing to use it now tries to decrement the semaphore’s value it would fail as the result would be -1. That process will be suspended until the first process has finished with the data file. When the first process has finished with the data file it will increment the semaphore’s value, making it 1 again. Now the waiting process can be woken and this time its attempt to increment the semaphore will succeed.
- Shared Memory: Shared memory allows one or more processes to communicate via memory that appears in all of their virtual address spaces. The pages of the virtual memory is referenced by page table entries in each of the sharing processes’ page tables. It does not have to be at the same address in all of the processes’ virtual memory. As with all System V IPC objects, access to shared memory areas is controlled via keys and access rights checking. Once the memory is being shared, there are no checks on how the processes are using it. They must rely on other mechanisms, for example System V semaphores, to synchronize access to the memory.
How to use shared memory with Linux in C
I have a bit of an issue with one of my projects. I have been trying to find a well documented example of using shared memory with fork() but to no success. Basically the scenario is that when the user starts the program, I need to store two values in shared memory: current_path which is a char* and a file_name which is also char*. Depending on the command arguments, a new process is kicked off with fork() and that process needs to read and modify the current_path variable stored in shared memory while the file_name variable is read only. Is there a good tutorial on shared memory with example code (if possible) that you can direct me to?
You may consider using threads instead of processes. Then the whole memory is shared with no further tricks.
The answers below discuss both the System V IPC mechanism, shmget() et al. and also the pure mmap() approach with MAP_ANON (aka MAP_ANONYMOUS ) — though MAP_ANON is not defined by POSIX. There is also POSIX shm_open() and shm_close() for managing shared memory objects. […continued…]
[…continuation…] These have the same advantage that the System V IPC shared memory has — the shared memory object can persist beyond the lifetime of the process that creates it (until some process executes shm_unlink() ), whereas mechanisms using mmap() require a file and MAP_SHARED to persist the data (and MAP_ANON precludes persistence). There’s a complete example in the Rationale section of the specification of shm_open() .
6 Answers 6
There are two approaches: shmget and mmap . I’ll talk about mmap , since it’s more modern and flexible, but you can take a look at man shmget (or this tutorial) if you’d rather use the old-style tools.
The mmap() function can be used to allocate memory buffers with highly customizable parameters to control access and permissions, and to back them with file-system storage if necessary.
The following function creates an in-memory buffer that a process can share with its children:
#include #include #include void* create_shared_memory(size_t size) < // Our memory buffer will be readable and writable: int protection = PROT_READ | PROT_WRITE; // The buffer will be shared (meaning other processes can access it), but // anonymous (meaning third-party processes cannot obtain an address for it), // so only this process and its children will be able to use it: int visibility = MAP_SHARED | MAP_ANONYMOUS; // The remaining parameters to `mmap()` are not important for this use case, // but the manpage for `mmap` explains their purpose. return mmap(NULL, size, protection, visibility, -1, 0); >The following is an example program that uses the function defined above to allocate a buffer. The parent process will write a message, fork, and then wait for its child to modify the buffer. Both processes can read and write the shared memory.
#include #include int main() < char parent_message[] = "hello"; // parent process will write this message char child_message[] = "goodbye"; // child process will then write this one void* shmem = create_shared_memory(128); memcpy(shmem, parent_message, sizeof(parent_message)); int pid = fork(); if (pid == 0) < printf("Child read: %s\n", shmem); memcpy(shmem, child_message, sizeof(child_message)); printf("Child wrote: %s\n", shmem); >else < printf("Parent read: %s\n", shmem); sleep(1); printf("After 1s, parent read: %s\n", shmem); >> This is why Linux is so frustrating for inexperienced devs. The man page doesn’t explain how to actually use it, and there is no sample code. 🙁
Haha I know what you mean, but it’s actually because we’re not used to reading manpages. When I learned to read them and got used to them, they became even more useful than lousy tutorials with particular demonstrations. I remember I got a 10/10 in my Operating Systems course using nothing but manpages for reference during the exam.
shmget is a really old-fashioned, and some would say deprecated, way to do shared memory. Better to use mmap and shm_open , plain files, or simply MAP_ANONYMOUS .
Well, this answer became popular for some reason, so I decided to make it worth the read. It only took 4 years
Here is an example for shared memory :
#include #include #include #include #include #include #define SHM_SIZE 1024 /* make it a 1K shared memory segment */ int main(int argc, char *argv[]) < key_t key; int shmid; char *data; int mode; if (argc >2) < fprintf(stderr, "usage: shmdemo [data_to_write]\n"); exit(1); >/* make the key: */ if ((key = ftok("hello.txt", 'R')) == -1) /*Here the file must exist */ < perror("ftok"); exit(1); >/* create the segment: */ if ((shmid = shmget(key, SHM_SIZE, 0644 | IPC_CREAT)) == -1) < perror("shmget"); exit(1); >/* attach to the segment to get a pointer to it: */ if ((data = shmat(shmid, NULL, 0)) == (void *)-1) < perror("shmat"); exit(1); >/* read or modify the segment, based on the command line: */ if (argc == 2) < printf("writing to segment: \"%s\"\n", argv[1]); strncpy(data, argv[1], SHM_SIZE); >else printf("segment contains: \"%s\"\n", data); /* detach from the segment: */ if (shmdt(data) == -1) < perror("shmdt"); exit(1); >return 0; > - Use ftok to convert a pathname and a project identifier to a System V IPC key
- Use shmget which allocates a shared memory segment
- Use shmat to attache the shared memory segment identified by shmid to the address space of the calling process
- Do the operations on the memory area
- Detach using shmdt
However this code doesn’t handle deletion of shared memory. After program exits, one have to delete it manually via ipcrm -m 0.
These are includes for using shared memory
#include #include int shmid; int shmkey = 12222;//u can choose it as your choice int main() < //now your main starting shmid = shmget(shmkey,1024,IPC_CREAT); // 1024 = your preferred size for share memory // IPC_CREAT its a flag to create shared memory //now attach a memory to this share memory char *shmpointer = shmat(shmid,NULL); //do your work with the shared memory //read -write will be done with the *shmppointer //after your work is done deattach the pointer shmdt(&shmpointer, NULL); try this code sample, I tested it, source: http://www.makelinux.net/alp/035
#include #include #include int main () < int segment_id; char* shared_memory; struct shmid_ds shmbuffer; int segment_size; const int shared_segment_size = 0x6400; /* Allocate a shared memory segment. */ segment_id = shmget (IPC_PRIVATE, shared_segment_size, IPC_CREAT | IPC_EXCL | S_IRUSR | S_IWUSR); /* Attach the shared memory segment. */ shared_memory = (char*) shmat (segment_id, 0, 0); printf ("shared memory attached at address %p\n", shared_memory); /* Determine the segment's size. */ shmctl (segment_id, IPC_STAT, &shmbuffer); segment_size = shmbuffer.shm_segsz; printf ("segment size: %d\n", segment_size); /* Write a string to the shared memory segment. */ sprintf (shared_memory, "Hello, world."); /* Detach the shared memory segment. */ shmdt (shared_memory); /* Reattach the shared memory segment, at a different address. */ shared_memory = (char*) shmat (segment_id, (void*) 0x5000000, 0); printf ("shared memory reattached at address %p\n", shared_memory); /* Print out the string from shared memory. */ printf ("%s\n", shared_memory); /* Detach the shared memory segment. */ shmdt (shared_memory); /* Deallocate the shared memory segment. */ shmctl (segment_id, IPC_RMID, 0); return 0; >This is good code, except I don't think it shows how to access the shared-memory segment by a client (by using shmget and shmat from a different process), which is kind of the whole point of shared memory. =(
#include #include #include #include /* * pvtmMmapAlloc - creates a memory mapped file area. * The return value is a page-aligned memory value, or NULL if there is a failure. * Here's the list of arguments: * @mmapFileName - the name of the memory mapped file * @size - the size of the memory mapped file (should be a multiple of the system page for best performance) * @create - determines whether or not the area should be created. */ void* pvtmMmapAlloc (char * mmapFileName, size_t size, char create) < void * retv = NULL; if (create) < mode_t origMask = umask(0); int mmapFd = open(mmapFileName, O_CREAT|O_RDWR, 00666); umask(origMask); if (mmapFd < 0) < perror("open mmapFd failed"); return NULL; >if ((ftruncate(mmapFd, size) == 0)) < int result = lseek(mmapFd, size - 1, SEEK_SET); if (result == -1) < perror("lseek mmapFd failed"); close(mmapFd); return NULL; >/* Something needs to be written at the end of the file to * have the file actually have the new size. * Just writing an empty string at the current file position will do. * Note: * - The current position in the file is at the end of the stretched * file due to the call to lseek(). * - The current position in the file is at the end of the stretched * file due to the call to lseek(). * - An empty string is actually a single '\0' character, so a zero-byte * will be written at the last byte of the file. */ result = write(mmapFd, "", 1); if (result != 1) < perror("write mmapFd failed"); close(mmapFd); return NULL; >retv = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, mmapFd, 0); if (retv == MAP_FAILED || retv == NULL) < perror("mmap"); close(mmapFd); return NULL; >> > else < int mmapFd = open(mmapFileName, O_RDWR, 00666); if (mmapFd < 0) < return NULL; >int result = lseek(mmapFd, 0, SEEK_END); if (result == -1) < perror("lseek mmapFd failed"); close(mmapFd); return NULL; >if (result == 0) < perror("The file has 0 bytes"); close(mmapFd); return NULL; >retv = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, mmapFd, 0); if (retv == MAP_FAILED || retv == NULL) < perror("mmap"); close(mmapFd); return NULL; >close(mmapFd); > return retv; >