Lecture 04 Unix IO - system programming PDF

Preview

Summary

system programming...

Description

Lecture 4

UNIX I/O

BIL 244 – System Programming

Chapter 4 of Robbins Book

Device Terminology • Unix uses device independence. • I/O is done through device drivers that have a standard interface. • 5 main system calls for I/O: • • • • •

open close read write ioctl

• These calls return -1 on error and set errno •

UNIX I/O is done with file descriptors. Normally, when your program starts, there are 3 file descriptors open: STDIN_FILENO STDOUT_FILENO STDERR_FILENO

BIL 244 – System Programming

Reading and Writing • Unix provides sequential access to files and other devices through read and write functions #include ssize_t read(int fildes, void *buf, size_t nbytes);

• The read function attempts to retreive nbytes from the file (or device) fildes into user variable buf • If succesful returns the number of bytes actually read. If unsuccesful, returns -1 and sets errno BIL 244 – System Programming

Reading and Writing • The write function attemps to output nbytes from the user buffer buf to the file represented by file descriptor fildes #include ssize_t write(int fildes, const void *buf, size_t nbyte);

• It is not an error if this returns a value greater than 0 and less than nbyte • You must restart the write if it returns fewer bytes than requested. • a return value of -1 with errno set to EINTR is not usually an error. BIL 244 – System Programming

#include #include #define BLKSIZE 1024 int copyfile(int fromfd, int tofd) { char *bp; char buf[BLKSIZE]; int bytesread; int byteswritten = 0; int totalbytes = 0; for ( ; ; ) { while (((bytesread = read(fromfd, buf, BLKSIZE)) == -1) && (errno == EINTR)) ; /* handle interruption by signal if (bytesread 0) { while(((byteswritten = write(tofd, bp, bytesread)) == -1 ) && /* handle interruption by signal (errno == EINTR)) ; if (byteswritten < 0) /* real error on tofd break; totalbytes += byteswritten; bytesread -= byteswritten; bp += byteswritten; } /* real error on tofd if (byteswritten == -1) break; } return totalbytes; }

*/ */

*/ */

*/

#include #include #include #include



#define READ_FLAGS O_RDONLY #define WRITE_FLAGS (O_WRONLY | O_CREAT | O_EXCL) #define WRITE_PERMS (S_IRUSR | S_IWUSR) /* function definitions */ int copyfile(int fromfd, int tofd); int main(int argc, char *argv[]) { int bytes; int fromfd, tofd; if (argc != 3) { fprintf(stderr, "Usage: %s from_file to_file\n", argv[0]); return 1; } if ((fromfd = open(argv[1], READ_FLAGS)) == -1) { perror("Failed to open input file"); return 1; } if ((tofd = open(argv[2], WRITE_FLAGS, WRITE_PERMS)) == -1) { perror("Failed to create output file"); return 1; } bytes = copyfile(fromfd, tofd); printf("%d bytes copied from %s to %s\n", bytes, argv[1], argv[2]); return 0; /* the return closes the files */ }

Opening and Closing Files • The open function associates a file descriptor with a file (or physical device). The path parameter of open points to the pathname of the file and the oflag parametre specifies status flags and access modes for the opened file #include #include int open(const char *path, int oflag); int open(const char *path, int oflag, mode_t mode);

BIL 244 – System Programming

Opening files • Possible values of the flag include: O_RDONLY: read only O_WRONLY: write only O_RDWR: read and write O_APPEND: writes always write to end O_CREAT: create the file if it does not exist O_EXCL: used with O_CREAT, return an error if file exists O_NOCTTY: do not become a controlling terminal O_NONBLOCK: do not block if not ready to open, also affects reads and writes O_TRUNC: discard previous contents BIL 244 – System Programming

Opening Files • You must use the 3-parameter form of open if the O_CREAT flag is used. This specifies permissions.

• POSIX defines symbolic names for masks corresponding to the permission bits you specify file permisions independently of the implementation (these names are defined in sys/stat.h) BIL 244 – System Programming

Closing Files • The close function has a single parameter fildes, representing the open file whose resources are to be relased #include int close(int fildes);

• If successful close returns 0, if unsuccessfull returns -1 and sets errno • Open files are closed when your program exits normally BIL 244 – System Programming

The select function • The handling of I/O from multiple resources is an important problem that arises in many different forms • What happens when a program expects input from different sources but does not know which input is available first ?? If the program tries to read from source A, and infact input was only available from source B, the program blocks. • To solve this problem we need to block until input from either source become available • Blocking until at least one member of a set of conditions becomes true is called “OR syncronization” BIL 244 – System Programming

One method of monitoring multiple file descriptors is to use a separete process for each one as in the following example : #include #include #include #include #include #include



“CopyFileHeader.h“

/* put copy file function described on the previous example here */

int main(int argc, char *argv[]) { int bytesread; int childpid; int fd, fd1, fd2; if (argc != 3) { fprintf(stderr, "Usage: %s file1 file2\n", argv[0]); return 1; } if ((fd1 = open(argv[1], O_RDONLY)) == -1) { fprintf(stderr, "Failed to open file %s:%s\n", argv[1], strerror(errno)); return 1; } if ((fd2 = open(argv[2], O_RDONLY)) == -1) { fprintf(stderr, "Failed to open file %s:%s\n", argv[2], strerror(errno)); return 1; } if ((childpid = fork()) == -1) { perror("Failed to create child process"); return 1; } /* parent code */ if (childpid > 0) fd = fd1; else fd = fd2; /* child code */ bytesread = copyfile(fd, STDOUT_FILENO); /* both processes work here */ fprintf(stderr, "Bytes read: %d\n", bytesread); return 0; }

Problem ?? • Using two separate processes to monitor two file descriptors can be usefull. But how do we interact them ?? • Answer: we need to set up a from of communication between the two processes (interprocess communication) before creating the child. For example the parent process could create a pipe and the child could send its byte count to the pipe when it has finished.

BIL 244 – System Programming

The select function • The select call provides a method of monitoring file descriptors from a single process. It can monitor for three possible conditions – A read can be done without blocking, – A write can be done without blocking, – A file descriptor has error conditions waiting

• The nfds parameter gives the range of the file descriptors to be monitored (the value of nfds must be at least one greater than the largest file descriptor to be checked) • The readfds specifies the set of descriptors to be monitores for reading similarly the writefds specifies the set of descriptors to be monitored for writing and errorfds specifies the file descriptors to be monitored for error conditions BIL 244 – System Programming

The select function • The descriptor sets are of type fd_set. Any of these parameters may be NULL, in these cases select does not monitor the descriptor for the corresponding event • The last parameter is a timeout value that forces a return form select after a certain period of time has elapsed, even if no descritors are ready (when timeout is NULL, select may block indefinetly) #include int select(int nfds, fd_set *restrict readfds, fd_set *restrict writefds, fd_set *restrict errorfds, struct timeval *restrict timeout); void FD_CLR(int fd, fd_set *fdset); int FD_ISSET(int fd, fd_set *fdset); void FD_SET(int fd, fd_set *fdset); void FD_ZERO(fd_set *fdset); BIL 244 – System Programming

The select function

• On success return select clears all the descriptors in each of readfds, writefds, errorfds excepts those descriptors thay are ready. • If successful, the function returns the number ıf file descriptors that are ready • If unsuccessful it returns -1 and sets errno • See the examples in the book to see how select can be used for concurrent file manupulations

BIL 244 – System Programming

The poll function • The poll function is similar to select function, but it organizes the information by file selector rather than by type conditions. • The possible events for one file descriptor are stored in a struct pollfd. In contrast, select organizes information by the type of event and has separate descriptor marks for read, write and error conditions. #include int

poll (struct pollfd fds[], nfds_t nfds, int timeout);

• The poll function returns 0 if it times out. If successfull, poll returns the number ıf descriptors that have events. If unsuccessful returns -1 and sets errno BIL 244 – System Programming

File Representation • Files are designated within C programs either by file pointers or by file descriptors. • The standard I/O library for ISO C use file pointers (as in fopen, fscanf, fprintf, fread, fwrite) • The UNIX I/O functions use file descriptors (as in read, write, open, close, ioctl) • File pointers and file descriptors provide logical designations called handles for performing device independent input and output.

BIL 244 – System Programming

File Representation • The File Descriptor Table is part of the user program area and can be thought of as an array of pointers indexed by the file descriptors. • The pointers point to entries in the System File Table. • The System File Table is in the kernel area. • It contains an entry for each open file. • Entries contain pointers to a table of inodes kept in memory. • Entries contain other information including the current file offset and a count of the number of file descriptors that are using this entry. • When a file is closed the count is decremented. The entry is freed when the count becomes 0. • The In-Memory Inode Table contains copies of the inodes that are being used. BIL 244 – System Programming

File Descriptors • The open function associates a file (or a physical device) with the logical handle used in the program. The file is specified by a charater string (like /home/ann/my.dat). The handle is an integer that can be thought of as an index into a file descriptor table that is specific to a process. • The file descriptor table is a part of the user area, but the program cannot access it except through functions using the file descriptor

BIL 244 – System Programming

File Descriptors A possible output of myfd = open (“/home/ann/my.dat, O_RDONLY”);

BIL 244 – System Programming

File Descriptors • The open function creates an entry in the file descriptor table, returning 3 specifying that the file descriptor entry is in position 3 of the process file descriptor table • The system file table which is shared by all processes in the system has an entry for each open. Each system sile table entry contains the file offset, an indication of access mode and a count of the number of file descriptor table entries pointing to it. • Each of these entries points to the same entry in the inmemory inode table. The in-memory inode table contains an entry for each active file in the system. BIL 244 – System Programming

File Pointers and Buffering • The ISO C standard I/O library uses file pointers as handlers for I/O. • A file pointer points a data structure called a FILE structure in the user area of the process • The following code segment FILE *myfp; if ((myfp = fopen("/home/ann/my.dat", "w")) == NULL) perror("Failed to open /home/ann/my.dat"); else fprintf(myfp, "This is a test");

can graphically interpreted as follows BIL 244 – System Programming

File Pointers and Buffering

BIL 244 – System Programming

File Pointers and Buffering • I/O using file pointers will read from or write to the buffer. • The buffer will be filled or emptied when necessary. • A write may fill part of the buffer without causing any physical I/O to the file. • The amount of buffer may vary. • If a write is done to standard output and them the program crashes, the data written may not show up on the screen. • Standard error is not buffered. • Interleaving output to standard output and standard error may cause output to appear in an unpredictable order. • You can force the physical output to occur with an fflush.

BIL 244 – System Programming

#include int main(void) { fprintf(stdout, fprintf(stderr, fprintf(stdout, fprintf(stderr,

"a"); "a has been written\n"); "b"); "b has been written\n");

fprintf(stdout, "\n"); return 0; }

#include int main(void) { int i; fprintf(stdout, "a"); scanf("%d", &i); fprintf(stderr, "a has been written\n"); fprintf(stdout, "b"); fprintf(stderr, "b has been written\n"); fprintf(stdout, "\n"); return 0; }

Inheritance of File Descriptors • When fork creates a child, the child inherits a copy of the parents address space, including the file descriptor table • The child inherints a copy of most of the parent’s environment and context including the signal state, the scheduling parameters (and the file descriptor table). • As the children recieve a copy of their parent’s file descriptor table at the time of the fork, the parent and the children share the same file offsets that were opened by the parent prior to the fork

BIL 244 – System Programming

#include #include #include #include



int main(void) { char c = '!'; int myfd; if ((myfd = open("my.dat", O_RDONLY)) == -1) { perror("Failed to open file"); return 1; } if (fork() == -1) { perror("Failed to fork"); return 1; } read(myfd, &c, 1); printf("Process %ld got %c\n", (long)getpid(), c); return 0; }

#include #include #include #include



int main(void) { char c = '!'; int myfd; if (fork() == -1) { perror("Failed to fork"); return 1; } if ((myfd = open("my.dat", O_RDONLY)) == -1) { perror("Failed to open file"); return 1; } read(myfd, &c, 1); printf("Process %ld got %c\n", (long)getpid(),c); return 0; }

Filters and Redirection • A filter reads from standard input, preforms a transformation, and outputs the result to standard output. • Filters write their error messages to standard error • All of the parameters of a filter are communicated as command-line arguments. The input data should have no header or trailers and a filter should not require any interaction with the user. • Examples of useful UNIX filters include head, tail, more, sort, grep and awk. BIL 244 – System Programming

Filters and Redirection • Recall that a file descriptor is an index into the file description table of that process. • Each entry of the file descriptor table points to an entry in the system file table, which is created when the file is opened. • A program can modify the file description table entry so that it points to a different entry in the system file table. This action is known as redirection. • Most shells interpret the (>) charater on the command line as redirection of standard output and (...