Re: Buffering in pipes

From: Haakon Riiser (hakonrk_at_fys.uio.no)
Date: 11/12/03 

Date: Wed, 12 Nov 2003 14:04:15 GMT

[Derk Gwen]

> Haakon Riiser <hakonrk@fys.uio.no> wrote:
> # I have a pair of programs that communicate using standard pipes
> # (i.e., pipe(2)). The process with the write-end of the pipe
> # downloads data from the net (or a local file), and dumps it into
> # the pipe. The read-end gets data from this pipe, never caring
> # where it came from, and feeds it into an MPEG decoder.
>
> I suppose if you flagellate your box and yourself long enough, you can get
> pipes to do what you want. Or else you can try a more appropriate technique.
> With IPC messages you have more direct control over what is in the queue,
> you can fake up asynchronous signals, and you can put your process to sleep
> on queue empty. You can set up a shared memory for the data transfer and use
> the queue to transfer available buffer addresses and control messages.
> You can also use semaphores to manage shared buffers.

Yes, I know all about the alternatives, but if there were an
easy way to discard buffered data from pipes, they would give
me everything I need right out of the box. I'd like to avoid
reinventing pipes using other forms of IPC, if at all possible.
(Actually, I have already done that once, but that implementation
was designed for IPC between threads, so it won't work here.
And it wasn't exactly painless to write either -- there's little I
hate more than race conditions.)

Anyway, I'm fairly sure that named pipes (using close/open to
flush old data) would work but, as I said, I am curious as to
_why_ my current solution doesn't work. How come there's still
leftovers in the pipe after read() returns EAGAIN? 

-- 
 Haakon

Relevant Pages

  • Re: pipes: how 2 programs can communicate
    ... Squirrel wrote: ... >You make two pipes in the parent program. ... potential problem related to buffering of the C printf() function. ... (if you write to a terminal that happens when either the buffer is full ...
    (comp.programming)
  • Re: Clueless: piping between 2 non-python processes
    ... ideally you'd close all other file descriptors in the child process ... Well, if these are the only child processes his program spawns, he can ... > brittle, because pipes have a fixed, limited buffer size and because ... to call fflushor change their I/O library's buffer settings as needed. ...
    (comp.lang.python)
  • Re: Best Duel Buffer Needed
    ... I'm looking for the best buffer for buffing briar pipes and their ... Let me offer another option Scott: a four-step BENCH drill press. ... the belts for 1100 RPM as you have better control of the pipe in hand, ...
    (alt.smokers.pipes)
  • Re: Pipe buffers limit of 16 * 4K
    ... faced a problem with the 4K buffer limit for the pipes. ... having these in the kernel for toying with minibuffer audio processing. ...
    (Linux-Kernel)
  • Re: New to FORTRAN - Reformatting One Column to Multiple Columns
    ... continue counting, rewind, and allocate the appropriate array. ... and is often used with pipes. ... seekable input it can use tricks with file positioning to ... the only possibility is a buffer, and there is no way to know ...
    (comp.lang.fortran)