SUMMARY: Tru64 pipe buffer size

From: Iain Barker (ibarker_at_aastra.com)
Date: 04/11/05

  • Next message: Kannberg, Andy: "problem with PK4"
    Date: Mon, 11 Apr 2005 15:40:11 -0400
    To: tru64-unix-managers@ornl.gov
    
    

    The problem was that the application detected whether its stdout was going to file (pipe) or term, and did buffering in the case of a file.

    Thanks to John Lanier, who wrote:

    Reference these parameters in the "vfs" kernel subsystem:

    --> sysconfig -q vfs|grep pipe
    pipe_maxbuf_size = 262144
    pipe_databuf_size = 8192
    pipe_max_bytes_all_pipes = 134217728

    --> sysconfig -Q vfs|grep pipe
    pipe_maxbuf_size - type=INT op=CRQ min_val=4096 max_val=1048576
    pipe_databuf_size - type=INT op=CRQ min_val=4096 max_val=32768
    pipe_max_bytes_all_pipes - type=INT op=CRQ min_val=819200
    max_val=1717986918

    In actual fact, there was an easier way around the problem in our particular situation: Pipe the output via the tee command.

    It seems that tee opens its stdin as a term, and its output as a file handle. So it can happily sit in the middle of the pipe and do the required un-buffering.

    - Iain

    -----Original Message-----
    From: tru64-unix-managers-owner@ornl.gov
    [mailto:tru64-unix-managers-owner@ornl.gov]On Behalf Of Iain Barker
    Sent: Wednesday, 06 April, 2005 14:57
    To: tru64-unix-managers@ornl.gov
    Subject: Tru64 pipe buffer size

    Managers,

    Is there any way to alter the buffer size used for the pipe (|) command on Tru64?

    We have a third-party tool which writes line-by-line to terminal device, but when its output is piped it seems to be buffering the data in chunks of approx 4KB at a guess.

    For example, assuming each line represents ~1.5K of data:

    ./foobar
    a
    b
    c
    d
    e
    f
    end of data

    ./foobar | tee outfile
    a
    b
    c
    <20 minute wait>
    d
    e
    f
    end of data

    thanks,
            Iain


  • Next message: Kannberg, Andy: "problem with PK4"

    Relevant Pages

    • Re: Design help...an explosion of interfaces......
      ... pipe is buffered are not; it is just loading or extracting from the pipe ... and call the right method with the right Pipe subclass instance. ... who understands the buffering context instantiate the relationship and ... the type attribute has a different semantics than the class ...
      (comp.object)
    • Re: stdout redirected to a processs own stdin
      ... Probably does not put anything in the pipe because output is buffered. ... > I suspect this is due to a misunderstanding of how pipes work. ... The UNIX system has no way of knowing about any ... turn off the output buffering with setbuf or flush the ...
      (comp.unix.programmer)
    • Re: fwrite over a pipe
      ... fprint or fwrite to output to the parent-to-child command pipeline, ... it might not be on the actual pipe when you do a read of the child." ... The fwrite and fprintf functions use buffered I/O. ... The buffering of these functions can be modified (or even turned ...
      (comp.unix.programmer)
    • Re: circular buffering using substr() ?
      ... Can I use substr() to do circular buffering? ... stream from a pipe, which I need to send out on another pipe, but ... $offset); # OUTPIPE is non-blocking ...
      (comp.lang.perl.misc)
    • Re: pipe buffering
      ... read-end of pipe, ... the problem is with buffering, is there a way to disable buffering on pipe? ... are you using fflush in the child process? ...
      (microsoft.public.vc.language)