Re: using pipe


junky_fellow@xxxxxxxxxxx wrote:
On Oct 25, 4:32 pm, j...@xxxxxxxxxxx (Jens Thoms Toerring) wrote:
junky_fel...@xxxxxxxxxxx <junky_fel...@xxxxxxxxxxx> wrote:
On Oct 25, 3:37 pm, loic-...@xxxxxxx wrote:
I wanted to know if the "pipe" can only be used for data transmission from one process
to another (parent to child and viceversa) ?Yes, pipe can be used as Inter-Process Communication (IPC) between
related processes.

Can a pipe be used to transfer data in the same prcoess ?This could done, albeit this can be tricky.

For instance, I want to execute some function and want to save its
output to a pipe then in the same process, I need to read the output from this
pipe to some buffer (one string at a time). Can this be done ?
I beg your pardon. But why not using directly a buffer for that? The
I don't want to use a buffer to store the output, as the output size
may be huge (saveral MBs).Then you definitely can't use pipes. A pipe typically has a rather
limited size (in the order of a few kBs). They're not meant as a way
to obtain some temporary storage.

I want to keep this in a pipe (believing pipe takes storage from the
file system).Going through the file system (and I guess what you were looking for
is a FIFO, not a pipe) is slow. Why not use memory, obtained by e.g.
malloc(), and let the system take care of swapping out that parts
of the memory that don't fit anymore and aren't needed at the moment?
While I haven't done any tests I would guess that this might be faster
- only writing to disk when necessary and using probably highly opti-
mized algorithms for doing it when there's no alternative. And I
probably would start with thinking about a linked list of the strings,
so you don't need a single huge block of memory that grows and shrinks
(possibly requiring to copy the whole thing when you need to increase
its size).

I can't use malloc due to some design restriction. I have to use some
files to store the
command output temporarily and later read the file to get the output.
Can I use memory file sustems to store these files? I believe
reading/writing from memory
file systems would be faster. But, I am not aware of any negative
impact of mfs on the system.
Is it a good idea to use mfs to stoe these files ?.

Did you think of mapped memory?

You could create an anonymous or named memory mapping. On Linux, for
example, memory mappings created by shm_open are backed by tmpfs
filesystem and you can use them just as regular files. shm_open ->
(ftruncate -> mmap -> munmap)* -> close -> shm_unlink.

.

Relevant Pages

  • Re: using pipe
    ... to another?Yes, pipe can be used as Inter-Process Communication between ... related processes. ... file system).Going through the file system (and I guess what you were looking for ... Why not use memory, obtained by e.g. ...
    (comp.unix.programmer)
  • Re: Pthread fd and memory leak
    ... and closes the socket file descriptor. ... When you send an fd over a pipe, the fd is still open in the sending ... memory at the pointer value and free it without errors, ...
    (comp.programming.threads)
  • Re: Your Greatest ASP Memory
    ... memory from ASP, but many many memories. ... few pipe shows I am able to attend (it looks like DC no longer hosts a pipe ... Because of ASP, I no longer smoke only Borkum ...
    (alt.smokers.pipes)
  • Re: Syslets, Threadlets, generic AIO support, v6
    ... * For lack of a better implementation, implement vmsplice() to userspace ... In both cases the output is a pipe, ... Splicing from user memory is a simple operation ... unsigned long nr_segs, unsigned int flags) ...
    (Linux-Kernel)
  • Re: using pipe
    ... to another?Yes, pipe can be used as Inter-Process Communication between ... related processes. ... Going through the file system (and I guess what you were looking for ... Why not use memory, obtained by e.g. ...
    (comp.unix.programmer)
Quantcast