Re: using fread

From: Billy Patton (bpatton_at_ti.com)
Date: 05/09/05 

Date: Mon, 9 May 2005 14:57:17 -0500

On Mon, 9 May 2005, Ralf Fassel wrote:

> * Billy Patton <bpatton@ti.com>
> | I'm trying to parse a prioriety ascii file.
> | I want to read it in chunks instead of a char at a time to allow me to
> | lookahead to tokenize the words in the input file.
>
> You're working too hard on this. I'd suggest to get rid of those
> gazillions of global variables, and work with a local buffer instead:
>
> const size_t buflen = 1024;
> char buffer[buflen];
> while (true) {
> // read one less than the buffer length, so we can
> // zero-terminate it:
> size_t read = fread(buffer,1,buflen-1,fp);
> if (!read) break; // nothing more to read
> // make sure buffer is zero-terminated if you want to use
> // strlen() on it
> buffer[read] = '\0';
> // invoke parser with the fresh stuff in 'buffer' of length 'read'
> parse_buffer(buffer, read);
> }
>
> In parse_buffer(), do all the necessary copying, probably using a
> std::string instead of memcpy(), since you're using C++ anyway.
>
> R'
>

Well, I finallly got it:
static void read_more(void)
{
   int len = bufend - bptr;
   if (len) memcpy(buffer,bptr,len);
   bptr = buffer;
   long read = fread(buffer+len,1,BUFSIZE-len,fp);
   buflen = len + read;
   bufend = buffer + buflen;
}

Now if someone would be so kind as to tell my how I can use mmap I would be
very appreciative.
This program will be used to parse multi gig files. It would probably be much
faster if I could use mmap and not read anything into memory.

    ___ _ ____ ___ __ __
   / _ )(_) / /_ __ / _ \___ _/ /_/ /____ ___
  / _ / / / / // / / ___/ _ `/ __/ __/ _ \/ _ \
/____/_/_/_/\_, / /_/ \_,_/\__/\__/\___/_//_/
            /___/
Texas Instruments ASIC Circuit Design Methodology Group
Dallas, Texas, 214-480-4455, b-patton@ti.com

Relevant Pages

  • Re: Reading a text file
    ... > I need to use VB6 to read in and parse through a text file one line ... If the file isn't too big read it all into a single string and use ... you'd like to work with in one piece then read chunks at a time and do the ...
    (microsoft.public.vb.general.discussion)
  • Re: XML to inMemory Hash
    ... Is there any way to create chunks so that it wont parse the entire file. ... I am planning to solve my memory issue by making the XML into Chunks ...
    (perl.beginners)
  • Re: XML to inMemory Hash
    ... Is there any way to create chunks so that it wont parse the entire file. ... I am planning to solve my memory issue by making the XML into Chunks ...
    (perl.beginners)
  • Re: create/use array from external txt file?
    ... Why not go for XML?...Its supported and will be a lot easier to parse it ... compare to text file... ... "Rissa" wrote in message ... I have broken all those chunks into separate ...
    (microsoft.public.pocketpc.developer)
  • mmap parsing...
    ... I have a file stored in memory using mmap() and I'd like to parse to ... Prev by Date: ...
    (comp.lang.c)