Re: Output to text files or a database?

kd wrote:
I was reading The Art of Unix Programming yesterday, and was inspired
to rework and split up an large monolithic application of mine.

This software has to do with the retrieval and processing of travel
information (flights, hotels, etc) from different sources. Each source
has travel data formatted differently.

With plain text files being the universally loved file format, I
planned on writing small apps that would pull down data from these
different sources, and write them to text files in a universal format.

Another small app would read these text files, generate sql insert
statements from the data, and dump all the data into a relational
database.

I proposed this to a coworker, and he asked why the text files were
even necessary -- why not just immediately generate sql statements from
the different formats, and use the tables in the DB as the universal
format? Textual representations of that data could just be generated
from the database, after all.

Is he right? Is there any reason to use text files as an intermediate
format? My gut is telling me to keep them, but my gut isn't good at
convincing people. :) What am I missing?

Well, there are nice things about text files:

You can read them with cat, page them with more
You can edit them with your favourite editor
You can keep histories with your favourite versioning system
You can "understand" them without needing to know the format
and, if some day you want to move over to UTF-8, you can convert them with iconv.

But if you can generate them anyway, then it is just a metter of taste.

Robert
.

Relevant Pages

  • Re: Output to text files or a database?
    ... This software has to do with the retrieval and processing of travel ... With plain text files being the universally loved file format, ... Another small app would read these text files, ... from the database, after all. ...
    (comp.unix.programmer)
  • Re: Directory Alert!
    ... One of the nice things about my recent trips was using my GPS unit. ... Less travel is always a good thing...... ... One new addition is to identify the folks using WebTV since we have some formatting issues for these folks that need to be addressed. ... a pdf format will be available as a choice of formats as well as the sortable spreadsheet format and non-sortable text format. ...
    (rec.crafts.textiles.quilting)
  • Re: Directory Alert!
    ... Since I have started being serious about quilting for a living (long arm ... Less travel = more quilting! ... formatting issues for these folks that need to be addressed. ... sortable spreadsheet format and non-sortable text format. ...
    (rec.crafts.textiles.quilting)
  • Output to text files or a database?
    ... With plain text files being the universally loved file format, ... even necessary -- why not just immediately generate sql statements from ... from the database, after all. ... My gut is telling me to keep them, but my gut isn't good at ...
    (comp.unix.programmer)
  • Re: Output to text files or a database?
    ... This software has to do with the retrieval and processing of travel ... With plain text files being the universally loved file format, ... Another small app would read these text files, generate sql insert ... This universe shipped by weight, ...
    (comp.unix.programmer)