From: firstname.dot.lastname@googlemail.com   
      
   On Sat, 02 Feb 2013 15:14:19 -0600, Öö Tiib wrote:   
      
      
   >> Depends on the kind of data stored in the file or database. Often, yes.   
   >   
   > Can you attach some reasoning? Unless you try to dodge me with nonsense   
   > that contents of PNG file are better as text than database. Contents of   
   > PNG file are binary already.   
      
   You seem to misunderstand. I was pointing out that a database is not the   
   most suitable thing for all binary formats. For example, an image is   
   better stored as a PNG (or perhaps TIFF or JPEG or FITS or PNM...) than   
   stored as records in a database.   
      
   Sure, you can cram a bunch of PNGs in a database if you wish, but unless   
   you need database like functionality to actually find them, there is   
   little point and the filesystem itself serves as an entirely adequate   
   database.   
      
   >> SQL databases are great for storing tabular, relational data,   
   >> especially if there are interesting constraints.   
   >   
   > SQL is fine with whatever large set of records.   
      
   Yes, but SQL is designed for manipulating relational data. There's no   
   point in using it to store an image or video.   
      
   > There is enormous amount of well- documented binary formats supported   
   > with well-tested libraries, codecs, code generators and tools.  Custom,   
   > application-specific binary format is basically never needed. Do not try   
   > to picture that I was suggesting such thing.   
      
   Yes, but essentially as the generality of the file format increases so   
   does the complexity up until the point where it becomes a filesystem. You   
   then have to consider how to store or lay out the data. There is no one   
   size fits all binary format, and often ploughing through complex APIs   
   becomes more error prone than a few cin/cout statements. Note that not   
   everyone needs to deal with arbitrarily mangled data files from untrusted   
   sources.   
      
   As for custom formats, sometimes the easiest and most efficient way to   
   deal with data is to simply mmap a region of memory.   
      
   >> > Without proper tools I am lost there. For database there are most   
   >> > tools one can dream of ... often even integrated into IDEs.   
   >>   
   >> That's a narrow view of the world. Databases are not the only way of   
   >> storing data, neither are they the most suitable way of storing every   
   >> type of data.   
   >   
   > Make my view of world wider then. Text-based multimedia formats do not   
   > compete much. Various configuration and layout descriptors where text is   
   > good never reach 100Mb. What that 100Mb of text contains that is not a   
   > database? What data it is that you talk about?   
   >   
   > If it is database then SQL provides far more power than AWK.   
      
   Which database? Doing general purpose programming in SQLite for instance   
   is not particularly easy.   
      
   > If it is again about that yet unrevealed mysterious sort of information   
   > that does not fit into database nicely and is best represented as 100Mb   
   > of text then ... what it is?   
      
   A common one to store in text based formats is for example logging data   
   from instrumentation, or output from simulations etc. Often it is   
   basically tabular, though often it is more record oriented with several   
   different record types. At this point, SQL does not provide vast   
   advantages (except for binaryness) because the data is tabular but not   
   relational. There's nothing to join.   
      
   And if you have stored it in text file you can easily load it into any   
   other further processing language. With luck, your language will have   
   some database bindings. If so you can then (after reading the   
   documentation) get on with doing "select * from data" before doing   
   further processing.   
      
   I'm not claiming text files are the be-all and end-all of data formats.   
   They are, however especially suitable for many things. A substantial part   
   of the unix philosophy is based on text files and pipelines and it does   
   work very well.   
      
   Finally, text files are also great for archival purposes.   
      
   -Ed   
      
      
   --   
         [ See http://www.gotw.ca/resources/clcm.htm for info about ]   
         [ comp.lang.c++.moderated.    First time posters: Do this! ]   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)