MSGID: 1:266/625.1 6842a36a
TZUTC: -0400
REPLY: 1:129/14.1 6842933f
PID: msged/lnx 6.3 2023-02-24
TID: hpt/lnx 1.9 2024-03-02
 AG> Can you please describe the cons and pros between Squish and JAM? 
 AG> Squish seems to be stable, but only through the latest Husky SMAPI, 
 AG> no more corruptions and bugs. I know nothing about JAM.

I have little experience with Squish, but I did find the file structures for a
Squish message base.  Quickly, the main difference is the style of how the
data gets stored, obviously right :) 

Full disclosure,  I've used JAM as my primary message datastore since 1987.  I
was an early adoptor of RemoteAccess and switched from the Hudson message base
to JAM rather quickly.  Since I was soon to be on OS/2 (1.3) as my system's
operating system (must have been 89, I remember going to a big computer
software retailer and purchasing a box copy of OS/2) JAM solved a lot of
performance and capacity issues that the Hudson message base suffered. (if
you're not familiar with the Hudson message base, it is the base developed by
the author of QuickBBS, that it, and later DOS bbses could use to store many
areas in a single message base, corruption was common; especially for
multi-line systems, as every line battled to lock, write, unlock using DOS's
lackluster file locking (SHARE.EXE) system.

JAM has three files that do the work of storing messages.  .JHR - the
headers,  .JDT - the message text, and .JDX - the header index.

    The JDX file is all fixed records, so counting the number of records is as
easy as getting the size of the file and dividing my the record size (which is
8 bytes, two ulongs (2*32 unsigned integers)

    The JHR file is the heart of the message store, the JDX can be rebuilt by
reading the JHR, but not the other way around.  The JHR consists of a fixed
record length structure that contains all of the numeric values of the
message, things like attributes, dates, and location of size of the text
stored in the JDT.  It also has a variable length set of fields that contain
the 'strings' part of the message, like the sender and receiver names,
subject, various control parts like MSGIDs, and REPLYIDs.

    The JDT file a file full of the text of the message.  the JHR points to
the first byte of the given message in the file and the JHR also provides the
length of the text to read.  think fread(file,&buffer,length) to grab the
whole chunk in one go.

    There is a fourth file, the .JLR, it is optional, used by the BBS software
to store last read information.  It's been a number of years since any of my
BBSs used, or even created this file.


Squish, in comparision, has two files.  (reading documentation here) .SQD for
the message header and body, and .SQI which is an index into .SQD   It appears
that it was laid out like a double-linked list; where each message has a
pointer to the previous and next messages.  the SQI is a series of fixed
record length entries, which the doc says "is  used primarily for  performing
random  access look-ups by message number."

    Thus, like JAM, Squish uses a fixed record length file as an index to find
the messages in the larger header file.

    However, Squish, unlike JAM, puts all the rest of the data into a single
file.  The documentation says this file contains a required "frame header"
that has links to the previous and next messages in the store, as well as
"optional message header, control information and message body fields."



I think the main difference between the two, is the 'difficulty' in updating
it.  Like editing header information, say changing the receiptant's name -
which being contained in the variable length record parts means:

    JAM:  add on a new header record to the JHR, update JDX record to point to
new JHR, mark the old JHR invalid (deleted)

    Squish: add on new header record to SQD, also write new body text. and
update SDI, mark the old header frame invalid, update the previous header with
next next pointer, and update the next header with a new previous pointer.

I suppose the difference is trivial with today's computers, but in the 80's
and early 90's every byte and CPU cycle counted -  especially moreso on
multi-line systems like mine was.   I still think that JAM is slightly
superior to Squish, but I admit I know JAM a whole lot more in depth then
Squish.
 

 AG> I was trying to write a squish2json converter in multiple languages 
 AG> (C++, Python, Go) using SMAPI for like a month and couldn't figure 
 AG> out SMAPI, it's weird.

SMAPI is absolute SPAGHETTI; it works but the learning curve is very steep.  I
would have thought that SMAPI would be a wrapper around all of the different
databases it supports, but in my initial attempt to use it (many years ago) it
was more of a goto place to get all the different store's API - you had to
code to each store.  I may have gotten it wrong, but it should have been
straight forward to use.  IE. call gizmo to open message base,  call function
in the gizmo to read and write messages, unaware of the specific store.

In my code, being object oriented and all -- I could create a template class
named "MsgBase" to which "JAMMsgBase" and "SquishMsgBase" would inherit. 
Thus, if the program knew how to interact with "MsgBase", it wouldn't matter
if it was really a "JAMMsgBase" or a "SquishMsgBase" as that abstraction is
contained in each of the implemented classes.  Thus adding more messages bases
"DOTMsgBase" for example,  or my goal "MySQLMsgBase" or "SQLiteMsgBase"

Do you still need that squish2json?  Bet I could rip one out in a couple days
if needed, in Python of course.


Cheers,
Scott 

---
 * Origin: <=-[ The Digital Post ]-=> (DevPoint) (1:266/625.1)
SEEN-BY: 105/81 106/201 124/5009 128/187 129/12 14 102 125 160 165
SEEN-BY: 153/7715 154/110 218/700 226/30 227/114 229/110 114 206 317
SEEN-BY: 229/426 428 470 664 700 705 261/1 220 266/75 512 618 625
SEEN-BY: 266/630 267/152 154 291/111 292/854 320/219 322/757 342/200
SEEN-BY: 396/45 460/58 633/280 712/848 902/26 5075/35
PATH: 266/625 512 229/426