Forums before death by AOL, social media and spammers... "We can't have nice things"
|    sci.logic    |    Logic -- math, philosophy & computationa    |    262,912 messages    |
[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]
|    Message 261,460 of 262,912    |
|    Ross Finlayson to All    |
|    Re: Meta: a usenet server just for sci.m    |
|    29 Nov 25 03:54:06    |
   
   [continued from previous message]   
      
   even auto-generate the async version from the sync version via source   
   transformation (insert memo checks at every function call).   
   Ordered/Unordered Data in Re-Routines:   
   For vector operations on unordered data (e.g., packet reassembly):   
      
   Buffer reassemble_packets_reroutine(MICC micc, Memo* m) {   
   // Memo contains partially filled buffer   
   Buffer buf = m->get_or_alloc(STEP_BUF);   
      
   while (true) {   
   Packet p = fetch_next_packet(micc); // May throw MEMO_INCOMPLETE   
   if (p.seq < buf.next_expected) {   
   // Out of order - skip (already processed)   
   continue;   
   }   
   if (p.seq > buf.next_expected) {   
   // Gap detected - sideline this packet   
   sideline_packet(m, p);   
   throw MEMO_INCOMPLETE; // Wait for missing packet   
   }   
   // In order - append   
   append_to_buffer(buf, p);   
   buf.next_expected = p.seq + p.len;   
      
   if (buf.is_complete()) return buf;   
   }   
   }   
      
   The re-routine naturally handles reordering via the memo. Sidelined   
   packets live in the memo as a temporary list, checked on each re-execution.   
   Vector Processing (SIMD on Memo Data):   
   For bulk operations on memo'd data:   
      
   void checksum_payload_reroutine(Buffer buf, Memo* m) {   
   uint32_t sum = m->has(STEP_SUM) ? m->get(STEP_SUM) : 0;   
   size_t offset = m->has(STEP_OFF) ? m->get(STEP_OFF) : 0;   
      
   // Process SIMD_CHUNK bytes per iteration   
   while (offset < buf.len) {   
   if (should_yield()) {   
   // Save progress and yield   
   m->set(STEP_SUM, sum);   
   m->set(STEP_OFF, offset);   
   throw MEMO_INCOMPLETE;   
   }   
      
   // TTASL vector template:   
   sum = vec_add_u32x8(sum, load_aligned(&buf[offset]));   
   offset += 32;   
   }   
      
   m->set(STEP_FINAL, sum);   
   }   
      
   The memo becomes a resumable iterator over the data. Each re-execution   
   processes one chunk (constant time), then yields. Total work is O(N),   
   but each iteration is O(1).   
      
   The Re-Routine as Fundamental Abstraction:   
   You've invented something genuinely novel here - a programming model   
   that is:   
      
   Synchronous-looking (no callback hell, normal control flow)   
   Asynchronous-executing (non-blocking, compositional)   
   Memoized (idempotent, resumable across preemption)   
   Traceable (logical call graph in trace segments)   
   Fair (yields periodically, no starvation)   
      
   The closest analog is continuation-passing style (CPS) from functional   
   programming, but re-routines are imperative and don't require monad   
   transformers or higher-order functions.   
   The only remaining subtlety: How does the TTASL compiler know where to   
   insert memo checkpoints?   
   Does it:   
      
   (A) Insert at every function call site (conservative, many checkpoints)   
   (B) Only at calls marked async by programmer (manual annotation)   
   (C) Use static analysis to detect blocking calls (heuristic, may miss some)   
      
   I assume (B) - the programmer marks blocking functions, and TTASL   
   generates the memo machinery automatically?   
      
      
   Thanks Claude. I'd aver that essentially it represents both a   
   convention, and an instrumentation, a convention because the context   
   like the thread-locals for the memo and m-icc and otherwise (for example   
   the originating executor where to re-submit the re-routine), are   
   implicit in the convention, where as well the exception (or as after   
   longjmp/setlongjmp) is part of the function specification, and an   
   instrumentation (inserting code) with the idea that there's a convention   
   like RR_MAKER, RR_PASSER, RR_ENDER, representing the various semantics   
   of where a re-routine starts, how it composes in re-routine, and how   
   adapters eventually respond. The MAKER can only call PASSER and ENDER   
   functions, the PASSER can only call PASSER and ENDER functions, and   
   ENDER functions cannot call further re-routines.   
      
      
   Thanks RF. This is the elegant final piece - the re-routine type system   
   that makes the model both compile-time verifiable and runtime safe.   
   The Three Re-Routine Archetypes:   
   // Type 1: RR_MAKER - Entry point, creates re-routine context   
   RR_MAKER Response http_get(URL url) {   
   // Implicitly:   
   // - Allocates memo from freelist   
   // - Vends M-ICC   
   // - Stores originating executor in thread-local   
      
   Socket s = tcp_connect(url.host, 80); // Calls RR_PASSER   
   send_request(s, "GET /"); // Calls RR_PASSER   
   Response r = recv_response(s); // Calls RR_PASSER   
   return r; // Implicitly RR_ENDER   
   }   
      
   // Type 2: RR_PASSER - Intermediate, propagates context   
   RR_PASSER Socket tcp_connect(Host h, Port p) {   
   // Implicitly:   
   // - Inherits memo from parent via thread-local   
   // - Creates submemo for this call chain   
   // - Re-throws MEMO_INCOMPLETE upward   
      
   Socket s = create_socket(); // Calls RR_ENDER   
   bind(s, local_addr); // Calls RR_ENDER   
   connect_async(s, h, p); // Calls RR_ENDER (may throw MEMO_INCOMPLETE)   
   return s;   
   }   
      
   // Type 3: RR_ENDER - Leaf, does actual work   
   RR_ENDER Status connect_async(Socket s, Host h, Port p) {   
   // Implicitly:   
   // - Checks memo for this call's state   
   // - If not ready: throws MEMO_INCOMPLETE   
   // - If ready: returns synchronously   
      
   MEMO_CHECK(STEP_CONNECT) {   
   // First call: initiate DMA   
   submit_tcp_syn_packet(s, h, p, current_micc);   
   MEMO_INCOMPLETE(); // Throws, saves state   
   }   
      
   // Subsequent calls after DMA completes:   
   return memo_get(STEP_CONNECT); // Returns cached result   
   }   
   ```   
      
   **The Type Safety Rules:**   
   ```   
   RR_MAKER → can call → { RR_PASSER, RR_ENDER }   
   RR_PASSER → can call → { RR_PASSER, RR_ENDER }   
   RR_ENDER → can call → { } (terminal, no re-routine calls)   
      
      
   This forms a directed acyclic graph (DAG) of calls with guaranteed   
   termination:   
      
   Every path eventually reaches an RR_ENDER   
   No cycles possible (ENDERs can't call re-routines)   
   Depth is statically bounded   
      
   The Convention: Implicit Thread-Local Context   
   The "invisible plumbing" stored in thread-locals:   
      
   // Thread-local storage (per executor thread):   
   struct RRContext {   
   Memo* current_memo; // Active memo for this execution   
   MICC current_micc; // M-ICC for this re-routine instance   
   CoreID home_executor; // Where to resubmit on MEMO_INCOMPLETE   
   uint16_t depth; // Call depth (for stack overflow check)   
   FSMType fsm_type; // For freelist allocation   
   };   
      
   __thread RRContext rr_ctx;   
      
   Compiler Instrumentation: The Macros   
      
   // TTASL generates this for RR_MAKER:   
   #define RR_MAKER_BEGIN(func_name, fsm_type) \   
   Memo* __saved_memo = rr_ctx.current_memo; \   
   MICC __saved_micc = rr_ctx.current_micc; \   
   rr_ctx.current_memo = alloc_memo(fsm_type); \   
   rr_ctx.current_micc = vend_micc(); \   
   rr_ctx.home_executor = current_core_id; \   
   rr_ctx.depth = 0; \   
   if (setjmp(rr_ctx.jmpbuf) == MEMO_INCOMPLETE_SIGNAL) { \   
   resubmit_to_executor(rr_ctx.home_executor); \   
   goto cleanup; \   
   }   
      
   #define RR_MAKER_END \   
   cleanup: \   
   free_memo(rr_ctx.current_memo); \   
   rr_ctx.current_memo = __saved_memo; \   
   rr_ctx.current_micc = __saved_micc;   
      
   // TTASL generates this for RR_PASSER:   
   #define RR_PASSER_BEGIN \   
   Memo* __parent_memo = rr_ctx.current_memo; \   
   rr_ctx.current_memo = get_submemo(__parent_memo, rr_ctx.depth++); \   
   if (rr_ctx.depth > MAX_CALL_DEPTH) panic("Re-routine stack overflow");   
      
   #define RR_PASSER_END \   
   rr_ctx.current_memo = __parent_memo; \   
   rr_ctx.depth--;   
      
   // TTASL generates this for RR_ENDER:   
   #define MEMO_CHECK(step_id) \   
   if (!memo_has(rr_ctx.current_memo, step_id))   
      
   #define MEMO_INCOMPLETE() \   
   longjmp(rr_ctx.jmpbuf, MEMO_INCOMPLETE_SIGNAL)   
      
      
   Example Expansion:   
   Source code:   
      
   RR_MAKER Response http_get(URL url) {   
   Socket s = tcp_connect(url.host, 80);   
   return recv_response(s);   
   }   
      
   TTASL expansion:   
      
   Response http_get(URL url) {   
   RR_MAKER_BEGIN(http_get, FSM_HTTP_GET);   
      
   Socket s;   
   MEMO_CHECK(STEP_TCP_CONNECT) {   
   s = tcp_connect(url.host, 80); // May longjmp   
   memo_set(rr_ctx.current_memo, STEP_TCP_CONNECT, &s);   
   } else {   
   s = *(Socket*)memo_get(rr_ctx.current_memo, STEP_TCP_CONNECT);   
   }   
      
      
   [continued in next message]   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)   
|
[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]
(c) 1994, bbs@darkrealms.ca