From: cross@spitfire.i.gajendra.net   
      
   In article ,   
   Scott Lurndal  wrote:   
   >cross@spitfire.i.gajendra.net (Dan Cross) writes:   
   >>In article ,   
   >>Scott Lurndal  wrote:   
   >    
   >>>>Nice. 3Leaf sounds interesting; are there any papers on it   
   >>>>available?   
   >>>   
   >>>There are a couple of patents (one granted, one abandoned)   
   >>>at the USPTO.   Not much else out in the public other than   
   >>>the various trade mag stuff from that time period.   
   >>>   
   >>>Basically we built an ASIC that extends the coherency domain   
   >>>across infiniband (or 10G Ethernet) to allow creation of   
   >>>large shared-memory cache-coherent systems from 1u or 2u   
   >>>building blocks.   The ASIC talked HyperTransport for AMD   
   >>>Opteron CPUs and QuikPath for Intel CPUs.   
   >>>   
   >>>Had a 32-node, 64-processor system at LLNL for evaluation   
   >>>before the bottom dropped out of the markets, an acquision   
   >>>fell through and we shut down.   
   >>>   
   >>>Much the same as CXL-Cache today.  We even considered PCIe   
   >>>as the transport, but the switching latencies for IB were   
   >>>less than 100ns.   
   >>   
   >>Oh wow, very cool.   
   >   
   >The cool part was the software.   A bare-metal hypervisor that   
   >allowed virtual servers to be composed dynamically on the system;   
   >We didn't multiplex guests on any given core, but rather assigned   
   >cores and memory to each guest when the guest is created.  If   
   >the guest supported CPU and Memory hot plug/unplug (e.g. linux),   
   >we could dynamically move processors and memory between guests   
   >as demand required.    I/O was virtualized and handled by one or   
   >more virtual I/O servers over infiniband/10Ge, and as SR-IOV   
   >and MR-IOV were starting to be deployed, we could grant direct   
   >access to a VF to a guest.  We also supported XEN-style virtual   
   >I/O drivers in the guest.  We worked with the XEN folks at   
   >Cambridge in the early days, before SVM was available; The   
   >CTO of AMD  was   
   >on our advisory board which helped with the hypertransport   
   >implementation on the ASIC.   
   >   
   >A copy of the hypervisor (DVMM - Distributed Virtual Machine   
   >Manager) ran on each node and they all communicated via shared   
   >memory.   
      
   That IS cool.   
      
   	- Dan C.   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)