From: james.harris.1@gmail.com   
      
   On 12/12/2023 23:56, Scott Lurndal wrote:   
   > James Harris  writes:   
   >> On 23/03/2023 19:49, Dan Cross wrote:   
   >>> In article ,   
   >>> Scott Lurndal  wrote:   
   >>>> cross@spitfire.i.gajendra.net (Dan Cross) writes:   
   >>   
   >> ...   
   >>   
   >>>>>    It was never clear to me   
   >>>>> how a hypervisor could, in general, know the format of the guest   
   >>>>> page tables.  I know the Disco folks had to make some changes to   
   >>>>> Irix to get it to work.   
   >>>>   
   >>>> When I was working on IRIX, I was not fond of either the software   
   >>>> managed TLB, coloring or the Kseg stuff; the MIPS project I worked on was   
   called   
   >>>> Teak and was a distributed version of Irix (eventually cancelled)   
   >>>> for networks of R10k boxes.   
   >>>   
   >>> I get it from a hardware perspective: fewer transistors with a   
   >>> software-managed TLB, but man...so many drawbacks.   
   >>   
   >> Handling a software-managed TLB may be more work, in a sense, but it   
   >> gives an OS developer more control, more feedback, more freedom, and   
   >> perhaps better opportunities for performance gains - as long as the TLB   
   >> is large enough.   
   >>   
   >> Having the hardware carry out a walk of page tables (the only option if   
   >> the TLB can is updated by hardware) has long seemed to me like a bad   
   >> idea, and it doesn't scale very well as addresses get wider.   
   >   
   > Having worked extensively with both models (SW: MIPS, HW: pretty much   
   > every other single mass-produced microprocessor), there is, hands down,   
   > no benefit to software table walks.   Zero.  Zilch.  Don't even bother.   
      
   On the contrary!   
      
   Brendan (BGB) has already covered hardware issues and I respect his   
   views and experience on that. The /software/ problem with forward page   
   tables is that while they are OK (but still unnecessary) for small   
   numbers of processes and concomitant address spaces they don't work so   
   well for larger systems.   
      
   Forward page tables take up a significant amount of RAM - as a rule of   
   thumb let's say up to 1/1000th of the address space, e.g. up to 4M for a   
   4G address space on x32. While that's no problem for a small number of   
   processes and the impact can be reduced in many cases it's still   
   unavoidably true that each time you add a new process you have to add a   
   new set of page tables. Have 5,000 processes and you need 5,000 sets of   
   forward page tables. The whole idea is, to use a technical term,   
   bonkers. As I said, the approach does not scale.   
      
   By contrast, with reverse page tables you need just ONE table per   
   machine, no matter how many processes you want to support. Instead of   
   taking 1/1000th of the address space for every process one would use,   
   say, 1/500th of the RAM for all processes combined.   
      
   I acknowledge that each process will need forward mappings but can   
   arrange to reserve them in runs of (first, last) addresses for much   
   greater space efficiency than you'd get with forward page tables.   
      
   >   
   > Hardware translation table walks scale rather well, and in modern   
   > incarnations (e.g. ARMv8) are very flexible, supporting multiple   
   > fundamental unit of translation sizes (e.g. 4k, 16k, 64k) and   
   > higher level "huge pages".   Add in the second level of walks   
   > required for hardware VM guest page table walkers[*] and the software   
   > walker becomes fragile and slow.      The hardware walkers have   
   > things like content addressible memory and intermediate translation   
   > walk caches that software cannot do as effectively or efficiently.   
      
   If one handles address-space allocation and mapping to physical   
   addresses in software then one is not limited to 4k, 16k, 4M etc ranges.   
      
   As for the time taken there should be relatively few table walks   
   compared with the number of memory accesses.   
      
   Software can arrange to access a reverse page table in any way it wants,   
   including something with a hashing primary stage.   
      
   At least at this point I'd rather see the CPU have instructions which   
   can help make the reverse lookup faster than it require a page table for   
   every process.   
      
   Overall, it's an idea worth considering. YMMV but as you may gather,   
   what I do take issue with you on is your blanket comment that there are   
   no benefits...!   
      
      
   --   
   James Harris   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)