From: cross@spitfire.i.gajendra.net
In article ,
James Harris wrote:
>On 12/12/2023 23:56, Scott Lurndal wrote:
>> James Harris writes:
>>> On 23/03/2023 19:49, Dan Cross wrote:
>>>> In article ,
>>>> Scott Lurndal wrote:
>>>>> cross@spitfire.i.gajendra.net (Dan Cross) writes:
>>>
>>> ...
>>>
>>>>>> It was never clear to me
>>>>>> how a hypervisor could, in general, know the format of the guest
>>>>>> page tables. I know the Disco folks had to make some changes to
>>>>>> Irix to get it to work.
>>>>>
>>>>> When I was working on IRIX, I was not fond of either the software
>>>>> managed TLB, coloring or the Kseg stuff; the MIPS project I worked on
was called
>>>>> Teak and was a distributed version of Irix (eventually cancelled)
>>>>> for networks of R10k boxes.
>>>>
>>>> I get it from a hardware perspective: fewer transistors with a
>>>> software-managed TLB, but man...so many drawbacks.
>>>
>>> Handling a software-managed TLB may be more work, in a sense, but it
>>> gives an OS developer more control, more feedback, more freedom, and
>>> perhaps better opportunities for performance gains - as long as the TLB
>>> is large enough.
>>>
>>> Having the hardware carry out a walk of page tables (the only option if
>>> the TLB can is updated by hardware) has long seemed to me like a bad
>>> idea, and it doesn't scale very well as addresses get wider.
>>
>> Having worked extensively with both models (SW: MIPS, HW: pretty much
>> every other single mass-produced microprocessor), there is, hands down,
>> no benefit to software table walks. Zero. Zilch. Don't even bother.
>
>On the contrary!
>
>Brendan (BGB) has already covered hardware issues and I respect his
>views and experience on that.
I plonked him a long time ago because he seems like a crank.
Has he ever worked on a real, non-hobbyist system?
I trust Waterman a lot more. He chose hardware page tables for
RISC-V. There's a reason for that (see the RISC-V reader).
>The /software/ problem with forward page
>tables is that while they are OK (but still unnecessary) for small
>numbers of processes and concomitant address spaces they don't work so
>well for larger systems.
Evidence suggests otherwise.
>Forward page tables take up a significant amount of RAM - as a rule of
>thumb let's say up to 1/1000th of the address space, e.g. up to 4M for a
>4G address space on x32. While that's no problem for a small number of
>processes and the impact can be reduced in many cases it's still
>unavoidably true that each time you add a new process you have to add a
>new set of page tables. Have 5,000 processes and you need 5,000 sets of
>forward page tables. The whole idea is, to use a technical term,
>bonkers. As I said, the approach does not scale.
Evidence shows that it scales, and scales pretty darn well.
A soft TLB simply punts the problem to software; software still
has to maintain a mapping of virtual to physical addresses for
each process. While it is true that it need not use an
architecturally-defined format for that mapping, it must still
maintain _some_ mapping that is, by definition, proportional to
the size of the process's memory footprint.
>By contrast, with reverse page tables you need just ONE table per
>machine, no matter how many processes you want to support. Instead of
>taking 1/1000th of the address space for every process one would use,
>say, 1/500th of the RAM for all processes combined.
First, inverted page tables are not the same thing as software-
managed TLBs; the two may be used in concert, but one does not
automatically imply the other. Second, on very large memory
systems, inverted page tables are wasteful since there's an
entry for each frame of physical memory (and it's conceivable
that not all of physical memory is present!). Third, a single
inverted page table as a global system resource can become a
point of contention in workloads where a process's virtual
address space is being modified frequently. Fourth, fast access
based on a virtual address still requires some sort of lookup
structure that is per-process (since a virtual address space is
presumably a per-process feature of a system).
>I acknowledge that each process will need forward mappings but can
>arrange to reserve them in runs of (first, last) addresses for much
>greater space efficiency than you'd get with forward page tables.
Not really.
>> Hardware translation table walks scale rather well, and in modern
>> incarnations (e.g. ARMv8) are very flexible, supporting multiple
>> fundamental unit of translation sizes (e.g. 4k, 16k, 64k) and
>> higher level "huge pages". Add in the second level of walks
>> required for hardware VM guest page table walkers[*] and the software
>> walker becomes fragile and slow. The hardware walkers have
>> things like content addressible memory and intermediate translation
>> walk caches that software cannot do as effectively or efficiently.
>
>If one handles address-space allocation and mapping to physical
>addresses in software then one is not limited to 4k, 16k, 4M etc ranges.
Eh? that's a function of the MMU, even in systems with soft
TLBs.
>As for the time taken there should be relatively few table walks
>compared with the number of memory accesses.
What makes you think that?
>Software can arrange to access a reverse page table in any way it wants,
>including something with a hashing primary stage.
>
>At least at this point I'd rather see the CPU have instructions which
>can help make the reverse lookup faster than it require a page table for
>every process.
>
>Overall, it's an idea worth considering. YMMV but as you may gather,
>what I do take issue with you on is your blanket comment that there are
>no benefits...!
You're conflating software-managed TLBs with inverted page
tables, which is already a mistake. Otherwise, I don't see any
evidence to back up your claim, but I see a lot for the inverse.
- Dan C.
--- SoupGate-Win32 v1.05
* Origin: you cannot sedate... all the things you hate (1:229/2)
|
