From: user5857@newsgrouper.org.invalid
scott@slp53.sl.home (Scott Lurndal) posted:
> MitchAlsup writes:
> >
> >Greetings everyone !
> >
>
> >Chip resources {Cores, on-Die Interconnect, {L3, DRAM}, {HostBridge,
> >I/O MMU, PCIe Segmenter}} have the first 32 DoubleWords of the
> >assigned MM I/O space defined as a "file" containing R0..R31. In all
> >cases:
> >R0 contains the Voltage and Frequency control terms of the resource,
> >R1..R27 contains any general purpose control registers of resource.
> >R28..R30 contains the debug port,
> >R31 contains the Performance Counter port.
> >The remaining 480 DoubleWords are defined by the resource itself
> >(or not).
>
> I'd allow for regions larger than 4096 bytes. It's not uncommmon
> for specialized on-board DMA engines to require 20 bits of
> address space to define the complete set of device resources,
> even for on-chip devices (A DMA engine may support a large number
> of "ring" structures, for example, and one might group the
> ring configuration registers into 4k regions (so they can be assigned
> to a guest in a SRIOV-type device)).
This is a fair point, but none of my current on-Die need more than 6 DWs,
So allocating a 4096 byte address space to them seems generous--remember
these are NOT PCIe peripherals, but resources in a My 66000 implementation.
They all use std PCIe confi headers so a STD #-1 to BAR{01} and a LDD
tells how big the allocation should be, so even tho the spaces are sparse
they still follow PCIe Config conventions.
> I've seen devices with dozens of performance registers (both
> direct-access and indirect-access).
>
>
> >
> >Because My 66000 ISA has memory instructions that "touch" multiple
> >memory locations, these instructions take on special significance
> >when using the debug and performance counter ports. Single memory
> >instructions access the control registers themselves, while multi-
> >memory instructions access "through" the port to the registers
> >the port controls.
>
> That level of indirection may cause difficulties when virtualizing
> a device.
These are on-Die resources not PCIe peripherals.
> >
> >For example: each resource has 8 performance counters and 1 control
> >register (R31) governing that port.
> >a STB Rd,[R31] writes a selection into the PC selectors
> >a STD Rd,[R31] writes 8 selections into the PC selectors
> >a LDB Rd,[R31] reads a selection from a PC selectors
> >a LDD Rd,[R31] reads 8 selections from the PC selectors
> >while:
> >a LDM Rd,Rd+7,[R31] reads 8 Performance Counters,
> >a STM Rd,Rd+7,[R31] writes 8 Performance Counters,
> >a MS #0,[R31],#64 clears 8 Performance Counters.
> >
> >The Diagnostic port provides access to storage within the resource.
> >R28 is roughly the "address" control register
> >R29 is roughly the "data" control register
> >R30 is roughly the "other" control register
> >For a Core; one can access the following components from this port:
> >ICache Tag
> >ICache Data
> >ICache TLB
> >DCache Tag
> >DCache Data
> >DCache TLB
> >Level-1 Miss Buffer
> >L2Cache Tag
> >L2Cache Data
> >L2Cache TLB
> >L2Cache MMU
> >Level-2 Miss Buffer
> >
> >Accesses through this port come in single-memory and multi-memory
> >flavors. Accessing these control registers as single memory actions
> >allows raw access to the data and associated ECC. Reads tell you
> >what HW has stored, writes allow SW to write "bad" ECC, should it
> >so choose. Multi-memory accesses allow SW to read or write cache
> >line sized chunks. The Core tags are configured so that every line
> >has a state where this line neither hits nor participates in set
> >allocation (when a line needs allocated on miss or replacement.)
> >So, a single bad line in a 16KB cache 4-way set looses 64-bytes
> >and one line becomes 3-way set associative.
> >----------------------------
>
> The KISS principle applies.
>
> >By using the fact that cores come out of reset with MMU turned on,
> >and BOOT ROM supplying the translation tables, I was able to achieve
> >that all resources come out of reset with all control register flip-
> >flops = 0, except for Core[0].Hypervisor_Context.v = 1.
>
> Where is the ROM? Modern SoCs have an on-board ROM, which
> cannot be changed without a re-spin and new tapeout. That
> ROM needs to be rock-solid and provide just enough capability
> to securely load a trusted blob from a programmable device
> (e.g. SPI flash device).
ROM is external FLASH in the envisioned implementations.
> I'm really leary about the idea of starting with MMU enabled,
> I don't see any advantage to doing that.
>
> >
> >Core[0] I$, D$, and L2$ come out of reset in the "allocated" state,
> >so Boot SW has a small amount of memory from which to find DRAM,
> >configure, initialize, tune the pin interface, and clear; so that
> >one can proceed to walk and configure the PCIe trees of peripherals.
>
> You don't need to configure peripherals before DRAM is initialized
> (other than the DRAM controller itself). All other peripheral
> initialization should be done in loadable firmware or a secure
> monitor, hypervisor or bare-metal kernel.
Agreed, you can't use the peripherals until they have DRAM in which
to perform I/O, and send interrupts{Thus, acting normal}.
> >----------------------------
> >Guest OS can configure its translation tables to emit {Configuration
> >and MM I/O} space accesses. Now that these are so easy to recognize:
>
> Security. Guest OS should only be able to access resources
> granted to it by the HV.
Yes, Guest physical MM I/O Space is translated by Host MM I/O Translation
tables. Real OS setup its translation tables to emit MM I/O Accesses, so
Guest OS should too, then that Guest Physical is considered Host virtual
and translated and protected again.
As far as I am concerned, Guest OS thinks it has 32 Devices each of which
have 8 Functions all on Bus 0... So, a Guest OS with fewer than 256 Fctns
sees only 1 BUS and can short circuit the virtual Config discovery.
These virtual Guest OS accesses, then, get redistributed to the
Segments and Busses on which the VFs actually reside by Level-2 SW.
> >Host OS and HyperVisor have the ability to translate Guest Physical
> >{Configuration and MM I/O} accesses into Universal {Config or MM I/O}
> >accesses. This requires that the PTE KNOW how SR-IOV was set up on
> >that virtual Peripheral.
>
> This seems unnecessarily complicated.
So did IEEE 754 in 1982...
What I have done is to virtualize Config and MM I/O spaces, so Guest OS
does not even see that it is not Real OS running on bare metal--and doing
so without HV intervention on any of the Config or MM I/O accesses.
> Every SR-IOV capable device
> is different and aside the standard PCIe defined configuration space
> registers, everything else is device-specific.
Only requires 3 bits in the MM I/O PTE.
[continued in next message]
--- SoupGate-Win32 v1.05
* Origin: you cannot sedate... all the things you hate (1:229/2)
|
