From: cross@spitfire.i.gajendra.net   
      
   In article ,   
   Scott Lurndal  wrote:   
   >cross@spitfire.i.gajendra.net (Dan Cross) writes:   
   >>In article ,   
   >>Scott Lurndal  wrote:   
   >>>cross@spitfire.i.gajendra.net (Dan Cross) writes:   
   >>>[snip]   
   >>>>It's still not entirely clear to me now the BSP/BSC is supposed   
   >>>>to boot, however.  If the world starts in 64-bit mode, and that   
   >>>>still requires paging to be enabled, then who sets up the page   
   >>>>tables that the BSP starts up on?   
   >>>   
   >>>I haven't dug into it, but perhaps they come up in some funky   
   >>>identity mode when the PT root pointer (CR3?) hasn't been programmed.   
   >>   
   >>Now that would genuinely be a useful change.   
   >   
   >The document describes IA32_SIPI_ENTRY_STRUCT as containing:   
   >   
   >  - A bit that selects startup or shutdown (+63 filler bits)   
   >  - The RIP for the AP to start executing at   
   >  - The a CR3 value for the AP to use when starting   
   >  - The CR0 value   
   >  - The CR4 value   
   >   
   >This is used to start all the secondary processors.   
      
   Yeah, the AP (secondary processor) case was pretty clear.   
      
   >The boostrap processor:   
   >   
   >  "The CPU starts executing in 64-bit paged mode after reset. The   
   >   Firmware Interface Table (FIT) contains a reset state structure   
   >   containing RIP and CR3 that defines the initial execution state of   
   >   the CPU."   
   >   
   >I'm presuming that a management processor provided by the   
   >mainboard vendor will initialize the FIT out-of-band before   
   >releasing the BSP from reset sufficent to execute the UEFI   
   >firmware and boot loader.   
      
   Ah, the AMD PSP approach, aka, how to start the engine on a   
   ship.  I guess the days of DRAM training from the BSP are over,   
   which isn't necessarily a bad thing.   
      
   I imagine that they'll just embed this into the SoC complex   
   directly.   
      
   >>>>>We do that all the time on our processors.  Applications like DPDK   
   >>>>>and Open Data Plane (ODP) rely on user-mode access to the   
   >>>>>device MMIO (often using SR-IOV virtual functions) space and direct   
   >>>>>DMA (facilitated by an IOMMU/SMMU) initiated by usermode code.   
   >>>>   
   >>>>Ok, sure, but that's not PIO.   
   >>>   
   >>>By PIO are you referring to 'in' and 'out' instructions that have   
   >>>been obsolete for three decades except for a few legacy devices   
   >>>like the UART   
   >>   
   >>Well, yes.  (The context was the removal of both ring 3 port   
   >>access instructions, as well as the IOPL from TSS.)   
   >   
   >Ok.  Coming from the unix/linux world, ring 3 access to those   
   >has generally not been allowed and I don't see removal of that   
   >capability as a loss, but rather a gain.   
      
   ioperm(2) and iopl(2)? :-)   
      
   >>>(and access to pci config space, although PCI   
   >>>express defines the memory mapped ECAM as an alternative which   
   >>>is used on non-intel/amd systems)?   
   >>   
   >>I try to blot that out of my mind.   
   >>   
   >>I believe that PCI express deprecates the port-based access   
   >>method to config space; MMIO _must_ be supported and in   
   >>particular, is the only way to get access to the extended   
   >>capability space.   
   >   
   >Intel cheated and used four unused bits in the high bits   
   >of cf8 for the extended config space on some of the southbridge   
   >chipsets in the early PCIe days.  I've not checked recently   
   >to see if newer chipsets/SoCs still support that.   
      
   Sigh.  Actually, I checked the PCIe spec last night and I think   
   I was just wrong.  It looks like you can still do it, but   
   address/data register pairs are problematic, so I always just   
   use ECAM.   
      
   I think port IO can be useful at very early boot for writing   
   debugging data to the UART, or for supporting legacy "PC"   
   devices; beyond that, I find it annoying.  We did use it once in   
   an experimental hypervisor to execute the equivalent of a VMCALL   
   without first having to trap into (guest) kernel mode.   
      
   >> So from that perspective we're not losing   
   >>anything.  Certainly, I've used memory-mapped IO for dealing   
   >>with PCI config space on x86_64.  The port-based access method   
   >>is really only for compatibility with legacy systems at this   
   >>point.   
   >   
   >For ARM, the standard is ECAM (See Server Base System Architecture   
   >document).   
   >   
   >https://developer.arm.com/documentation/den0029/latest/   
      
   ARM has always been saner than x86.  :-)  It makes sense that   
   they'd start with and stick to ECAM, since (AFAIK) they never   
   had programmed IO instructions.  How else _would_ you do it?   
   (That's a rhetorical question, btw.)   
      
   >>Yup.  It really seems like the only devices that demand access   
   >>via port IO are the legacy "PC" devices; if the 8159A is going   
   >>away, what's left?  The RTC, UART and keyboard controller?  Is   
   >>the PIT dual-wired to an IOAPIC for interrupt generation?   
   >   
   >Don't the have an architected high precision timer (HPET) that   
   >is used instead of the PIT in these modern times?   
      
   It does, but the HPET has weird problems of its own and is more   
   rarely used than one might otherwise expect.  It's a really   
   annoying device in a lot of ways.   
      
   >>>>>Interrupts are still mediated by the OS (virt-io provides these   
   >>>>>capabilities), although DPDK/ODP generally poll completion rings   
   >>>>>rather than use interrupts.   
   >>>>   
   >>>>Really?  Even with SR-IOV and the interrupt remapping tables in   
   >>>>the IOMMU?  Are you running in VMX non-root mode?  Why not use   
   >>>>posted interrupts?   
   >>>   
   >>>Hmm. I do seem to recall some mechanisms for interrupt virtualization   
   >>>in the IOMMU, but I've been, as noted above, in the ARMv8 world for a while   
   now.   
   >>   
   >>Is this the point where I express my jealousy?  :-D   
   >   
   >I've quite enjoyed the last decade working on a   
   >significant architectural upgrade from ARMv7.   
   >   
   >Watching the architecture grow from initial early   
   >release documents and modeling the Processor, SMMU, and   
   >Interrupt Controller has been educational and fun.   
      
   Definitely some really cool things have happened in that space   
   while those of us slumming it with x86 have looked on with envy.   
      
   >>>Speaking for ARM systems, the guts of the interrupt controller   
   >>>(including the interrupt acknowledge registers) are privileged. There   
   >>>is no way to segregate user-mode-visible interrupts from all others   
   >>>which is needed to ensure that a user-mode program can't royally screw   
   >>>up the system, the kernel must accept and end the interrupt.   
   >>   
   >>I'm not sure I understand; I thought the GIC was memory mapped,   
   >>including for the banked per-CPU registers?   
   >   
   >That was GICv2.  That interface only supported 8 cores/threads,   
   >so they designed GICv3 for ARMv8.   That uses CPU system registers   
   >to interface rather than the former memory mapped CPU interface.   
   >Much cleaner and easily accomodates many thousands of CPUs; also   
   >expanded to handle large numbers (2^22) of interrupts, including   
   >software generated interrupts for IPI, per-processor local interrupts   
   >for things like timers, profiling, debugging, wired interrupts   
   >(level or edge) and message signaled interrupts (edge only).   
   >   
      
   [continued in next message]   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)