@leftwo had a VM that got "stuck", and reasonably tried to:
./propolis-cli-boot-order --server 0.0.0.0 --port 12400 inject-nmi
... which seems like a good idea. unfortunately:
panic[cpu0]/thread=fffffe003d005c20: NMI received
fffffffffbc15ed0 fffffffff38cc343 ()
fffffffffbc15f00 unix:av_dispatch_nmivect+32 ()
fffffffffbc15f10 unix:nmiint+155 ()
fffffe003d005ba0 unix:mach_cpu_idle+b ()
fffffe003d005bd0 unix:cpu_idle+103 ()
fffffe003d005be0 unix:cpu_idle_adaptive+19 ()
fffffe003d005c00 unix:idle+a8 ()
fffffe003d005c10 unix:thread_start+b ()
skipping system dump - no dump device configured
rebooting...
BdsDxe: loading Boot0001 "UEFI " from PciRoot(0x0)/Pci(0x4,0x0)/NVMe(0x1,00-00-00-00-00-00-00-
00)
BdsDxe: starting Boot0001 "UEFI " from PciRoot(0x0)/Pci(0x4,0x0)/NVMe(0x1,00-00-00-00-00-00-00
-00)
/
after which point everything resumed and the VM appears happy now.
it would have been nice to know where vCPUs were, or if they were exited and being handled either in bhyve or Propolis, maybe even propolis-server's pstack just for an idea of what was up. I think we should capture this stuff by default from propolis-cli sending an NMI because if you're kicking a guest it's probably gotten borked somehow and it's better to capture a bit extra that may be redundant with guest state than to just lose it because we've faithfully delivered the NMI and the guest did something with it. we might want all this to happen on the propolis-server side, and dropped into ... I dunno, the current working directory? same problem as OVMF debug.out imo.
it probably makes sense to have a --quiet flag that doesn't try capturing VM state at the point of NMI in case you really just want to kick it and don't care about what's going on. that is, as far as I know, the much less common use of NMI'ing a guest, so I really think it ought to be non-default.
@leftwo had a VM that got "stuck", and reasonably tried to:
... which seems like a good idea. unfortunately:
after which point everything resumed and the VM appears happy now.
it would have been nice to know where vCPUs were, or if they were exited and being handled either in bhyve or Propolis, maybe even
propolis-server's pstack just for an idea of what was up. I think we should capture this stuff by default frompropolis-clisending an NMI because if you're kicking a guest it's probably gotten borked somehow and it's better to capture a bit extra that may be redundant with guest state than to just lose it because we've faithfully delivered the NMI and the guest did something with it. we might want all this to happen on thepropolis-serverside, and dropped into ... I dunno, the current working directory? same problem as OVMFdebug.outimo.it probably makes sense to have a
--quietflag that doesn't try capturing VM state at the point of NMI in case you really just want to kick it and don't care about what's going on. that is, as far as I know, the much less common use of NMI'ing a guest, so I really think it ought to be non-default.