KVM LibVirt

Voir :

• Notes LibVirt
• https://github.com/kholia/OSX-KVM

Votre CPU gère t-il VT ou AMD-V ?

egrep --color '^flags.*(vmx |svm )' /proc/cpuinfo

Ou

apt-get install cpu-checker
kvm-ok

Nested KVM ?

cat /sys/module/kvm_intel/parameters/nested
cat /sys/module/kvm_amd/parameters/nested

Parfois il faut activer l'option dans le BIOS

# virt-host-validate
...
   LXC: Checking if device /sys/fs/fuse/connections exists                   : FAIL (Load the 'fuse' module to enable /proc/ overrides)
   
echo fuse >> /etc/modules
modprobe fuse

Sous RedHat désactiver le NetworkManager

systemctl stop NetworkManager
systemctl disable NetworkManager

et ajouter la ligne suivante dans /etc/sysconfig/network-scripts/ifcfg-*

/etc/sysconfig/network-scripts/ifcfg-*

NM_CONTROLLED=no

apt-get install --no-install-recommends qemu-kvm libvirt-clients libvirt-daemon-system

adduser jean libvirt
#adduser jean kvm

~/.bash_aliases

alias virsh="virsh -c qemu:///system"

ou

~/.bashrc

export LIBVIRT_DEFAULT_URI="qemu:///system"

virsh net-autostart default
virsh net-start default

/etc/default/libvirt-guests

#ON_BOOT=ignore
ON_BOOT=start
 
#START_DELAY=0
START_DELAY=30
 
#PARALLEL_SHUTDOWN=0
PARALLEL_SHUTDOWN=1
 
#SHUTDOWN_TIMEOUT=300
SHUTDOWN_TIMEOUT=180

Pour virt-sysprep (après clonage de VM)

apt-get install libguestfs-tools

apt-get install ovmf

Exemple :

• https://raymii.org/s/articles/virt-install_introduction_and_copy_paste_distro_install_commands.html
• http://serverfault.com/questions/481244/preseed-command-string-fail-with-newline-character-using-virt-install-initrd-inj

virt-install \
--name vm1 \
--ram 512 \
--location=http://ftp.us.debian.org/debian/dists/stable/main/installer-amd64/ \
--initrd-inject=/tmp/preseed.cfg \
--extra-args="console=ttyS0,115200n8 serial locale=fr_FR console-setup/ask_detect=false keyboard-configuration/layoutcode=fr hostname=virtual domain=unassigned-domain interface=auto" \
--vcpu=1 \
--vnc \
--vnclisten=0.0.0.0 \
--os-type=linux \
--os-variant=debianwheezy \
-w bridge=virbr0 \
-w bridge=br1 \
-w network=default \
--console pty,target_type=serial \
--disk format=qcow2,size=8,bus=virtio,path=/var/lib/libvirt/images/vm1.qcow2

#--connect qemu:///system
#--location=http://archive.ubuntu.com/ubuntu/dists/precise/main/installer-amd64/ \
#--noautoconsole \
#--autostart \
#--cdrom=/var/lib/libvirt/images/test.iso \

--extra-args ne marche que si --location est défini.

KVM Réseaux Bridge TUN

http://doc.fedora-fr.org/wiki/Discussion:Virtualisation_:_KVM,_Qemu,_libvirt_en_images

Voir :

• https://blog.zerosector.io/2018/07/28/kvm-qemu-windows-10-gpu-passthrough/
• https://www.deltasight.fr/pci-passthrough-nvidia-gtx-1050-kvm/
• https://jugulaire.github.io/GPU-passthrough-KVM-QEMU/
• https://www.debian-fr.org/t/kvm-et-passthrough-afin-de-se-passer-du-dual-boot/60794/5
• https://pve.proxmox.com/wiki/Pci_passthrough
• https://davidwin93.github.io/vfio_guide/
• https://developers.redhat.com/articles/2024/04/05/enable-gpu-acceleration-kernel-module-management-operator

BIOS/UEFI

Première étape, faire un tour dans le BIOS/UEFI pour activer les choses suivantes :

  VT-D (cas d’un CPU intel)
  AMD-Vi (cas d’un CPU AMD)

Pour vérifier :

# dmesg | grep -e "Directed I/O"
[    1.693161] DMAR: Intel(R) Virtualization Technology for Directed I/O

Pour du AMD :

dmesg | grep AMD-Vi

Dans le BIOS activer “SR-IOV” ?

# ls /sys/class/iommu/*/
/sys/class/iommu/dmar2/:
devices  intel-iommu  power  subsystem  uevent

/sys/class/iommu/dmar4/:
devices  intel-iommu  power  subsystem  uevent

/sys/class/iommu/dmar5/:
devices  intel-iommu  power  subsystem  uevent

/sys/class/iommu/dmar6/:
devices  intel-iommu  power  subsystem  uevent

/sys/class/iommu/dmar7/:
devices  intel-iommu  power  subsystem  uevent

/etc/modprobe.d/blacklist-nouveau.conf

blacklist nouveau
options nouveau modeset=0

/etc/default/grub

#GRUB_CMDLINE_LINUX_DEFAULT="modprobe.blacklist=nouveau modprobe.blacklist=nvidiafb"
#GRUB_CMDLINE_LINUX="intel_iommu=on iommu=pt rd.driver.pre=vfio-pci video=efifb:off vfio_iommu_type1.allow_unsafe_interrupts=1"
 
#GRUB_CMDLINE_LINUX="kvm_iommu=on iommu=pt"
GRUB_CMDLINE_LINUX="intel_iommu=on iommu=pt pci-stub.ids=10de:1b81,10de:10f1 video=efifb:off"

update-grub

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
reboot
dmesg | grep 'remapping'

Pour nvidia

echo "options kvm ignore_msrs=1 report_ignored_msrs=0" > /etc/modprobe.d/kvm.conf

# lspci -nn | grep -i nvidia
3b:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104GL [Quadro P5000] [10de:1bb0] (rev a1)
3b:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
af:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104GL [Quadro P5000] [10de:1bb0] (rev a1)
af:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)

/etc/modprobe.d/vfio.conf

#options vfio-pci ids=10de:1bb0,10de:10f0
options vfio-pci ids=10de:1bb0
options vfio-pci disable_vga=1

echo 'vfio-pci' > /etc/modules-load.d/vfio-pci.conf
update-initramfs -u

# dmesg |grep "Directed I/O"
[    1.705358] DMAR: Intel(R) Virtualization Technology for Directed I/O

#!/bin/bash
shopt -s nullglob
for g in /sys/kernel/iommu_groups/*; do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;

dmesg | grep -i vfio

# lspci -nnk |grep -i -A 2 nvidia
3b:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104GL [Quadro P5000] [10de:1bb0] (rev a1)
        Subsystem: Dell GP104GL [Quadro P5000] [1028:11b2]
        Kernel driver in use: vfio-pci
--
3b:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
        Subsystem: Dell GP104 High Definition Audio Controller [1028:11b2]
        Kernel driver in use: snd_hda_intel
--
af:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104GL [Quadro P5000] [10de:1bb0] (rev a1)
        Subsystem: Dell GP104GL [Quadro P5000] [1028:11b2]
        Kernel driver in use: vfio-pci
--
af:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
        Subsystem: Dell GP104 High Definition Audio Controller [1028:11b2]
        Kernel driver in use: snd_hda_intel

# echo "0000:3b:00.0" | tee /sys/bus/pci/drivers/vfio-pci/unbind
0000:3b:00.0
# echo "0000:3b:00.0" | tee /sys/bus/pci/drivers/vfio-pci/unbind
0000:3b:00.0
tee: /sys/bus/pci/drivers/vfio-pci/unbind: No such device

cd /sys/bus/pci/devices/0000\:3b\:00.0
echo 1 > rom
cat rom > /tmp/gpu.rom
echo 0 > rom

/etc/libvirt/qemu/vm1.xml

   <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x3b' slot='0x00' function='0x0'/>
      </source>
      <rom bar='on'/>
      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0xaf' slot='0x00' function='0x0'/>
      </source>
      <rom bar='on'/>
      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
    </hostdev>