vGPU_Unlock_mirror/vgpu_unlock
1
0
Fork 0
mirror of https://github.com/DualCoder/vgpu_unlock.git synced 2025-08-05 04:16:24 -07:00
Code Issues Projects Releases Wiki Activity

feat: update readme with patching instructions

Browse source
This commit is contained in:
Igor Moura 2021-04-14 20:39:27 -03:00
parent 886c1cfe61
commit d4561a43e0
No known key found for this signature in database
GPG key ID: 31EE944948C858C7
1 changed files with 34 additions and 21 deletions
Download patch file Download diff file Expand all files Collapse all files

55
README.md
View file

@ -1,13 +1,11 @@
# vgpu\_unlock # vgpu_unlock
Unlock vGPU functionality for consumer grade GPUs. Unlock vGPU functionality for consumer grade GPUs.
## Important! ## Important!
This tool is very untested, use at your own risk. This tool is very untested, use at your own risk.
## Description ## Description
This tool enables the use of Geforce and Quadro GPUs with the NVIDIA vGPU This tool enables the use of Geforce and Quadro GPUs with the NVIDIA vGPU
@ -15,23 +13,33 @@ software. NVIDIA vGPU normally only supports a few Tesla GPUs but since some
Geforce and Quadro GPUs share the same physical chip as the Tesla this is only Geforce and Quadro GPUs share the same physical chip as the Tesla this is only
a software limitation for those GPUs. This tool aims to remove this limitation. a software limitation for those GPUs. This tool aims to remove this limitation.
## Dependencies: ## Dependencies:
* This tool requires Python3, the latest version is recommended. - This tool requires Python3, the latest version is recommended.
* The python package "frida" is required. `pip3 install frida`. - The python package "frida" is required. `pip3 install frida`.
* The tool requires the NVIDIA GRID vGPU driver. - The tool requires the NVIDIA GRID vGPU driver.
* "dkms" is required as it simplifies the process of rebuilding the - "dkms" is required as it simplifies the process of rebuilding the
driver alot. Install DKMS with the package manager in your OS. driver alot. Install DKMS with the package manager in your OS.
## Installation: ## Installation:
### Easy patch:
In the following instructions, it's assumed that you have the `vgpu_unlock` folder under `/opt/`. As in, the file `vgpu_unlock_hooks.c` should be located at `/opt/vgpu_unlock/vgpu_unlock_hooks.c`. If that's not the case, then proceed with the manual installation bellow.
- Make sure that both the NVIDIA vGPU-KVM driver installer (the `.run` file) and the `vgpu_unlock.patch` are in the same folder.
- Apply the patch with `sh NVIDIA-INSTALLER.run --apply-patch vgpu_unlock.patch`. This should create a new file ending with `-custom.run`.
- Install the driver using the new custom installer: `sh NVIDIA-INSTALLER-custom.run --dkms`.
- Reboot.
### Manually:
In the following instructions `<path_to_vgpu_unlock>` need to be replaced with In the following instructions `<path_to_vgpu_unlock>` need to be replaced with
the path to this repository on the target system and `<version>` need to be the path to this repository on the target system and `<version>` need to be
replaced with the version of the NVIDIA GRID vGPU driver. replaced with the version of the NVIDIA GRID vGPU driver.
Install the NVIDIA GRID vGPU driver, make sure to install it as a dkms module. Install the NVIDIA GRID vGPU driver, make sure to install it as a dkms module.
``` ```
./nvidia-installer --dkms ./nvidia-installer --dkms
``` ```
@ -39,11 +47,13 @@ Install the NVIDIA GRID vGPU driver, make sure to install it as a dkms module.
Modify the line begining with `ExecStart=` in `/lib/systemd/system/nvidia-vgpud.service` Modify the line begining with `ExecStart=` in `/lib/systemd/system/nvidia-vgpud.service`
and `/lib/systemd/system/nvidia-vgpu-mgr.service` to use `vgpu_unlock` as and `/lib/systemd/system/nvidia-vgpu-mgr.service` to use `vgpu_unlock` as
executable and pass the original executable as the first argument. Ex: executable and pass the original executable as the first argument. Ex:
``` ```
ExecStart=<path_to_vgpu_unlock>/vgpu_unlock /usr/bin/nvidia-vgpud ExecStart=<path_to_vgpu_unlock>/vgpu_unlock /usr/bin/nvidia-vgpud
``` ```
Reload the systemd daemons: Reload the systemd daemons:
``` ```
systemctl daemon-reload systemctl daemon-reload
``` ```
@ -51,22 +61,26 @@ systemctl daemon-reload
Modify the file `/usr/src/nvidia-<version>/nvidia/os-interface.c` and add the Modify the file `/usr/src/nvidia-<version>/nvidia/os-interface.c` and add the
following line after the lines begining with `#include` at the start of the following line after the lines begining with `#include` at the start of the
file. file.
``` ```
#include "<path_to_vgpu_unlock>/vgpu_unlock_hooks.c" #include "<path_to_vgpu_unlock>/vgpu_unlock_hooks.c"
``` ```
Modify the file `/usr/src/nvidia-<version>/nvidia/nvidia.Kbuild` and add the Modify the file `/usr/src/nvidia-<version>/nvidia/nvidia.Kbuild` and add the
following line at the bottom of the file. following line at the bottom of the file.
``` ```
ldflags-y += -T <path_to_vgpu_unlock>/kern.ld ldflags-y += -T <path_to_vgpu_unlock>/kern.ld
``` ```
Remove the nvidia kernel module using dkms: Remove the nvidia kernel module using dkms:
``` ```
dkms remove -m nvidia -v <version> --all dkms remove -m nvidia -v <version> --all
``` ```
Rebuild and reinstall the nvidia kernel module using dkms: Rebuild and reinstall the nvidia kernel module using dkms:
``` ```
dkms install -m nvidia -v <version> dkms install -m nvidia -v <version>
``` ```
@ -74,12 +88,12 @@ dkms install -m nvidia -v <version>
Reboot. Reboot.
--- ---
**NOTE** **NOTE**
This script will only work if there exists a vGPU compatible Tesla GPU that This script will only work if there exists a vGPU compatible Tesla GPU that
uses the same physical chip as the actual GPU being used. uses the same physical chip as the actual GPU being used.
--- ---
## How it works ## How it works
@ -92,18 +106,18 @@ ID is unique for each type of PCI device. In order to enable vGPU support we
need to tell the driver that the PCI device ID of the installed GPU is one of need to tell the driver that the PCI device ID of the installed GPU is one of
the device IDs used by a vGPU capable GPU. the device IDs used by a vGPU capable GPU.
### Userspace script: vgpu\_unlock ### Userspace script: vgpu_unlock
The userspace services nvidia-vgpud and nvidia-vgpu-mgr uses the ioctl syscall The userspace services nvidia-vgpud and nvidia-vgpu-mgr uses the ioctl syscall
to communicate with the kernel module. Specifically they read the PCI device ID to communicate with the kernel module. Specifically they read the PCI device ID
and determines if the installed GPU is vGPU capable. and determines if the installed GPU is vGPU capable.
The python script vgpu\_unlock intercepts all ioctl syscalls between the The python script vgpu_unlock intercepts all ioctl syscalls between the
executable specified as the first argument and the kernel. The script then executable specified as the first argument and the kernel. The script then
modifies the kernel responses to indicate a PCI device ID with vGPU support modifies the kernel responses to indicate a PCI device ID with vGPU support
and a vGPU capable GPU. and a vGPU capable GPU.
### Kernel module hooks: vgpu\_unlock\_hooks.c ### Kernel module hooks: vgpu_unlock_hooks.c
In order to exchange data with the GPU the kernel module maps the physical In order to exchange data with the GPU the kernel module maps the physical
address space of the PCI bus into its own virtual address space. This is done address space of the PCI bus into its own virtual address space. This is done
@ -111,7 +125,7 @@ using the ioremap\* kernel functions. The kernel module then reads and writes
data into that mapped address space. This is done using the memcpy kernel data into that mapped address space. This is done using the memcpy kernel
function. function.
By including the vgpu\_unlock\_hooks.c file into the os-interface.c file we can By including the vgpu_unlock_hooks.c file into the os-interface.c file we can
use C preprocessor macros to replace and intercept calls to the iormeap and use C preprocessor macros to replace and intercept calls to the iormeap and
memcpy functions. Doing this allows us to maintain a view of what is mapped memcpy functions. Doing this allows us to maintain a view of what is mapped
where and what data that is being accessed. where and what data that is being accessed.
@ -127,9 +141,9 @@ to let us know where that section begins and ends.
### How it all comes together ### How it all comes together
After boot the nvidia-vgpud service queries the kernel for all installed GPUs After boot the nvidia-vgpud service queries the kernel for all installed GPUs
and checks for vGPU capability. This call is intercepted by the vgpu\_unlock and checks for vGPU capability. This call is intercepted by the vgpu_unlock
python script and the GPU is made vGPU capable. If a vGPU capable GPU is found python script and the GPU is made vGPU capable. If a vGPU capable GPU is found
then nvidia-vgpu creates an MDEV device and the /sys/class/mdev\_bus directory then nvidia-vgpu creates an MDEV device and the /sys/class/mdev_bus directory
is created by the system. is created by the system.
vGPU devices can now be created by echoing UUIDs into the `create` files in the vGPU devices can now be created by echoing UUIDs into the `create` files in the
@ -137,7 +151,7 @@ mdev bus representation. This will create additional structures representing
the new vGPU device on the MDEV bus. These devices can then be assigned to VMs, the new vGPU device on the MDEV bus. These devices can then be assigned to VMs,
and when the VM starts it will open the MDEV device. This causes nvidia-vgpu-mgr and when the VM starts it will open the MDEV device. This causes nvidia-vgpu-mgr
to start communicating with the kernel using ioctl. Again these calls are to start communicating with the kernel using ioctl. Again these calls are
intercepted by the vgpu\_unlock python script and when nvidia-vgpu-mgr asks if intercepted by the vgpu_unlock python script and when nvidia-vgpu-mgr asks if
the GPU is vGPU capable the answer is changed to yes. After that check it the GPU is vGPU capable the answer is changed to yes. After that check it
attempts to initialize the vGPU device instance. attempts to initialize the vGPU device instance.
@ -148,7 +162,7 @@ The kernel module maps the physical PCI address range 0xf0000000-0xf1000000 into
its virtual address space, it then performs some magical operations which we its virtual address space, it then performs some magical operations which we
don't really know what they do. What we do know is that after these operations don't really know what they do. What we do know is that after these operations
it accesses a 128 bit value at physical address 0xf0029624, which we call the it accesses a 128 bit value at physical address 0xf0029624, which we call the
magic value. The kernel module also accessses a 128 bit value at physical magic value. The kernel module also accessses a 128 bit value at physical
address 0xf0029634, which we call the key value. address 0xf0029634, which we call the key value.
The kernel module then has a couple of lookup tables for the magic value, one The kernel module then has a couple of lookup tables for the magic value, one
@ -170,7 +184,7 @@ to generate a valid HMAC-SHA256 signature and the AES-128 decrypted data blocks
has to contain a vGPU capable PCI device ID. If any of these checks fail, then has to contain a vGPU capable PCI device ID. If any of these checks fail, then
the error code 0x56 "Call not supported" is returned. the error code 0x56 "Call not supported" is returned.
In order to make these checks pass the hooks in vgpu\_unlock\_hooks.c will look In order to make these checks pass the hooks in vgpu_unlock_hooks.c will look
for a ioremap call that maps the physical address range that contain the magic for a ioremap call that maps the physical address range that contain the magic
and key values, recalculate the addresses of those values into the virtual and key values, recalculate the addresses of those values into the virtual
address space of the kernel module, monitor memcpy operations reading at those address space of the kernel module, monitor memcpy operations reading at those
@ -180,4 +194,3 @@ the signature and data bocks, validate the signature, decrypt the blocks, edit
the PCI device ID in the decrypted data, reencrypt the blocks, regenerate the the PCI device ID in the decrypted data, reencrypt the blocks, regenerate the
signature and insert the magic, blocks and signature into the table of vGPU signature and insert the magic, blocks and signature into the table of vGPU
capable magic values. And that's what they do. capable magic values. And that's what they do.