How to Reserve an Virtual Machine with H100 Virtual GPU

This article is for providing information about how to reserve a virtual machine with virtual GPUs created from the limited number of H100 GPUs in the lab.

If you do not specifically need to use the H100 GPU, you can send a general request for computing servers or workstations without specifying the H100. We have plenty of CPU only computing servers and tens of RTX 3070 GPU workstations and RTX 4000 Ada GPU workstations.

If you are a student who wants to use the H100 virtual GPU, you need to send a request using the following post in order to get an approval from your advisor. Please select "Remote Access to Computing Servers or Virtual Machines" and  specifically request H100 virtual GPU in your Request Details.

Through the support from Office of the Provost and School of Business, we had the chance to test and set up servers with H100 GPUs when it was first released. In many cases in research with financial applications, we see significant speed increases when this GPU is used.

However, not all projects can fully use the GPU RAM in the H100 GPU and not many research groups will demand running large-scale GPU jobs everyday. To let more people use the H100 GPU, we deployed the GPU virtualization software from NVIDIA which creates “vGPUs” that each independently reserve some amount of GPU RAM for a short period of time (https://www.nvidia.com/en-us/data-center/virtual-solutions/).

Here are the types of virtual machines that we can provide from the host servers with H100 GPUs:

Type A: 96 CPU cores, 128 GB CPU RAM, 80 GB GPU RAM

Type B: 96 CPU cores, 128 GB CPU RAM, 40 GB GPU RAM

Type C: 96 CPU cores, 128 GB CPU RAM, 20 GB GPU RAM

Type D: 96 CPU cores, 128 GB CPU RAM, 10 GB GPU RAM

Type Z: 96 CPU cores, 128 GB CPU RAM, No GPU

For the vast majority of use cases, it is best to initially reserve a Type Z CPU VM, so you can obtain a virtual machine to set up your job and test it without delay, then reserve the Type A GPU when you need to run the jobs properly.

Type A, B, C and D are types that require reservations to use. When a user is not reserving any of those types, their virtual machine will be converted to Type Z. When the virtual machine needs to be converted, one reboot will be enough. There will be no changes or very little changes needed from the user in their code. The user will be assigned to a specific host and migrations of the user and the virtual machine can be requested through emails to fscadmin@stevens.edu.

Additionally, one of the hosts is an H100NVL GPU with 94 GB of GPU RAM. If you require more RAM than the 80 GB provided by the standard VM, please email us.

There are two hosts with one H100 GPU each. Each H100 GPU can host either one Type A, two Type B, four Type C or eight Type D at any given time. GPU types cannot be mixed on a single host. Be mindful of this if there are existing reservations that may conflict with yours.

Here are the rules for all virtual GPU users of the Hanlon Labs.

1. If this is your first H100 virtual GPU request, you must obtain approval: Students must receive approval from their advisor using the form located at https://fsc.stevens.edu/how-to-request-computing/. Faculty or staff can email us at fscadmin@stevens.edu. We will email you for setting up access.
2. Before submitting a reservation request using the form below, carefully check the H100 GPU reservation calendar below for available numbers and types of virtual machines.
3. Reservations are made on a first-come, first-served basis. If you have an urgent need and there are existing reservations, please email us to discuss your case further.
4. Each user can only reserve one virtual machine for any time period. Please email us for special cases where more than one virtual machine is needed for one user.
5. Check your email for acceptance notifications; only accepted requests are guaranteed.

For any questions, you can contact us at fscadmin@stevens.edu or visit Hanlon Financial Systems Lab for assistance.

Use this form to make reservations. The system automatically approves or declines the reservations based on the rules above.