How to Enable and Test NVIDIA NVLink on Quadro and GeForce RTX Cards in Windows 10

How to Enable and Test NVIDIA NVLink on Quadro and GeForce RTX Cards in Windows 10

Introduction NVLink is a technology from NVIDIA to create a large bandwidth link between two of their video cards. It can be used for many things, from the basic SLI to even the fastest games to include potential GPU memory to deliver large and complex scenes. The use of NVLink depends on how developers write their applications, and there are many exciting possibilities for this technology. 

 However, starting with this deployment time, there is no easy way to see if NVLink is enabled and running under Windows 10. Furthermore, the way to enable NVLink varies depending on the type of video cards you have. We've developed this guide to help: covering different ways to enable NVLink, along with a small tool to test it easily and make sure it works as expected. 
 If you have already set up your system for NVLink, you can skip to download and run the NVLinkTest utility. 

Install the NVLink Physical Bridge

Regardless of the cards you use, to operate NVLink, you must be connected to an actual NVLink bridge (or, in some cases, bridges). The bridge must be the correct size to reach between the cards, which can be 2, 3 or 4 separate PCI-E slots.   In addition to being the right size, the bridge you use must be compatible with video cards. For example, the older Quadro GP100 bridges do not work on newer GeForce RTX series cards. On the other hand, the GeForce bridges seem to work on Quadro GP100 cards - but they are physically larger, so you can only install one bridge instead of the GP100 designed to use. The safest option is to stick to the bridges specifically designed for the video cards you use. Some other groups may work, but are not likely to be officially supported by NVIDIA.  
Here are examples of what NVLink bridges can look like, as well as what they look like on your cards:   NVLink Bridges from Quadro GP100 and GeForce RTX cards   NVLink Bridges from Quadro GP100 and GeForce RTX cards   Install NVLink Bridge on two 2080 GeForce RTX cards   Install NVLink Bridge on two 2080 GeForce RTX cards   

Enable NVLink on GeForce RTX video cards

Once the physical connector is installed securely, it is very easy - although it is not clear - how to enable NVLink on GeForce RTX series cards. While NVLink is not mentioned in the NVIDIA control panel (as far as it can be found), all you have to do is enable the SLI. To do this, open the NVIDIA Control Panel, navigate to "Configure SLI, Surround, PhysX" under "3D Settings" in the menu tree on the left, select "Maximize 3D performance" under "Initialize SLI", and click Apply. Here's what this screen should look like after SLI is enabled: 
 SLI is used mostly for games, and we have recommended in the past to avoid other applications - but if you want to use NVLink, this is the way to activate them on GeForce RTX 2080 and 2080 Ti cards. Once enabled, you can test to make sure it works. 

Enable NVLink on the Quadro GP100 and GV100 cards

NVLink originally appeared on the Quadro GP100 video cards, as found in the GV100. Both models require a pair of NVLink bridges for full performance, as well as a more complex setup than simply running the SLI: 
1) You must install a third video card - Ideally the Quado of the same generation or newer of the cards you are bridging. This is required for video output, because enabling NVLink on GP100 and GV100 cards will stop video output. 
2) With both the physical bridges and the third card connected to the monitor (screens) installed, open the Windows command line. One easy way to do this is to right-click the Start icon (not the normal left button) and choose Command Prompt. 
3) Once opened, go to "C: \ Program Files \ NVIDIA Corporation \ NVSMI" 
4) Run "nvidia-smi.exe -s" to see which numbers (starting from 0) are assigned to the two video cards that you want. 
5) For each of these cards, run "nvidia-smi.exe -i # -dm TCC", where # is the GPU number you want to obtain in NVLink.
 6) Once this command is successful on both cards, reboot the system and test it to see if NVLink is working. As mentioned before, TCC mode disables direct video output from these modules - so you need to connect any screens with additional cards. 
 TCC mode is enabled on Quadro GP100 video cards 
This is how TCC is enabled on the Quadro GP100s via the command line in Windows 10.
 Download NVLinkTest to check NVLink functionality As of this article, there is no way to check the status of NVLink in the NVIDIA Control Panel. However, NVIDIA does provide some code samples in the CUDA Toolkit that they can verify.