If you came here, you probably wonder how to increase the CUDA kernel timeout under windows. This mechanism of “recovering” the graphics driver is in its base not a bad idea. Users are not forced to manually reset the whole computer in case of driver or application failure. Drawback of this functionality comes in scenarios, where an app is still running and working correctly, however it just requires more time to finish computation. In this scenario, windows after 2 seconds (Default) resets the graphics driver thinking that the app has stopped responding. This is not a wanted scenario and one of the recommendations is to just cut the execution time of a kernel by splitting the calculation into parts. Even I was forced to do so one time, however for example cuda-memcheck can produce very high execution times and if you are debugging an application and want to check memory addressing, you will eventually face the need to increase the Timeout.

The easiest way to do so is to use the nVidia Nsight monitor and change (or even disable) the timeout under options. This solution worked for me, but in any case,I would recommend rather increasing the timeout delay rather than completely disabling the TDR, as this can lead to a frozen system. 

For additional info,please visit: developer.nvidia.com


Share This