DEP settings with Server Operating Systems

STB Suite | The Industry Standard in Peripheral Testing.

Introduction

Data Execution Prevention (DEP) is a security feature that is indented to prevent viruses or malicious exploits from corrupting files on your computer system. There are hardware-enforced DEP methods and software-enforced DEP.

The STB Suite requires certain device drivers be installed on the target test system. This is the type of activity that makes DEP “nervous”, and so it is required that DEP be set to a mode that will allow our drivers to be installed.

This DEP setting must be correctly set before installing the STB Suite. If not set correctly the STB Suite installation will abort.

The correct DEP setting is “Turn on DEP for essential Windows programs and services only

The incorrect DEP setting is “Turn on DEP for all programs and services…”

Chkinstall.exe

You will find the executable chkinstall.exe in your install folder. Run this program to check the integrity of your installation package and to check your DEP settings.

Note:

With most 64-bit versions of operating systems it is not possible for a program like chkinstall.exe to determine the systems DEP settings. In this case you will see a warning to that effect and you should follow the instructions below to be certain that DEP is set correctly.

Windows 7 and Server 2003/2008

These operating systems make setting DEP relatively easy. Here are the steps to check and/or set DEP before installing the STB Suite:

  1. Press the Start + Break keys to enter Control Panel System
  2. On the left of the System screen click “Advanced system settings” to bring up the System Properties page
  3. Click on the Data Execution Prevention tab
  4. Check to make sure that the top radio button is chosen as shown above
  5. If the top choice is not picked then pick it now, then click Apply
  6. Reboot your system now. DEP setting changes will not take effect until you have rebooted.

Server 2012, and some other 64-bit OS’s

These OS’s seem to have an option that will disallow the above DEP check/change process from being run. This is set as a security option, and the results will be that the DEP options described above will be grayed-out and cannot be changed.

There is information available on the web that would lead you to believe that you could achieve the proper DEP settings by using bcdedit via the command

bcdedit/set nx AlwaysOff

However, this does not actually work!

What you must do is follow these steps to first enable the System Properties->DEP dialog

  1. Open an elevated command prompt window. This can be done by pressing Start and opening All Programs->Accessories.
  2. Right-click on  Command Prompt and choose Run As Administrator. This will open a command prompt window
  3. Now enter bcdedit /set {current} ns OptOut
  4. Reboot your system
  5. Follow steps 1-6 above
  6. Reboot one last time
  7. Now you can install STB Suite