Windows Firewall Part 3: Domain IPSec Configuration

Part 3 of this series will go over the preparation work required to utilize IPSec in the future. This work will allow for the creation of firewall rules that either require authentication, or require authentication and encryption, for greater access control and security. A new top-level GPO will define all of the specific parameters, for ease of changes later. This GPO will only set the domain baseline; the configuration can still be modified further on a case by case basis by using more policies applied closer to the specific endpoint if necessary.

Note: This post assumes a basic knowledge of IPSec functionality, including how the various protocols work together, various algorithms used for encryption and hashing, etc. For more information, please see the following links:

GPO Creation

Start by creating a new GPO linked to the domain root named “Domain IPSec Configuration. In this policy, head to the Windows Firewall section, view the properties, and look at the IPSec Settings tab.

Windows Firewall Properties – IPSec Tab

From here, customize the IPSec defaults by clicking Customize. This will lead to a new dialog, where the ideal default settings can be configured.

Customize IPSec Defaults

Key Exchange (Main Mode) Configuration

Relevant MS Docs: Configure Key Exchange (Main Mode) Settings

Under the Key Exchange (Main Mode) section, select Advanced, and click Customize. The following two screenshots show the default configuration, and the configuration I have decided to utilize in my lab environment. There’s a number of possible combinations than can be configured here; what I have configured is extremely secure for a modern environment. If you have older operating systems in your environment, you may need to include alternative combinations.

Key Exchange – Default Configuration
Key Exchange – New Configuration

After configuring settings as desired, click OK here to save the Key Exchange configuration and return to the Customize IPSec Defaults dialog.

Data Protection (Quick Mode) Configuration

Relevant MS Docs: Configure Data Protection (Quick Mode) Settings

Under the Data Protection (Quick Mode) section, select Advanced, and click Customize. The following two screenshots show the default configuration, and the configuration I have decided to utilize in my lab environment. There’s a number of possible combinations than can be configured here; what I have configured is extremely secure for a modern environment. If you have older operating systems in your environment, you may need to include alternative combinations.


Data Protection – Default Configuration
Data Protection – New Configuration

After configuring settings as desired, click OK here to save the Data Protection configuration and return to the Customize IPSec Defaults dialog.

Authentication Method

Relevant MS Docs: Configure Authentication Methods

Finally, configure the default Authentication Method by selecting Advanced and Customize. Here, the default configuration will appear empty, but for reference, the default configuration is to utilize Computer Kerberos and User Kerberos. In my configuration I’ll configure it to be the same as what the default configuration actually, but explicitly defined.

Authentication Method Configuration

Once configured, click OK to save the Authentication Method settings, return to the Customize IPSec Defaults dialog, and click OK here as well.

IPSec Exemptions

Now back in the Windows Firewall properties IPSec Settings tab, configure ICMP to be exempted from IPSec, as this will make simple troubleshooting such as using ping easier.

IPSec Exemptions

After clicking OK here, the domain-wide IPSec configuration is complete, and it’s now ready to be utilized.

Summary

At this point, a baseline IPSec configuration has been applied to all domain endpoints, but no changes ave been made as to how endpoints communicate with each other. This baseline will be used in conjunction with later created policies to secure communication between endpoints.

Related Posts

4 thoughts on “Windows Firewall Part 3: Domain IPSec Configuration

  1. Hi. I configured ipsec policy for my win 10 clients , but not yet for domain controller. Am unable to rdp under ipsec between 2 clients. Do I have to exempt domain controller ip ?

    1. It’s going to depend on what exactly has been configured so far. You may want to confirm you can still communicate with the Domain Controller from the client and if not, troubleshoot that. You might need to configure IPSec on the DC side, or exempt it via “do not authenticate” rules.

  2. I want to move certificate from sha1 to sha 256, In windows 2012r2, I have the option in IPSec Main Mode for sha256 but in Quick Mode I have the sha1 option only. Could this cause any conflict?

    1. Are you referring to the integrity algorithm options? Those aren’t directly related to the certificate used to authenticate, but you should have SHA256 available for integrity on Server 2012 R2, as it was initially added in Vista.

Leave a Reply