Second WAP not establishing trust with secondary ADFS 3.0 Farm node

Hi,

I've been working on setting up our corporate ADFS environment with a mostly successful outcome however I am having an issue with one of our ADFS WAP servers not establishing a trust with a secondary ADFS server in our internal farm.

I believe this relates to the ADFSTrustedDevices certificate store not replicating between the two internal farm nodes.

THE SETUP

Diagram

Internal

• 2 Windows 2012 R2 ADFS 3.0 Servers in one farm
• Each server is in a different site with resilient WAN connections, sites are GB1 and GB2

External

• 2 Windows 2012 R2 ADFS 3.0 WAP Servers
• Each server is in a different site with resilient WAN connections, sites are GB1 and GB2 (same as above)
• Each server is independent of the other (IE, no NLB or load balancer)
• Both servers are in the same DMZ network (multi-site is achieved via a stretch VLAN between GB1 and GB2)
• Internet/DMZ is resilient across both sites via BGP routing

So, GB1 contains the primary ADFS server and a proxy, GB2 contains secondary ADFS server and a proxy.
Both ADFS servers are in a farm.

DNS
Externally we use DNS round robin to the two proxies. Not best practice but the infrastructure is highly resilient so it's cost effective.

From each proxy, HOSTS files are used to lock the traffic to the internal ADFS server in the same site. IE, Proxy in GB1 will only communicate with internal ADFS server in GB1. Proxy in GB2 to internal ADFS server in GB2.

The reasoning behind this is for a site failure. Half the external traffic may hit the down proxy server and timeout but the other half will hit the working proxy. We didn't want half of the working proxy requests trying to contact the downed server in the failed site, giving us only one quarter of successful requests, if that makes sense.

THE ISSUE

During the setup of the second proxy in GB2 I could not establish a trust to the internal ADFS server in GB2, the secondary server.

Spent some time investigating with no success so I changed the HOSTS file to contact the primary internal ADFS server in GB1 and the trust was established and WAP configured

At this stage I could see that the ADFSTrustedDevice certificate store on the secondary ADFS server in GB2 was empty whilst the certificate store on the primary ADFS server in GB1 was populated with both proxy servers.

I changed the HOSTS file on GB2 proxy back to GB2 ADFS server and this continued to work for a while.

I had hoped that the automatic process would populate GB2 ADFS server with the certificates but it did not.

Eventually the trust broke down and I cannot re-establish the trust without pointing the GB2 proxy back to GB1.

I also cannot sync the certificate stored from GB1 ADFS server to GB2 ADFS server using the script found in this extremely useful article from Ian Parramore:

http://blogs.technet.com/b/applicationproxyblog/archive/2014/05/28/understanding-and-fixing-proxy-trust-ctl-issues-with-ad-fs-2012-r2-and-web-application-proxy.aspx#pi148362=2

WHAT HAVE I TRIED?
Ran the script in the above blog and no issues found, including using the switch -syncproxytrustcerts

KB2964735 / KB2962409 is installed an both ADFS servers

I have not initialised Device Registration as this will require updating the AD schema to 2012 which we are not ready to perform however this may well be the root of the problem, forcing us to move the AD schema forward.

SUGGESTIONS?
If you have any suggestions or advise on how to overcome this issue I've really appreciate some assistance.

Thanks in advance

Adam Callaghan