Home | Site Map | Cisco How ToNet How To | Wireless |Search | Forums | Services | Donations | Careers | About Us | Contact Us|

Event ID: 276 ADFS proxy was not able to authenticate

Active Directory, Domain, DNS, WINS, DHCP, SBS, New Releases.

Event ID: 276 ADFS proxy was not able to authenticate

Postby blin » Thu Jul 21, 2016 2:05 pm

Situation: when configuring ADFS Proxy server, you may receive this message:
Source: AD FS
Event ID: 276
Task Category: None
Level: Error
Keywords: AD FS
User: S-1-5-21-2461873363-3247490994-1217893606-1157
Computer: adfs.chicagotech.net
Description:
The federation server proxy was not able to authenticate to the Federation Service.

User Action
Ensure that the proxy is trusted by the Federation Service. To do this, log on to the proxy computer

with the host name establish trust between the proxy and the Federation Service using the

Install-WebApplicationProxy cmdlet.

Troubleshooting: run this powershell command:netsh http show sslcert
Resolution steps will depend on what bindings are found and on why these are there. The simplest solution is to ensure that there is not a specific IP:port binding in place for the IP address that the AD FS Service FQDN resolves to.
How to Configure and Troubleshoot Cisco
http://www.howtocisco.com

Tablet and Smartphone Setup Guide
http://www.quicksetupguide.com
blin
Site Admin
 
Posts: 3609
Joined: Wed Dec 31, 1969 7:00 pm
Location: Chicago, USA

Re: Event ID: 276 ADFS proxy was not able to authenticate

Postby blin » Thu Jul 21, 2016 2:07 pm

Quoted from online:
Root Cause 1 – a specific IP:port certificate binding on the AD FS IP address

As we know, the Certificate Trust List is generated based on the CTL Store Name defined on the http.sys certificate binding. We can use “netsh http show sslcert” output to examine the certificate bindings from a problem AD FS server. In a number of customer cases we’ve found a binding similar to the following:-


C:\Users\administrator.CONTOSO>netsh http show sslcert

SSL Certificate bindings:
IP:port : 1.2.3.4:443
Certificate Hash : 3638de9b03a488341dfe32fc3ae5c480ee687793
Application ID : { 4dc3e181-e14b-4a21-b022-59fc669b0914}
Certificate Store Name : (null)
Verify Client Certificate Revocation : Enabled
Verify Revocation Using Cached Client Certificate Only : Disabled
Usage Check : Enabled
Revocation Freshness Time : 0
URL Retrieval Timeout : 0
Ctl Identifier : (null)
Ctl Store Name : (null)
DS Mapper Usage : Disabled
Negotiate Client Certificate : Disabled


There are 2 important things to note here:-
1.This is an IP:port binding where 1.2.3.4 is the IP address of the ADFS server. This is also the IP address that the ADFS FQDN resolves to.
2.The Ctl Store Name is null so the OS will fallback to use the Trusted Root CA store to build the Certificate Trust List



In this case as there is a specific IP:port binding in place this binding was being used ahead of the ADFS Hostname:port binding. Despite the certificate being used was exactly the same as the IP:port binding does not have a Ctl Store specified the Trusted Root CA store was being used to build the CTL list causing the Proxy Trust certificate on Web Application Proxy to be filtered out.

The only missing piece of the puzzle is why this binding was in place and, in the above example, the Application ID helped us understand this.

Application ID : { 4dc3e181-e14b-4a21-b022-59fc669b0914}

This happens to be the SSL Certificate Owner Application ID for w3svc (IIS) so it looked as if the customer had configured this binding in IIS and this proved to be the case. We have seen a number of customers install IIS on the AD FS server either due to believing it is required for AD FS 2012 R2 (it isn't) or as a way to generate a Certificate request.

Note – when you configure http.sys certificate bindings using netsh you can configure whatever Application ID you want so a binding may have been created manually and appear to have been configured by an application.

The PowerShell script provided in the ‘Issue Detection PowerShell Script’ section below can be run on an AD FS 2012 R2 server to determine if this issue is present.



Root Cause 1 – Resolution Options

When looking at a resolution the first thing to determine is why the binding is in place.

This might have been created by mistake or as a leftover from an install of a different product or Windows component but it may equally be there for a valid reason. As mentioned above it’s also possible that a binding may have been manually created possibly to support non-SNI clients.

Once you have determined the reason for the binding being in place, you can decide which of the following resolutions is more appropriate:-

1. Remove the specific IP:port binding

Be sure to check that the binding does not come back as if there is an application configured with such a binding it may recreate this automatically or on next service start-up.

2. Use an additional IP address for AD FS traffic

In our example above if the 1.2.3.4 SSL Binding is expected and required, then using a 2nd IP such as 1.2.3.5 for ADFS and resolving the ADFS service FQDN to this IP address would mean that the Hostname:port bindings would then be used (assuming that there isn’t an IP:port binding for 1.2.3.5:443 of course).

3. Configure the AdfsTrustedDevices store as the Ctl Store for the specific IP:port binding

This will again have some dependence on why the specific IP:port binding is there and if this relies on the default CTL Store for client certificate authentication but an option would be to set the Ctl Store on the IP:port binding to be the AdfsTrustedDevices store

At this point in time, our recommendation is to avoid having IP specific bindings and use option 1 as the preferred resolution.






Root Cause 2 – the AD FS Hostname:port binding does not have a Ctl Store defined

This was a bit more of a strange one as when we checked the http.sys cert bindings everything looked correct except that the AD FS Hostname:port binding did not have a Ctl Store defined:-


C:\Users\administrator.CONTOSO>netsh http show sslcert

SSL Certificate bindings:
————————-
Hostname:port : adfs.contoso.com:443
Certificate Hash : 3638de9b03a488341dfe32fc3ae5c480ee687793
Application ID : {5d89a20c-beab-4389-9447-324788eb944a}
Certificate Store Name : MY
Verify Client Certificate Revocation : Enabled
Verify Revocation Using Cached Client Certificate Only : Disabled
Usage Check : Enabled
Revocation Freshness Time : 0
URL Retrieval Timeout : 0
Ctl Identifier : (null)
Ctl Store Name : (null)
DS Mapper Usage : Disabled
Negotiate Client Certificate : Disabled


Of interest here is that the Application ID is the AD FS 2012 R2 SSL Certificate Owner Application ID so on the surface does look to have been created by AD FS. We did not manage to identify why the CTL Store was not correctly set in this case. The PowerShell script provided in the ‘Issue Detection PowerShell Script’ section below can be run on an AD FS 2012 R2 server to determine if this issue is present.

Root Cause 2 – Resolution

In this case we can use the following PowerShell cmdlet to regenerate the ADFS Certificate bindings on the AD FS server:-

Set-AdfsSslCertificate -Thumbprint 3638de9b03a488341dfe32fc3ae5c480ee687793

The Thumbprint here is the Thumbprint for the ADFS Server SSL certificate and needs to be set accordingly. This cmdlet should be run on each AD FS server in a farm. For further details on this cmdlet please see the following article – http://technet.microsoft.com/en-us/libr ... 79374.aspx






Root Cause 3 – CA issued server certificate placed in the AdfsTrustedDevices store

The AdfsTrustedDevices store is an AD FS specific store and should not need to have any certificates manually placed in here.

We found in one case that the ADFS service SSL certificate was incorrectly placed in the AdfsTrustedDevices store as well as the correct Local Machine, Personal store. The AdfsTrustedDevices store should only contain the MS-Organization-Access certificate which is the self-signed cert used for issuing Workplace Join certificates, and the Proxy Trust certificates for each of the Web Application Proxy servers.

As detailed earlier, having a CA Issued certificate in a store where only Self-Signed certs would normally exist affects the CTL generated from this store and the CTL will then only contain the CA Issued certificate. The presence of a CA issued certificate in the AdfsTrustedDevices store changed the CTL being returned and led to the proxy trust failure.

The PowerShell script provided in the ‘Issue Detection PowerShell Script’ section below can be run on an AD FS 2012 R2 server to determine if this issue is present.

Root Cause 3 – Resolution

Deleting the non-self signed SSL Server Certificate from the AdfsTrustedDevices store resolved the issue and meant that the CTL being generated from the AdfsTrustedDevices store was now correct.






Root Cause 4 – Proxy Trust certificate propagation issues across an AD FS 2012 R2 farm

When a Proxy Trust is established with an AD FS server, the Proxy Trust certificate is written to the AD FS configuration database and added to the AdfsTrustedDevices store on the AD FS server that handled the Proxy Trust set-up. In an AD FS farm a regular task runs on each AD FS server in the farm which polls the AD FS configuration store for the current list of Proxy Trust certificates and updates the local AdfsTrustedDevices store.

We have recently discovered an issue where this task only successfully runs if Device Registration has been initialized.

If Device Registration has not been initialized then Proxy Trust certificate propagation across the AD FS farm may fail and a Proxy Trust will only work against the AD FS server the trust was established with. As the CTL is based on the AdfsTrustedDevices store, if this has not been populated with the correct Proxy Trust certificates and a Web Application Proxy server talks to an AD FS server other than the one it established the proxy trust with, then the proxy trust may fail due to the CTL being incorrect.

You can check if Device Registration has been initialized by running the following PowerShell cmdlet on one of the AD FS servers:-

Get-AdfsDeviceRegistration

This requires Enterprise Administrator credentials to run. If Device Registration has been initialized you would see output similar to the following:-


:-PS C:\users\administrator.CONTOSO\Desktop> Get-AdfsDeviceRegistration

DrsObjectDN : CN=DeviceRegistrationService,CN=Device Registration Services,CN=Device Registration
Configuration,CN=Services,CN=Configuration,DC=contoso,DC=com
DevicesPerUser : 10
MaximumInactiveDays : 90
IsEnabledOnPremises : True
IsEnabledInCloud : False
DeviceObjectLocation : CN=RegisteredDevices,DC=contoso,DC=com


The PowerShell script provided in the ‘Issue Detection PowerShell Script’ section below can also be run on an AD FS 2012 R2 server to determine if this issue is present.

Note – if your load balancing for your AD FS server farm uses client IP based affinity we would generally expect a Web Application Proxy server to use the same AD FS server other than in a failover scenario.



Root Cause 4 – Resolution Options



Update 1st July 2014 – A fix is now available for this specific issue in the June 2014 Windows update rollup – http://support.microsoft.com/kb/2964735



1. Initialize Device Registration in Active Directory

The current recommended resolution for this is to initialize Device Registration using the following PowerShell script on one of the AD FS 2012 R2 servers which creates the DRS containers in Active Directory.

Initialize-ADDeviceRegistration

Note – this needs to be run with Enterprise Administrator credentials and requires that the Windows 2012 R2 AD schema update has been deployed. For further details on this cmdlet please see the following article – http://technet.microsoft.com/en-us/libr ... 79332.aspx

This cmdlet does *not* enable the device authentication nor the device registration service in the ADFS servers. Enabling Device Registration is a second step in the DRS set-up process.

The AD FS Product Group are currently investigating a fix for this issue but at the time of writing we do not have further details on this or any expected timelines. We will provide an update when we have further information on this.

2. Manually Sync Proxy Trust Certificates using PowerShell

The PowerShell script provided in the ‘Issue Detection PowerShell Script’ section below can be run with the – syncproxytrustcerts switch to sync down missing Proxy Trust certificates from the AD FS configuration to the AdfsTrustedDevices certificate store. This script should be run on each of the AD FS servers in the farm.

Note – this is not a permanent solution as the Proxy Trust certificates will be renewed on a regular basis so until a fix is available Option 1 is the recommended resolution.

3. Use source IP based affinity on your AD FS 2012 R2 load balancing solution

Using source IP based affinity on the AD FS Load Balancer will mean a Web Application Proxy server is generally going to use the same AD FS server it established the initial trust with and therefore not be reliant on the proxy trust certificate being propagated to other members of the AD FS farm. If the preferred AD FS node a Web Application Proxy server changes or in a failover situation then this issue may still be exposed.










Root Cause 5 – System time mis–match between WAP and ADFS

It is expected that the Web Application Proxy server and AD FS server are time synchronized. Where Web Application Proxy is not domain joined this may not be the case and we have seen Proxy Trust issues as a result of this e.g.

AD FS Server time = 12pm GMT timeone

Web Application Proxy Server time = 12pm PST timezone (8pm GMT)

In this scenario while the system time may seem synchronized the timezone difference means that the Web Application Proxy server is 8 hours ahead of the AD FS server time. When the Proxy Trust relationship is established, the self-signed Proxy Trust certificate generated by the Web Application Proxy is not yet valid on the AD FS server and so the Proxy Trust fails.

While this is not directly related to CTL issues it’s one to be aware of with similar symptoms.

Root Cause 5 – Resolution

Ensure that the system time of the Web Application Proxy server and AD FS 2012 R2 server are in sync including that the timezones are set appropriately.






Root Cause 6 – SSL termination taking place on a network device between Web Application Proxy and AD FS server

We saw a recent issue where we were seeing the following in the AD FS event logs on the Web Application Proxy server and AD FS server.

In the AD FS event log on the Web Application Proxy server:-


Log Name: AD FS/Admin
Source: AD FS
Date: 3/24/2014 2:09:22 PM
Event ID: 422
Task Category: None
Level: Error
Keywords: AD FS
User: S-1-5-21-2461873363-3247490994-1217893606-500
Computer: wap.contoso.com
Description:
Unable to retrieve proxy configuration data from the Federation Service.

Additional Data

Trust Certificate Thumbprint:
48C0E6811E4631C1D115CD28B02BFA4214D4A179

Status Code:
Unauthorized

Exception details:
System.Net.WebException: The remote server returned an error: (401) Unauthorized.
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.IdentityServer.Management.Proxy.StsConfigurationProvider.GetStsProxyConfiguration()


In the AD FS event logs on the AD FS server:-


Error 3/24/2014 2:49:52 PM AD FS 276 None
Log Name: AD FS/Admin
Source: AD FS
Date: 3/24/2014 2:49:52 PM
Event ID: 276
Task Category: None
Level: Error
Keywords: AD FS
User: S-1-5-21-2461873363-3247490994-1217893606-1157
Computer: adfs.contoso.com
Description:
The federation server proxy was not able to authenticate to the Federation Service.

User Action
Ensure that the proxy is trusted by the Federation Service. To do this, log on to the proxy computer

with the host name establish trust between the proxy and the Federation Service using the

Install-WebApplicationProxy cmdlet.

Additional Data

Certificate details:

Subject Name:
<CN=adfsapps.contoso.com, OU=Contoso, O=Contoso, L=Contoso, S=Contoso, C=US>

Thumbprint:
<AD467089E3407BD72CD5F3D46A26B3873C5F0F4C>

NotBefore Time:
<2014-05-09 00:00:00>

NotAfter Time:
<2015-05-09 23:59:59>





There are a number of things to note here:-
1.The AD FS server was presented a client certificate. This is different to most of the above issues where a client certificate is not seen at the AD FS server.
2.The thumbprint seen at the AD FS server is different to the one seen in the Web Application Proxy AD FS event logs.
3.The Subject Name seen by the AD FS proxy is not what we would expect from a proxy trust certificate which should have the form 'ADFS Proxy Trust – webappproxymachinename'

Given the difference in Thumbprint and unexpected Subject Name we investigated the network devices between the Web Application Proxy server and the AD FS server and determined that the hardware load balancer sitting in-front of the AD FS farm was carrying SSL bridging i.e. terminating the SSL connection from Web Application Proxy and establishing a new SSL connection to the AD FS server. The certificate seen by the AD FS server was being presented by the hardware load balancer.

As the Proxy Trust relationship between Web Application Proxy and AD FS server is client certificate based this was breaking the proxy trust relationship.

The SSL communications between the Web Application Server and AD FS server cannot be pre-terminated on a load balancer, Firewall, other reverse proxy device and doing so will break the proxy trust relationship.

Root Cause 6 – Resolution

Disabling SSL Termination on the hardware load balancer to allow the SSL negotiation to happen between Web Application Proxy and AD FS server resolved this issue.







Issue Detection PowerShell Script

The following PowerShell script can be run on an AD FS server to determine if any of the first 4 issues detailed above are present. In an AD FS farm the script should be run on each farm member.

The script can also be run with the –syncproxytrustcerts switch to sync down missing Proxy Trust certificates from the AD FS configuration to the AdfsTrustedDevices certificate store as a temporary resolution for Issue 4. This sync would need to be done on each AD FS server.





param
(
[switch]$syncproxytrustcerts
)

function checkhttpsyscertbindings()
{

Write-Host; Write-Host("1 – Checking http.sys certificate bindings for potential issues")

$httpsslcertoutput = netsh http show sslcert
$adfsservicefqdn = (Get-AdfsProperties).HostName
$i = 1
$certbindingissuedetected = $false

While($i -lt $httpsslcertoutput.count)
{
$ipport = $false
$hostnameport = $false

if ( ( $httpsslcertoutput[$i] -match "IP:port" ) ) { $ipport = $true }
elseif ( ( $httpsslcertoutput[$i] -match "Hostname:port" ) ) { $hostnameport = $true }

# Check for IP specific certificate bindings
if ( ( $ipport -eq $true ) )
{
$httpsslcertoutput[$i]
$ipbindingparsed = $httpsslcertoutput[$i].split(":")

if ( ( $ipbindingparsed[2].trim() -ne "0.0.0.0" ) -and ( $ipbindingparsed[3].trim() -eq "443") )
{
$warning = "There is an IP specific binding on IP " + $ipbindingparsed[2].trim() + " which may conflict with the AD FS port 443 cert binding." | Write-Warning
$certbindingissuedetected = $true
}

$i = $i + 14
continue
}

# check that CTL Store is set for ADFS service binding
elseif ( $hostnameport -eq $true )
{
$httpsslcertoutput[$i]
$ipbindingparsed = $httpsslcertoutput[$i].split(":")
If ( ( $ipbindingparsed[2].trim() -eq $adfsservicefqdn ) -and ( $ipbindingparsed[3].trim() -eq "443") -and ( $httpsslcertoutput[$i+10].split(":")[1].trim() -ne "AdfsTrustedDevices" ) )

{
Write-Warning "ADFS Service binding does not have CTL Store Name set to AdfsTrustedDevices"
$certbindingissuedetected = $true
}
$i = $i + 14
continue
}
$i++
}

If ( $certbindingissuedetected -eq $false ) { Write-Host "Check Passed: No certificate binding issues detected" }

}

function checkadfstrusteddevicesstore()
{
# check for CA issued (non-self signed) certs in the AdfsTrustedDevices cert store
Write-Host; Write-Host "2 – Checking AdfsTrustedDevices cert store for non-self signed certificates"

$certlist = Get-Childitem cert:\LocalMachine\AdfsTrustedDevices -recurse | Where-Object {$_.Issuer -ne $_.Subject}

If ( $certlist.count -gt 0 )
{
Write-Warning "The following non-self signed certificates are present in the AdfsTrustedDevices store and should be removed"
$certlist | Format-List Subject
}
Else { Write-Host "Check Passed: No non-self signed certs present in AdfsTrustedDevices cert store" }

}

function checkproxytrustcerts
{
Param ([bool]$repair=$false)

Write-Host; Write-Host("3 – Checking AdfsTrustedDevices cert store is in sync with ADFS Proxy Trust config")

$doc = new-object Xml
$doc.Load("$env:windir\ADFS\Microsoft.IdentityServer.Servicehost.exe.config")

$connString = $doc.configuration.'microsoft.identityServer.service'.policystore.connectionString

$command = "Select ServiceSettingsData from [IdentityServerPolicy].[ServiceSettings]"

$cli = new-object System.Data.SqlClient.SqlConnection
$cli.ConnectionString = $connString

$cmd = new-object System.Data.SqlClient.SqlCommand
$cmd.CommandText = $command
$cmd.Connection = $cli

$cli.Open()

$configString = $cmd.ExecuteScalar()

$configXml = new-object XML
$configXml.LoadXml($configString)

$rawCerts = $configXml.ServiceSettingsData.SecurityTokenService.ProxyTrustConfiguration._subjectNameIndex.KeyValueOfstringArrayOfX509Certificate29zVOn6VQ.Value.X509Certificate2

#$ctl = dir cert:\LocalMachine\ADFSTrustedDevices

$store = new-object System.Security.Cryptography.X509Certificates.X509Store("ADFSTrustedDevices","LocalMachine")
$store.open("MaxAllowed")

$atLeastOneMismatch = $false
$badCerts = @()

foreach($rawCert in $rawCerts)
{

$rawCertBytes = [System.Convert]::FromBase64String($rawCert.RawData.'#text')
$cert=New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(,$rawCertBytes)

$now = Get-Date

if ( ($cert.NotBefore -lt $now) -and ($cert.NotAfter -gt $now))
{
$certThumbprint = $cert.Thumbprint
$certSubject = $cert.Subject
$ctlMatch = dir cert:\localmachine\ADFSTrustedDevices\$certThumbprint -ErrorAction SilentlyContinue
if ($ctlMatch -eq $null)
{
$atLeastOneMismatch = $true
Write-Warning "This cert is NOT in the CTL: $certThumbprint – $certSubject"

if ($repair -eq $true)
{
write-Warning "Attempting to repair"
$store.Add($cert)
Write-Warning "Repair successful"
}
else
{
Write-Warning ("Please install KB.2964735 or re-run script with -syncproxytrustcerts switch to add missing Proxy Trust certs to AdfsTrustedDevices cert store")
}

}
}

}
$store.Close()

if ($atLeastOneMismatch -eq $false)
{
Write-Host("Check Passed: No mismatched certs found. CTL is in sync with DB content")
}

}

checkhttpsyscertbindings
checkadfstrusteddevicesstore
checkproxytrustcerts($syncproxytrustcerts)
Write-Host; Write-Host("All checks completed.")
How to Configure and Troubleshoot Cisco
http://www.howtocisco.com

Tablet and Smartphone Setup Guide
http://www.quicksetupguide.com
blin
Site Admin
 
Posts: 3609
Joined: Wed Dec 31, 1969 7:00 pm
Location: Chicago, USA

Re: Event ID: 276 ADFS proxy was not able to authenticate

Postby blin » Thu Jul 21, 2016 2:13 pm

In our case, the ADFS server changes the hostname so that the proxy server can't retrieve the data from the adfs server.
First of all, use netsh http show sslcert to check the ssl setting. Then reset ssl:


netsh http delete sslcert hostnameport=tfsvr01.payroll.com:443
netsh http delete sslcert hostnameport=tfsvr01.payroll.com:49443
netsh http delete sslcert ipport=10.0.0.176:443

netsh http add sslcert ipport=0.0.0.0:443 Certhash=31a91103f8df3d0bd5c633ca526c6cf30901c3af appid={5d89a20c-beab-4389-9447-324788eb944a}

netsh http add sslcert hostnameport=hsatfsvr01.hsapayroll.com:443 Certhash=31a91103f8df3d0bd5c633ca526c6cf30901c3af appid={5d89a20c-beab-4389-9447-324788eb944a} certstore=my

netsh http add sslcert hostnameport=hsatfsvr01.hsapayroll.com:49443 Certhash=31a91103f8df3d0bd5c633ca526c6cf30901c3af appid={5d89a20c-beab-4389-9447-324788eb944a} certstore=my

netsh http add sslcert hostnameport=localhost:443 Certhash=31a91103f8df3d0bd5c633ca526c6cf30901c3af appid={5d89a20c-beab-4389-9447-324788eb944a} certstore=my
How to Configure and Troubleshoot Cisco
http://www.howtocisco.com

Tablet and Smartphone Setup Guide
http://www.quicksetupguide.com
blin
Site Admin
 
Posts: 3609
Joined: Wed Dec 31, 1969 7:00 pm
Location: Chicago, USA


Return to Windows

Your Ad Here

Who is online

Users browsing this forum: Yahoo [Bot] and 8 guests