I wrote in November 2017 about Windows Server Essentials certificates getting renewed every two days. That problem suddenly stopped. The only change I had made was replacing a router. Now, I face a new and more serious issue: when trying to use Remote Web Access recently, I discovered that the certificate had been revoked but not replaced, essentially killing RWA on this machine.
Browsing to the RWA site on my local server, Chrome informs me that the certificate is revoked even though it is does not expire for another 22 months:
From the certificate’s Details tab, I found the CRL Distribution Points field and downloaded the Certificate Revocation List (CRL) from Go Daddy:
http://crl.godaddy.com/gdig2s1-763.crl. The list of revoked certificates is huge, so I ran
certutil –urlcache gdig2s1-763.crl > revoked.txt
to get it into a text file. Then I searched for my certificate’s serial number in a text editor. Sure enough, there it is:
Serial Number: 3761334d0dc18a5c
Revocation Date: 3/9/2019 1:37 PM
Extensions: 1
2.5.29.21: Flags = 0, Length = 3
CRL Reason Code
Cessation of Operation (5)
Obviously I haven’t ceased to operate, so why was the certificate revoked?
Essentials Log
Let’s review C:\ProgramData\Microsoft\Windows Server\Logs\SharedServiceHost-ManagementServiceConfig.2.log at the time of revocation. That log uses a local timestamp. I assume the CRL is in UTC. So converting 3/9/2019 1:37pm to UTC, we have 190309.0537.
Nothing. The log covers that time period but there are no relevant events at that time. What I do see in the log are repeated failures to renew the certificate. These errors occur every day going back to the beginning of the log on 2/27, i.e. well before it was revoked.
[9952] 190309.164814.1090: DomainManager:Service: CertificateExpirationHandler: trusted cert near expiration, will submit renew request [9952] 190309.164814.1090: DomainManager:WindowsLiveProvider: SubmitRenewCertificateRequest called [9952] 190309.164820.4845: DomainManager:WindowsLiveProvider: SubmitRenewCertificateRequest failed with exception System.ServiceModel.FaultException: System.Web.Services.Protocols.SoapException: Live Dynamic DNS has encountered an internal error. This error has been logged. ---> Microsoft.WindowsServerSolutions.DDNS.DDNSException: The GoDaddy certificate signing service returned error code "INTERNAL_ERROR" during an operation on the domain name "mydomain.remotewebaccess.com". at Microsoft.WindowsServerSolutions.DDNS.LiveDynDnsService.CheckForGoDaddyError(String result, String domainName, OperationInstrumentation instrumentation) in D:\rdnext\WSE\ServicesAndTools\src\ServicesAndTools\DDNS\DDNS\LiveDynDns.cs:line 823 at Microsoft.WindowsServerSolutions.DDNS.LiveDynDnsService.InvokeGoDaddyOperationAndCheckError(String domainName, String operationName, Func`1 goDaddyFunction, String[] ignoreErrorCodes) in D:\rdnext\WSE\ServicesAndTools\src\ServicesAndTools\DDNS\DDNS\LiveDynDns.cs:line 809 at Microsoft.WindowsServerSolutions.DDNS.LiveDynDnsService.RenewSslCertificate(String domainName, Byte[] secretKey) in D:\rdnext\WSE\ServicesAndTools\src\ServicesAndTools\DDNS\DDNS\LiveDynDns.cs:line 675 --- End of inner exception stack trace --- at Microsoft.WindowsServerSolutions.DDNS.LiveDynDnsService.RenewSslCertificate(String domainName, Byte[] secretKey) in D:\rdnext\WSE\ServicesAndTools\src\ServicesAndTools\DDNS\DDNS\LiveDynDns.cs:line 703 Server stack trace: at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at Microsoft.WindowsServerSolutions.RemoteAccess.Domains.LiveDynDnsServiceSoap.RenewSslCertificate(RenewSslCertificateRequest request) at Microsoft.WindowsServerSolutions.RemoteAccess.Domains.WindowsLiveDomainMaintenanceProvider.<SubmitRenewCertificateRequest>b__12_0() at Microsoft.WindowsServerSolutions.RemoteAccess.Domains.WindowsLiveProviderBase.InvokeLiveDNSServiceOperation[T](Func`1 operation, String operationName) [9952] 190309.164820.4845: DomainManager:Service: CertificateExpirationHandler: failed with exception: Microsoft.WindowsServerSolutions.RemoteAccess.Domains.DomainException: DomainManagerFault:[Reason:CommunicationFailure, Message:SubmitRenewCertificateRequest failed, Detail:System.Web.Services.Protocols.SoapException: Live Dynamic DNS has encountered an internal error. This error has been logged. ---> Microsoft.WindowsServerSolutions.DDNS.DDNSException: The GoDaddy certificate signing service returned error code "INTERNAL_ERROR" during an operation on the domain name "mydomain.remotewebaccess.com". at Microsoft.WindowsServerSolutions.DDNS.LiveDynDnsService.CheckForGoDaddyError(String result, String domainName, OperationInstrumentation instrumentation) in D:\rdnext\WSE\ServicesAndTools\src\ServicesAndTools\DDNS\DDNS\LiveDynDns.cs:line 823 at Microsoft.WindowsServerSolutions.DDNS.LiveDynDnsService.InvokeGoDaddyOperationAndCheckError(String domainName, String operationName, Func`1 goDaddyFunction, String[] ignoreErrorCodes) in D:\rdnext\WSE\ServicesAndTools\src\ServicesAndTools\DDNS\DDNS\LiveDynDns.cs:line 809 at Microsoft.WindowsServerSolutions.DDNS.LiveDynDnsService.RenewSslCertificate(String domainName, Byte[] secretKey) in D:\rdnext\WSE\ServicesAndTools\src\ServicesAndTools\DDNS\DDNS\LiveDynDns.cs:line 675 --- End of inner exception stack trace --- at Microsoft.WindowsServerSolutions.DDNS.LiveDynDnsService.RenewSslCertificate(String domainName, Byte[] secretKey) in D:\rdnext\WSE\ServicesAndTools\src\ServicesAndTools\DDNS\DDNS\LiveDynDns.cs:line 703 ] at Microsoft.WindowsServerSolutions.RemoteAccess.Domains.WindowsLiveProviderBase.InvokeLiveDNSServiceOperation[T](Func`1 operation, String operationName) at Microsoft.WindowsServerSolutions.RemoteAccess.Domains.WindowsLiveDomainMaintenanceProvider.SubmitRenewCertificateRequest() at Microsoft.WindowsServerSolutions.RemoteAccess.Domains.DomainProviderManager.CertificateExpirationHandler()
So, that’s likely the reason that my server stopped renewing the certificate every two days: it was trying and failing. Now, maybe there is some “feature” that decides if your server hasn’t checked in lately, you have ceased operations and it should just revoke the certificate.
The question of course, is why are we getting this error:
The GoDaddy certificate signing service returned error code “INTERNAL_ERROR” during an operation on the domain name “mydomain.remotewebaccess.com”.
Since remotewebaccess.com isn’t my domain, I doubt that Go Daddy would talk to me about why they are returning an internal error. And extensive interactions with Microsoft support never yielded an answer on the original issue (although they did contact me after I thought it was fixed, and I told them I was no longer experiencing the issue).
I think at this point, my best options are to buy a commercial certificate or to set up a Let’s Encrypt service (with auto-renewal) as described here. I’ve had good experience with Let’s Encrypt on my web site, so I’ll give that a try on my Essentials server.
I’m in the same boat. Thinking of just upgrading to WS 2016 but my 2012 is running fine except the expired cert. Looked over the link you provided and it looks interesting. Wondering how it turned out for you.
Jason, Let’s Encrypt is working but I had to do some tweaking after it auto-renewed to get it to apply to the RDS gateway. Some more notes in this post:
https://www.mcbsys.com/blog/2019/05/certify-the-web-on-server-2016-with-essentials/
and in this thread:
https://www.server-essentials.com/community/discussions/thanks-for-the-lets-encrypt-install-guide
Considering that you can buy single-domain certificates for under $6/year (e.g. from https://cheapsslsecurity.com/), I’m tempted to just start paying and stop fiddling.