SqlServerWriter VSS 8229 error 0x800423f4

I recently installed XProtect Essential+ 2017 R3 on a Windows 7 x64 Pro machine. XProtect installed SQL 2014. The machine is backed up daily by a Server 2012 R2 Essentials process.

The backup seems to finish, but it raises VSS event 8229 as it is shutting down:

Log Name:      Application
Source:        VSS
Date:          12/26/2017 7:25:45 PM
Event ID:      8229
Task Category: None
Level:         Warning
Keywords:      Classic
Description:
A VSS writer has rejected an event with error 0x800423f4, The writer experienced a non-transient error.  If the backup process is retried,
the error is likely to reoccur.
Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer.

Operation:
PostSnapshot Event

Context:
Execution Context: Writer
Writer Class Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Name: SqlServerWriter
Writer Instance Name: Microsoft SQL Server 2014:SQLWriter
Writer Instance ID: {05610d2f-cf55-4c05-9058-3c391a4e307a}
Command Line: “C:\Program Files\Microsoft SQL Server\90\Shared\sqlwriter.exe”
Process ID: 2528

The output of vssadmin list writers includes these lines:

Writer name: ‘SqlServerWriter’
Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance Id: {05610d2f-cf55-4c05-9058-3c391a4e307a}
State: [11] Failed
Last error: Non-retryable error

In this Microsoft forum thread, the September 4, 2014 post by Qiuyun Yu led me to the official description of event ID 8229. That article suggests restarting the writer, so I restarted the SQL Server VSS Writer service. After that, vssadmin list writers returns:

Writer name: ‘SqlServerWriter’
Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance Id: {3364db22-a1b1-4f44-acd7-fb8928553e59}
State: [1] Stable
Last error: No error

Hopefully that resolves the error!

Update December 28, 2017

Got the same error last night, so just restarting the SQL Server VSS Writer service was not enough.

One of the articles I came across recommended running that service as a domain user to allow writing across the network. Since the Essentials backup is a kind of network backup, I’ll try that.

After changing SQL Server VSS Writer to log on as a domain admin and restarting the service, I noticed that vssadmin list writers shows four writers are “Waiting for completion” (System Writer, BITS Writer, MSSearch Service Writer, and IIS Metabase Writer). This article says that if any of the VSS writers show “Waiting for completion”, you should restart the Cryptographic Services service. I did that, but the writers still show “Waiting for completion”.

Restarting the machine. We’ll see if the backup runs without errors tonight.

Update December 29, 2017

That didn’t fix it either—got the same 8229 event during last night’s backup. Trying a repair install of SQL 2014 Express. I did two repairs, one for VIDEOOSDB instance and one for the MSSQLSERVER instance.

Update January 25, 2018

I set this aside for a while but am back to it.

The repair install of SQL 2014 Express did not help.

Uninstalling the previous version of XProtect did not help.

I found a forum thread where people were trying to solve a similar conflict between Veeam backup and SQL 2014. One post cites a fix from the ACT forum:  change the SQL instance to run as LocalService rather than LocalSystem. Since both ACT and XProtect install SQL 2014 Express, maybe that suggestion could help here too.

I found that the VIDEOSDB SQL instance was already running as NetworkService (which I would think is even more inclusive than LocalService), so I left that as is. However, the default SQLSERVER instance was running as LocalSystem:

I changed that to LocalService, which restarted the SQL instance:

By the way, the file C:\Program Files\Microsoft SQL Server\MSSQL12.VIDEOOSDB\MSSQL\DATA\VideoOSDB.mdf hasn’t changed since I uninstalled the old XProtect. My hunch is that VIDEOSDB was used by XProtect 2016 and is no longer used. Don’t know why It wasn’t uninstalled when I uninstalled the program.

The folder containing the SQLSERVER databases (C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\DATA) contains several files that were changed just a few minutes ago, so this instance is in active use.

Update January 27, 2018

This seems to finally be resolved after setting MSSQLSERVER to run as LocalService. There was no VSS warning during the backup last night and the output of vssadmin list writers says there is No error on SqlServerWriter:

Writer name: ‘SqlServerWriter’
Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance Id: {2d39723a-4071-4b91-a5d7-36ad478fdc09}
State: [5] Waiting for completion
Last error: No error

Note The error did recur once after changing the SQLSERVER startup account. I realized that I hadn’t restarted the SQL Server VSS Writer service. To be on the safe side, I restarted the entire computer yesterday morning.

Update February 18, 2018

After several weeks without issues, I got the same error 8229 the same day that I installed the February Security and Quality patches (including a .NET rolllup). MSSQLSERVER is still running as LocalService. I am restarting the SQL VSS Writer service (to clear the “non-retryable error” status). We’ll see if this was a one-time issue.

Update February 21, 2018

The 8229 errors continue. Today I temporarily set the MSSQLSERVER back to run as LocalSystem, then set it back to run as LocalService. Finally I restarted the SQL Server VSS Writer service

Update February 23, 2018

Yesterday I set the MSSQLSERVER service to run as NetworkService. Didn’t help.

Today I set MSSQLSERVER back to LocalService. Somewhere in all these reconfigurations, SQL Server VSS Writer was set to run as a domain admin; today, I’m setting that back to its default LocalSystem.

I noticed that the file date on the sqlwriter.exe file is January 11, 2018, so the problems could well have returned due to an update to the SQL Server VSS Writer. The file version is now 2014.120.5571.0, which corresponds to Cumulative Update 10 (CU10) for Microsoft SQL Server 2014 Service Pack 2, which was released as a security bulletin on January 16, 2018 (see KB4057117).

Update March 26, 2018

Still getting this error, even after another update to SQLwriter.exe on February 21, 2018. Today I’m giving four services domain admin rights. If it’s a simple permissions thing, this should solve it:

SQL Server (MSSQLSERVER)
SQL Server (VIDEOSDB)
SQL Server Browser
SQL Server VSS Writer

Restarted the machine.

Update May 8, 2018

Still getting the VSS error every day. I’m going to open a case on the Microsoft partner forum.

Update May 21, 2018

Microsoft Support offered some suggestions but they did not resolve the issue.

I checked with XProtect vendor Milestone Systems, who confirmed that the SQL instance named VIDEOOSDB is not needed by my current Essentials+ software. So I disabled the SQL service for that instance. I also changed the MSSQLSERVER root instance, which is still in use, to run as LocalSystem. After that, the VSS 8229 warnings stopped.

So I still don’t know what was causing the VSS warning, but disabling the SQL Server in question works around the issue.