A client uses Windows Server Backup to back up a small Hyper-V server running Windows Server 2008 R2. Today free space on the backup drive F: dropped below 100GB of 698GB, triggering a 15% monitoring threshold. Some quick notes on what I did.
Research
Review this Microsoft blog post:
Backup Version and Space Management in Windows Server Backup
List shadow copies:
wbadmin get versions -backupTarget:f:
vssadmin list shadows /for=f:
Delete Old Shadow Copies
Delete 5 oldest shadow copies on the target volume:
diskshadow
delete shadows oldest f:
delete shadows oldest f:
delete shadows oldest f:
delete shadows oldest f:
delete shadows oldest f:
That freed up about 4.5GB per shadow copy, so now 122GB free (17%), 175GB in shadow copies. I could delete more—there must be 20 left.
No System State Backups to Delete
Tried this command to delete system state backups:
wbadmin delete systemstatebackup -deleteOldest
Got this message:
Enumerating system state backups...
ERROR - No system state backups were found.
This command can be used to delete backups that only contain the system state.
So apparently that’s only needed if you’re doing separate system state backups. I do system state backups together with other backups.
Update November 17, 2012
About three hours after I did the above, free disk space fell under 15% again. By the next morning, it was 97.7GB free and 199GB in shadow copies. What? How did I drop from 122GB free to 97.7GB free in a matter of hours? I did delete the shadows while a backup was running, but should that matter? Does Windows Server Backup just use as much space as possible? I just deleted another five shadow copies, this time with no backup running, and I’m back to 126GB free and 171GB in shadow copies. We’ll see what happens.
Update November 19, 2012
Two days and two backups later, the F: drive is showing 115GB (16.5%) free and 187GB in shadow copies. Better.
Update January 17, 2013
2012 Essentials Issues and References
I’m having similar issues with Windows Server Backup (WSB) on Server Essentials 2012. I had a backup fail for lack of disk space, even though 61 backups were available. I had to manually delete old shadow copies to get the backup running again. More research and forum posts have turned up some additional information.
The Windows Server Backup Overview on TechNet says, “Windows Server Backup does not require user intervention to periodically delete older backups to free up disk space for newer backups—older backups are deleted automatically.” Unfortunately, that is not always the case.
Windows Server Backup automatic disk usage management, also on TechNet, explains how the auto-delete works, why it sometimes doesn’t delete enough, and why backups fail. Basically WSB always leaves 1/8 of the backup volume allocated to shadow copies, which is how it stores previous backups. If that doesn’t leave enough space for the backup (which, I’ve discovered, is not compressed), the backup fails.
Update February 26, 2013
Configure Shadow Copies
I’m trying a different approach for maintaining free space on a dedicated backup volume: Configure Shadow Copies.
- In Disk Management, assign a drive letter to the backup drive. You probably did this already if you’ve been using wbadmin or vssadmin as described above.
- In Windows Explorer, right-click on the drive and select Configure Shadow Copies.
- In the Shadow Copies dialog, make sure the drive is highlighted. The Next Run Time should show Disabled. Click on Settings.
- Change Maximum Size from No limit to Use limit and set the value to leave about 15% of the drive free. For example, if a 100GB drive has become full using 40GB of shadow copy space, reduce the shadow copy space to a fixed 25GB to leave 15GB free.
I’ve had that set for a few days and it does seem to be maintaining 15% free space without my having to manually delete backups.
Caution I have not seen this method recommended anywhere so I can’t promise that it’s a safe approach to managing backup space. Use at your own risk!
Update April 14, 2016
To configure shadow copies on a desktop computer, you can select Control Panel > System, open System Protection, select the target drive, then click the Configure button. You’ll want to leave Restore Settings off for the backup target, but you can still adjust maximum Disk Space Usage for shadow copies on that drive.
This article mentions a couple more helpful vssadmin commands for doing the same thing. This should work on a desktop (if it’s a Pro version of Windows) or a server.
List shadow storage in use on all drives:
vssadmin list shadowstorage
Change shadow storage on one drive:
vssadmin resize shadowstorage /on=F: /for=F: /maxsize=25%
Thanks for posting this. It just serves to underscore how badly Microsoft is documenting their products.
Thanks for the post Mark. I came to the same conclusion regarding shadow copy limits.
Seeing as it has been a year since you did this, has it worked well??
Brit – yes, I’ve pretty much forgotten about shadow copies and my disks are not filling up, so I’d say it’s working!
Cheers Mark, that looks like it might be the answer I’ve been looking for too, just testing it on a few servers. What I don’t get is how this was meant to work, and why everyone else is just saying “manually delete your old snapshots whenever a backup fails” as though it’s an answer…
Pingback: Windows Server Backup – Manually Delete Snapshots - BlueCompute
In Server 2012 R2, you don’t seem to need to assign a drive letter to the backup drive before you can adjust the Shadow Copies. You can open File Explorer, highlight/select any available drive letter, right click on it and select properties, then click on the Shadow Copies tab. From the list of drives, you’ll see some with drive letters and some without. Select your backup target from the list (you’ll know which one it is by the large amount of space taken up by shadow copies and it won’t have a drive letter). Change it to a smaller value (I use 300 GB on a 2 TB backup drive). YMMV. On my 2 TB Backup drives, I seemed to run out of space when the shadow copies exceeded 500 GB and the backups would fail with the out of space message. Reducing the Shadow Copy limit to 300 GB seemed to do the trick for my environment.
If you look at the drive with the Windows Backup GUI (lower right, under destination usage), you’ll see the change almost immediately in the available space on the drive when you adjust the shadow copy limit.
Thanks chucko, good tip!
Thanks Mark. (Chucko’s option are not seen in our Windows 2012r2.) But your suggestions are good.
Chucko’s option didn’t work for me either. I see no Shadow Copies tab in File Explorer in Windows Server 2012R2. HOWEVER, you can right click the drive in Disk Management, click properties, and get to the Shadow Copies tab. We just reset our limit on 3 1TB USB backup drives with no drive letters and immediately regained free space. Hopefully this will solve our problem! Thanks.
Ravi, Chucko’s solution does not start from a TAB. Instead, the suggestion is to right-click e.g. on the C: drive in Windows Explorer and select “Configure Shadow Copies” from the CONTEXT MENU.
Your approach to work through Disk Management is good too. I realized when testing it that one has to right-click on the PARTITION, not on the DISK, to get access to the Shadow Copies tab. Oddly it only shows me that tab when I click on volumes on disks 0 and 1. Volumes in disks 2 and 3 (also internal, physical disks) do not show a Shadow Copies tab. But Chucko’s trick works here too: once you have the Shadow Copies tab, you can scroll through the list of ALL disks.
Thanks for posting this! It seems to be working for me too!
Question: can anyone explain why Chucko can only use 300GB of his 2TB drive? Is the other 1.7 being used for the Full backup?
Gabe, what I understand from Chucko is that on his 2TB drive, he set it up to allow 300GB of Shadow Copies. That leaves the other 1.7TB free for backups and perhaps other data.
Mark, I think I know where my confusion is at and I’m betting it’s just going over my head…allow me to explain. In your blog post you talk about 100GB of your 698GB F: drive is free. You then went into diskshadow to delete shadows and then said, “so now 122GB free (17%), 175GB in shadow copies”. What I understand is that you have a 698GB drive and with 122GB free, this means 576GB is used. What I don’t understand is that 175GB is shadow copies. How do you find that number? Also, is there a way to see that 401GB is NOT in shadow copies?
Gabe – right-click on any local drive and select Configure Shadow Copies. You will see how much is being used by Shadow Copies on each volume.
I don’t know of any way to determine the non-shadow-copy space other than the math you are already using: total capacity less shadow drive storage equal non-shadow space available.
Mark – I guess I’m not confused then because that’s how I understood it. I wanted to make sure there wasn’t some data from VSSADMIN, WBADMIN or DISKSHADOW you might be referring too and that I was missing. I’m going to play around with the limit on the drives. Before setting this limit, I could go about 2 months and they’d ultimately get full and then the errors would start because nothing was being automatically deleted. Now, even with only a 300GB limit set, 5 copies are on each drive (Monday – Friday) and during a backup, one falls off and a new one gets put on with no errors thrown. Thanks a lot for the feedback and again for posting this blog.
Gabe – sounds like you’re on the right track!
Pingback: W2K8+ wbadmin backup tips | Jonson Yang
I think Chucko is right about small limits. Last night I had my first “there is not enough disk space” failure since I’ve configured SC limits on all of my external drives. Wednesday’s external drive is 2TB and like all 5 drives (M-F), I’ve configured them to have a limit, though unlike Chucko, I chose 1.5TB instead of 300GB. Last Wednesday completed and Vssadmin showed it having 474GB Used SC Storage. WBAdmin showed 11 copies starting at October 15th. This week it failed after it hit 539GB and still has 11 copies with the oldest being October 15th. So, even with a limit set, the dreaded “not enough space” can still occur and the oldest copy not be deleted. I’m going to set all the drives back from 1.5TB to 300GB and they should succeed on a regular basis. I tested with a limit of 150GB when I first found this blog but when I read Chucko’s comment I changed all my drives to near max to see if I would have the same experience.
Gabe, that makes sense. The whole point of this exercise is to force the shadow storage to be *less* than it would be by default.
Please help:
I had the space full up issue on windows backup on windows 2012 R2 server. I have assign the letter to E to the backup on disk management and re-set the volume shadow to 25350 from no limit. The full space on external back up drive is 931 GB. I then ran a manual backup but come up with error “There was a failure in creating a directory on the backup storage location” . how to rectify this? I have now change the volume shadow copy size back to no limit but still no luck. please help
Ahilan – I would open Backup, choose Backup Schedule, and go through the wizard again. You may need to switch from “Back up to a hard disk that is dedicated for backups” to “Back up to a volume.” On the next page, choose “Modify backup storage destinations,” then on the next page, choose your new drive E:.
Thanks for your response Mark. I have solved it by remove the drive letter and remove and re-insert the external drive. Then started the backup again which worked.
Ahilan, that probably works because Backup is once again able to “own” the drive. If you want to try again with the drive letter assignment, change the backup destination in Backup, then you can test the shadow copy restriction.
Thought I’d follow-up since my last post. In the 4 months since I’ve set the limit to 300GB, I’ve not had a single “out of space” issue. Really glad Chucko posted that suggestion.
As for drive letters, I currently have all 5 of my external drives configured with a drive letter. Maybe Alihan is having trouble because it’s internal? After configure the drives through the backup setup wizard, I did the letter assignments through the Disk Management utility. I don’t have any errors during backup. The drive letters make it easier when working with the command line utilities so I don’t have to read or copy/paste all the GUID’s. One limitation is that they all drives have to be the same drive letter. Originally I wanted 5 different letters and even though it let me assign “K” or “L”, as soon as I removed that drive and plugged it back in, it dropped to the next available letter, which in my case is “F”.
Also, I’m really glad you have this forum, Mark. Thanks for sharing your knowledge and monitoring these posts.
Pingback: Deleting old Windows Server Backups from an external hard drive | DavidAtkin.com – A Technology Blog
thanks for this. literally the only article on the internet that actually explains this and has a solution.