adventures in software development: automatic, scheduled wsl2 backups

As we rejoin our brave adventurer, we find that he has hurdled another piece of the backup puzzle, to be faced by yet even more hurdles when automatically scheduling Windows Subsystem for Linux (WSL2) backups. He has scoured and squirmed his way to timely exports of the distribution of concern and beat back weird interactions with the foul, yet, beloved beast, Docker.

The first step was creating an archive of the instance. That is made possible by the wsl command using the export sub-command.

# Get a list of WSL `distributions` and their status$ wsl -l -v  NAME                   STATE           VERSION* Ubuntu                 Running         2  docker-desktop         Running         2  docker-desktop-data    Running         2# Note the current directory/drive and change to a location to save the exported distribution to; I recommend not saving this to a location that is cloud synchronized, because it will be large and extremely time consuming to upload (and then sync to all connected devices).$ pwdPath----C:\Users\justi\Documents# Export a distribution for backup# Note that this takes it offline and for several minutes, but ensures absolute consistency$ wsl --export Ubuntu$ wsl -l -v  NAME                   STATE           VERSION* Ubuntu                 Converting      2  docker-desktop         Running         2  docker-desktop-data    Running         2# Verify it completed once the 'State' shows as 'Stopped' for the given distribution.$ ls    Directory: C:\Users\justi\DocumentsMode                 LastWriteTime         Length Name----                 -------------         ------ -----a---           4/12/2021  2:52 PM    12001935360 Ubuntu.tar

Now that we are confident that we can archive the distribution, the next step is to do it under automatic execution.

After fiddling around from near zero powershell or programatic task-scheduler knowledge, I currently can run the backup under the task scheduler for reoccurring, time-of-day and day-of-week based execution. Personally, I set it to the late evening-to-very-early morning hours, like 2:30AM locally, but only weekly, because that is my risk vs cost threshold and don't feel more frequent backups would be helpful; however, you might. I could register the task through the 'Task Scheduler' GUI, but what fun is that? Lets look a little more .

Initial registration of the task is roughly prepared using power shell script. The archives are date-tagged in the filename scheme to easily identify when they were created and to make sorting and visualizing other aspects like growth easy to see.

# .\Register-ScheduledBackup-WSL2.ps1...# this is the meat; a lot of sides surround the main course, but this is how you add a scheduled taskRegister-ScheduledTask -Action $action -Trigger $trigger -TaskPath $TaskPath -TaskName $taskName -Description $taskDescription -Principal $principle -Settings $settingsSet...# using a subscription to the task scheduler log$subscription = @"<QueryList>    <Query Id="0" Path="Microsoft-Windows-TaskScheduler/Operational">        <Select Path="Microsoft-Windows-TaskScheduler/Operational">            *[EventData[@Name="ActionSuccess"][Data[@Name="TaskName"]="\$TaskPath\$taskName"][Data[@Name="ResultCode"]="0"]]        </Select>    </Query></QueryList>"@...# add a task that fires based on that subscriptionRegister-ScheduledTask -Action $dockerRestartAction -Trigger $dockerRestartTrigger -TaskPath $TaskPath -TaskName 'Restart Docker' -Description "Restart Docker on completion of '$taskName'" -Principal $dockerRestartPrinciple -Settings $dockerRestartSettings...

Note: the above example only creates the scheduled task. To update the scheduled task requires a different script. That is not shown. It is not implemented. Instead, I just unregister, and register anew. The actual backup script that is run via the above registration process is:

# .\Backup-WSL2.ps1# parameter handling and logging omitted; here is the crux$command = (Get-Command wsl).Definition$dateExecuted = $(Get-Date -Format FileDate)$commandArgs = "--export $Distribution $DestinationPath\$Distribution-$dateExecuted.tar"# execute the backupInvoke-Expression "& $command $commandArgs"

There is a problem with this. For some reason when the export/archive of the WSL distribution finishes, the connection to the docker socket of the host is lost or corrupted or something, so Docker needs to be restarted before your distribution can interact with the host docker system: We

I could reboot the entire machine, but just restarting docker seems to be sufficient. I could use the Docker Desktop GUI/menus, but automatic restart would be nice while the underlying challenge remains. TLDR; Be .

Note: restarting all of docker has it's own consequences, so proceed cautiously as any other running containers on the same host, your machine, will also be stopped.

The process to accomplish a docker restart via powershell is something that I am less comfortable showing, as it is based on a general instruction demonstrated in the following link:

Stackoverflow: Restart docker Windows 10 command line

The reason I am less comfortable showing how I do it because:

1. Licensing attached to Stackoverflow user-contributed code may impede personal or professional efforts.
2. The example solution does not work cleanly as is. So modification is required, and then I refer back to '#1' .

But I will highlight the relevant commands here (with parameter handling, logging, and sleep or await commands omitted):

...# kill the underlying Windows services used by docker, as gracefully as possibleStop-Service -InputObject $_ -ErrorAction Continue -Confirm:$false -Force...# kill the user facing docker desktop app without delay (the app has little consequence on the underlying docker system, and is mostly just a dashboard; not a criticism) Stop-Process -InputObject $_...# start the desktop app because it generally takes the longest to completely start and won't complain terribly if the underlying docker system isn't ready or available$_.Start()...# start the underlying Windows services for dockerStart-Process -FilePath $clientAppPath -PassThru |...# execute a `docker info` command and parse the results for indicators of readiness$healthCheckResult = $($ServiceHealthCommand)...

The complete script used to restart docker is:

.\Restart-Suite.ps1

In any case, once Docker is restarted, I am able to rejoin/restart my now safely archived distribution.

The complete effort to-date is found here:

backup-wsl2

Repeating the disclaimer in the repository README: this is very rookie stuff you will find here, and more importantly, it may be a terrible approach to managing wsl2 backups at the end of the day; but it is an option and maybe inspirational to someone with more foresight, talent, and time.

Insert Tim Allen as Tim the Toolman Taylor trope. I connected a service restart to the end of the backup script to automatically restart Docker after exporting the wsl instance at a regular time of day when I wasn't heavily using the machine. I should add more steps:

• modify my process to also copy a complete, intact archive periodically to another location; one other option might be rysnc the archive chunks to remote storage for isolation from the source (likely you want to stage it to local network storage first and let that NAS/SAN sync to offsite during off-peak periods)
• test the recovery process; if you can't recover from it, the backup is not worth making
• shrink the live distribution somehow, as the growth of the image has a negative affect on the archive process in terms of time and space (experience shows this is going to be fruitless in most cases)