Speed Up Driver Package Downloads for ConfigMgr OSD

In ConfigMgr, when doing OS Deployment from HTTPS enabled DPs, or via Peer Cache (uses HTTPS too), or via BranchCache (encrypted data + encrypted transfers), you will find that using traditional driver packages, with thousands of small files, is not going as fast as you would like. In fact, a single deployment can take a good 5-10 minutes to download a large driver package depending on the environment.

Note: While this post deals with driver packages in ConfigMgr, you can apply the same strategy for application packages that have many small files, or simply really large applications that could benefit from a bit of compression. For example, I've worked with organizations that create wim images for their larger applications, and instead of extracting the wim prior to running the application setup, they simply mount the wim image, and run the installer from that one.

Update, March 1, 2020: Maurice Daly is has updated his Driver Automation Tool to create WIM, ZIP or 7-zip versions of the driver packages too. It's a fantastic tool, I highly recommend using it.

TL;DR

By switching your ConfigMgr driver packages to archived versions, you don't only make them much smaller, but also increases overall download speed and peer-to-peer efficiency. Usually between 5-10 times faster depending on content source. WIM and Zip will give you much better deduplication rates, and better P2P performance. But 7-zip format will reduce package size a bit more.

Some Real-World Data

In this example, I was using BranchCache as the peer-to-peer solution for ConfigMgr OSD, but you'll see similar behavior for normal HTTPS downloads either directly from the DP, or HTTPS downloads via Peer Cache.

As an example: When deploying a Dell Latitude 7250, with its 1 GB driver package (1000+ files) in my lab. The driver step alone took 7 minutes, and this was on a 155 mbit network with a remote DP. Deploying five of them at the same time, ended up with a total of 10 minutes for the drivers step (peering saved me some WAN bandwidth). Bottom line, downloading, or peering large packages with many small files is just not very efficient. But it can be much better…

Note: When zipping driver packages you do lose the single instance store benefits of the ConfigMgr Content Library for these packages, but in my testing the benefits of reducing package size by zipping driver packages totally outweighs the single instance store. As an example, I imported 26 driver models the traditional way, their source size was 43 GB, and the Content Library grew with 29 GB. For the zipped versions, even on the lowest compression rate, the resulting size was 22 GB. When adding data deduplication to the distribution point, another 8 GB was saved, and disk usage was down to 14 GB.

Zip or WIM your Driver Packages

Many organizations have already switched to use legacy packages for the drivers, and have DISM just inject them during the task sequence. But why not take it one step further, and simply zip the content in your driver packages.

You can either use PowerShell (5) or 7-Zip to archive the driver package into an individual zip or WIM file. If you are optimizing for BranchCache data deduplication efficiency among many packages, use the Compress-Archive cmdlet with compression level set to Fastest, and for 7-Zip, use the zip format with the deflate algorithm. BranchCache really likes these algorithms, and is quite good in figuring out deltas for these formats too. If you are optimizing for fastest possible deployments, and don't pre-cache lot of data, 7-zip on ultra compression will create quite small packages.

Note: While it may be tempting to use super-high compression algorithms, doing so will lower data deduplication efficiency, and delta updates of packages (BranchCache uses data deduplication). That being said, higher compression algorithms will usually reduce the size with another 30 percent or so, so for direct download from DP having data deduplication enabled, or from Peer Cache, which can't do data deduplication, you might want to use it.

Below is the syntax for Compress-Archive cmdlet:

# Compress a single driver package
Compress-Archive -Path "E:\Sources\OSD\DriverSources\Dell\Latitude 7250" -DestinationPath "E:\Sources\OSD\DriverSources\Dell\Latitude 7250\Drivers.zip" -CompressionLevel Fastest

# Compress multiple driver packages, looping through a sources folder
$DriverSources = Get-ChildItem "E:\Sources\OSD\DriverSources\Dell"
$DriverPackages = "E:\Sources\OSD\DriverPackages\Dell"

Foreach ($folder in $DriverSources){
    New-Item -ItemType Directory "$DriverPackages\$($folder.Name)"
    Compress-Archive -Path $folder.FullName -DestinationPath "$DriverPackages\$($folder.Name)\Drivers.zip" -CompressionLevel Fastest
}

After archiving my 1 GB driver package for the Dell Latitude 7250 model, the resulting drivers.zip file was about 425 MB in size.

After creating the drivers.zip file, you just copy it to a folder, and create a new legacy package in ConfigMgr using that folder as the source. And, don't forget to distribute the package 🙂

Modify your Task Sequence

The final step is to configure your task sequence to use the Zip-version of your driver package. Basically it will just download the package first, and then extract it using PowerShell.

Note: If you follow this example, don't forget to add PowerShell to your boot image(s), or the Expand-Archive command is not going to work very well 🙂

There are many options to get the download/extraction going, but for example a regular "Run Command Line" action together with a WMI condition, will do just fine. The command line I used to extract the content of the Zip-version of the driver package is the following:

PowerShell -ExecutionPolicy Bypass -Command "Expand-Archive -Path .\Drivers.zip -DestinationPath %_SMSTSMDataPath%\Drivers"

The final step is to instruct the task sequence to inject the drivers that were extracted into the driver store using DISM. For this you add another Run Command Line action, with the following command:

DISM.exe /Image:%OSDTargetSystemDrive%\ /Add-Driver /Driver:%_SMSTSMDataPath%\Drivers\ /Recurse /logpath:%_SMSTSLogPath%\dism.log

Don't forget to add a condition to the command so it only runs if the folder exist. You don't want the task sequence to error just because you deployed a machine where you didn't need to use any drivers (or forgot to add them).

Additional Reading

If you want to learn more on how Data Deduplication and BranchCache works together I recommend reading the below post from Mike Terrill.

Getting Network Efficiency from Data Deduplication and BranchCache
https://miketerrill.net/2019/12/08/getting-network-efficiency-from-data-deduplication-and-branchcache

Jörgen Nilsson also have a good post on using 7-Zip for ConfigMgr Drivers

Distribute Drivers at Mach speed
Distribute Drivers at Mach speed – CCMEXEC.COM – Enterprise Mobility