Experiment: access UE from a VHDX instead of a ZIP. #1450
Conversation
|
Well, this certainly makes the UE install faster. 2m43s in this branch, versus 8m5s in the most recent main build. The plugin build itself was 1m17s slower in this branch. That's probably within the normal variability of Unreal plugin builds, so hard to say if it's significant (though a little overhead from the on-the-fly decompression is to be expected with this PR). Overall the whole process was about 4 minutes faster in this PR. Another upside of this PR is that it requires significantly less disk space to be available on the runner. But one downside is that the bigger size of the VHDX versus the ZIP means we're going to pay more in data transfer costs from S3. At least until we move the UE distributions to Azure, which should drive the cost to zero, because then the transfers will be within the same cloud. |
Currently we download Unreal Engine from S3 in a ZIP file, and then extract it. The download is pretty fast (about a minute), but unzipping takes much longer. So this is an experiment to see if we can make our CI go faster by downloading a VHDX instead. The VHDX uses NTFS compression, so it's reasonably small, but still much bigger than the ZIP. (ZIP is 14 GB, VHDX is 24 GB). But once we have a VHDX locally, we can mount it directly; no need to spend time unzipping the whole thing.