They say a picture speaks a thousand words:
Utilisation is of all cores and varies between about 50% and about 66%. What is it doing?!?!
IIRC it does some cpu work for file integrity verification.
It maintained this level of usage for the entity of the download which was probably around 30 minutes.
My guess is checking file integrity in real time as you download it, to catch potential man in the middle meddling.
Did not notice any issues with actual pc usability when getting the client myself(but then again 16t and all that)
I haven’t noticed any issue myself either, but the download only takes a minute or two. I agree that for 30 mins that looks like a lot, I’ll mention it to Keith.
2 Likes
I didn’t notice any performance impact either. But still, it seems excessive.
How big is the download? That screenshot is showing 23.2Mbps… 30 minutes of that is 5.2GB. Note 30 mins was just a guess but it was certainly more than 20 minutes.
Regardless of it’s size, let’s assume the download is split into 64kb individually verified chunks (which would be excessively small). My PC can generate the SHA1 hash of 1319 64kb files per second (675.2Mbps) with CPU usage around 20-25%.
If you increase the chunk size to 1MB, it can hash 390 files a second (3120Mbps), with similar CPU usage.
My tests include reading from SSD whereas i guess the launcher would have the downloaded chunk in memory.
If this is a laptop, or even a desktop CPU in Balanced mode, you could be seeing the effects of speed scaling.
As long as no CPU core reaches 100% use the CPU knows that it is running fast enough. That can result in things like all cores at 80% load at 800 MHz.
You can view the current CPU speed on the Performance graph page.
I tried to replicate the issue without success so far, mostly because my connection is very fast ( 250 MBps ) so the download only takes two minutes. Cpu usage during that is around 45%.
The download size is 2.7 GB, so either it didn’t take 20-30 mins, either the average download rate is half lower than in your screenshot. Which makes even less sense, as the cpu should only get busy as it receives a chunk. So on a slower connection, the cpu should mostly be idle awaiting for the next chunk to be downloaded.
We’re keeping this issue in mind, however as you can imagine, it isn’t a very high priority at the moment.
Next patch I’ll try to gather more data
1 Like
I’ll take a look as well.
2 Likes
Maybe nothing to do with but I already seen full cpu usage on some 2d games I wrote because I wasn’t limiting the max framerate. With the limit I was at 1-2%
Full CPU usage on most games comes from the engine not enforcing a target update rate, the update loop runs continuously, in essence you could get 200fps in such a setup.
This problem however stems from @hrobertson running KSP with 4GB worth of mods. 
That’s why on the Discord channel I tongue-in-cheek commented that it was probably the progress bar causing the problem. If Flavien has tested it and not been able to reproduce then it must be something about that particular environment. Could the disk be heavily fragmented?
Same issue this patch.
The fact that I was running KSP is irrelevant but this time I had nothing else running.
It’s downloading to an SSD.
Rate according to Task Manager is steady at around 24Mbps.
CPU clock was scaled up, not down. i5 4670K at ~3.7GHz
GPU usage varied throughout the download. For some of the time it was at 20% and other times it was at 0%… Maybe it is something to do with drawing the progress bar?!
I looked at what the process was doing and saw it writing 4KB chunks which seems small. Is it separately verifying 4KB chunks?
It downloads, hashes, and decrypts 4kb chunks while writing them to disk.
I’m not saying this is the reason for the high CPU usage as I’d expect other people to report the same problem if it were, but 4KB is an incredibly small chunk size!
As I said in post #6, even 64KB would be excessively small.
I can calculate the sha1 hash of ~1500 4KB files per second. Thats about 48Mbps.
I hadn’t anticipated it being encrypted as well! Depending on the algorithm that would add significant CPU time.
If we’re super generous and say the decryption takes the same time as the hashing (in reality I expect it takes longer), then we’re in the situation where the chunk processing rate matches the download rate.
With a slower decryption time, processing the chunks becomes the bottleneck.
Is there any reason not to increase chunk size to upwards of 256KB? Regardless of whether this has anything to do with the high CPU utilisation, it seems like a no-brainer.
I’ve been tinkering with this over the last 24 hours and have made significant improvements. Will roll out a patch in the coming weeks.
5 Likes
Is that launcher patch coming pre-alpha?
I’ll be releasing it today.
1 Like