Bug (?): 100% CPU load while playing Queen's Wish

[Fingolfin]

So I've been tremendously enjoying Queen's wish so far, but one thing is really bugging me: While playing it, one of my laptop's CPU core is maxed out at 100% load. This causes the machine to heat up, the fan spins up, and it gets *loud*.

 

This is on a MacBook Pro 15" (2018, with hexacore i7 at 2.6 Ghz;  32 GB RAM;  Radeon Pro Vega 16),  running macOS 10.14.6 . It happens both with the Steam version, as well as the DRM free standalone version.

 

Anybody else having this issue?

Edited September 16, 2019 by Lord Fingolfin

[Randomizer]

I didn't check CPU usage, but Spiderweb games almost always cause my laptops to overheat. This goes back for over 10 years on different models. At least my MacBook has a quieter fan than my old one.

[Fingolfin]

Well, that may be, but it's not really a nice situation, is it? I mean, I love these games, but the graphics and game play logic hardly should require that kind of CPU power. As a programmer, my impression is that something is doing a busy wait instead of only using CPU resources when action is needed (i.e., an event driven game loop). And indeed, looking at a sampling of the Queen's Wish process, almost all the time is spent inside SDL_PollEvent. I wonder if Jeff ever considered using SDL_WaitEvent instead, or perhaps SDL_WaitEvenTimeout with a timeout of, say 10 milliseconds -- that allows for 100 Hz game updates, which should be plenty?

[Chopkinsca]

I run the newer games cooler by making a custom power plan and lowering max CPU%. I haven't had any issues doing this but it is a bit annoying.

[Kelandon]

9 hours ago, Fingolfin said:

So I've been tremendously enjoying Queen's wish so far, but one thing is really bugging me: While playing it, one of my laptop's CPU core is maxed out at 100% load. This causes the machine to heat up, the fan spins up, and it gets *loud*.

 

This is on a MacBook Pro 15" (2018, with hexacore i7 at 2.6 Ghz;  32 GB RAM;  Radeon Pro Vega 16),  running macOS 10.14.6 . It happens both with the Steam version, as well as the DRM free standalone version.

 

Anybody else having this issue?

I've been having the same issue on a similar computer (MacBook Pro 2019). I flagged it for Jeff near the end of beta and he basically just shrugged. I'd love to know how to fix it. 

[Randomizer]

Jeff's wife has had the issue since I think Geneforge 4 or maybe earlier so he isn't concerned. 

 

Only seconds after starting the game I had the fan on mine.

[TriRodent]

On my laptop, whenever I want to start up QW I have to turn off the grid computing that's otherwise (4 cores) always cranking away in the background with no overheating issues.  If I don't I rarely make it through the startup/loading of the save game before it shuts itself down from overheating. It's annoying to say the least

Edited September 16, 2019 by TriRodent
Windows if that matters

[Spidweb]

MacBooks are infamous for their poor heat management. When my wife was playing Into the Breach on her laptop the other days, the fans sounded like jet engines. I don't "just shrug" about it. It makes me really unhappy (especially when I fall victim to it). But I can't make Apple correctly engineer their laptops. They're too busy making the new butterfly keyboards never work right.

 

if you really still believe it's my fault, Google "macbook game fan running".

 

Truly, I am sorry about this, but it's past my control.

[cef]

FWIW, this also happens on desktop Macs.

 

27" 2019 iMac, 64GB, upgraded i9 CPU, upgraded Pro Vega 48 8GB, 1TB SSD, CPU goes to 100% (sometimes 101%) on Queen's Wish the moment it launches, and there it stays. 

 

I realize Spidweb says in the post above that it's past their control, but I thought I'd nonetheless add another data point in case others search for desktop Mac 100% CPU usage issues on here as I did.

 

To further add, it was also doing this on my 2015 13" RMBP (i7 16GB), but I don't plan on running it there what with the new iMac.

 

All of that said, looking forward to the game! I named my character Aldous 😝

[Fingolfin]

 

9 hours ago, Spidweb said:

Truly, I am sorry about this, but it's past my control.

 

Oh come on! You wrote that engine, and the problem is caused by that engine, not the design of the laptop fans. The game shouldn't be using 100% CPU time, esp. when idling. 

 

So as a quick proof of concept for my suggestion above, I took out my disassembler to hack the SDL2 copy bundled within Queen's Wish, to see if I could fix the issue. Luckily, it seems the SDL2.framework you are using was compiled without optimizations (which, by the way, is another thing where perhaps some performance is lost?). But for me that was perfect, because that gave me ample opportunity to replace useless instructions in the code with alternate code.

 

First, I edited SDL_PollEvent (or rather, SDL_PollEvent_REAL) to call SDL_WaitEventTimeout with a timeout of 1, instead of 0 (basically I replaced two useless mov instruction with NOPs, plus "inc esi").  That resulted in a Queen's wish that only took 2% CPU time, but the mouse was super sluggish. Turns out that SDL_WaitEventTimeout(_REAL) always sleeps for at least 10 milliseconds, which apparently is a bit long. So I located the call to SDL_Delay in there, and replaced the "mov edi, 10" by "mov edi, 1".

 

The result: an apparently fully function Queen's Wish binary that "only" takes up 10% CPU time when idling. Granted, that's still not ideal, but sufficient to keep my mac nice and cool, and also use up much less battery power. I bet one can do better, but without knowing anything further about your game's main event loop, I can't say how (OK, I could now reverse engineer *that*, but while that is a fun thing to do, I'd rather use the time to play some more Queen's Wish ;-).

 

Of course you could now simply perform the same changes, which would require using a modified SDL. But since you have access to the source code of Queen's Wish, there are muchbetter ways: Basically, when calling SDL_PollEvent, if it return 0 (= no event was found), then call SDL_Delay(1).  There are various possible variations of this, of course, depending on how your main game loop looks like; also, if you determine that even a delay of 1 millisecond is a bit too much, one can also sleep with nanosecond resolution, but unfortunately SDL does not expose a portable API for this, so it'd require some "#ifdef MACOSX" style of logic (not too horrible, though).

[Fingolfin]

I should actually clarify: the 10% are for idling in the background (actually, it's more like 8-9%). When idling in the foreground, it's 23-24%. When doing something, this goes up to 40-50%, depending on what kind of "action" is going on. Which is fair enough, of course.

[Unbound Servile]

Fingolfin, your last post piqued my interest.
It is not required - and even a bad idea - to sleep the thread that handles the message pump for a fixed amout of time when one has chosen for an event-driven architecture.
Either use a blocking message queue or use a Mac OSX alternative to DwmFlush (a Windows function call that locks the current thread until the next DirectX batch draw).
On Windows (and Mac OS classic), system timers are also an option, however other frameworks often mess around with timers, one of those being XAudio2.

 

However, even on Windows (10), Queen's Wish uses an awful lot of processor resources, while the game uses DirectX
(http://spiderwebsoftware.com/queenswish/support.html).

This should not be happening, unless Jeff Vogel (or SDL2) is doing something wrong. However, when I google a couple of variations of "game loop Win32" or "game loop DirectX", every implementation I see is even more horrible than the previous. Most notably is the usage of "PeekMessage", a call which doesn't lock the thread that handles the corresponding window handle's message queue. This is a mistake.
A correct message pump uses GetMessage to retrieve messages from the message queue. A second thread sends (user-defined) render messages to the active window (this thread should be delayed, preferrably with DwmFlush; but the message pump should never ever be delayed for a fixed amount of time). The handler for the render message initializes the DirectX batch draw. A handler for the wm_paint message is used the validate the updated window areas.

 

What does matter is that Fingolfin's changes have a rather significant effect on performance, and, that with a couple of minor changes, Jeff Vogel could improve the performance of his games without much ado. I hope Jeff realizes that this issue isn't as much out of his hands as he seems to think.

 

EDIT - to Fingolfin who posted after this post: I'm sorry, I must have accidently pressed the "Save" button before I was done writing.

Edited September 17, 2019 by Unbound Servile

[Fingolfin]

1 hour ago, Unbound Servile said:

Fingolfin, your post piqued my interest.
It is not required - and even a bad idea - to sleep the thread that handles the message queue for

 

That question was cut off, so I am not completely sure what you wanted to ask. But anyway, sleeping for a short duration is fine, and in fact standard with SDL apps.

 

I also noticed that considerable time is spent on querying for joysticks -- Jeff, looking at the disassembly again, it seems you are passing SDL_INIT_EVERYTHING tp SDL_Init, which include SDL_INIT_JOYSTICK and SDL_INIT_GAMECONTROLLER. Replacing that with "SDL_INIT_TIMER | SDL_INIT_AUDIO | SDL_INIT_VIDEO | SDL_INIT_EVENTS" might be a good idea.

 

Also, perhaps turning on V-sync would help (add SDL_RENDERER_PRESENTVSYNC to the flags passed to SDL_CreateRenderer).

[Unbound Servile]

1 hour ago, Fingolfin said:

But anyway, sleeping for a short duration is fine, and in fact standard with SDL apps.

I will not argue over what is considered "standard", let go over the workings of some 3rd-party framework.
What I will argue over is that if one uses common sense, one should come to the conclusion that the message pump should never* be delayed for a fixed amount of time for two noteworthy reasons:

 

• There is no guarantee that the scheduler will respect the given delay time, neither on Windows nor on Mac OSX, albeit that Mac OSX is more time-precise than Windows; so for time-critical operations such as frame rendering, a fixed-time delay is a bad idea. If one wants to delay the message pump per sé, use a frame timer.

 

• A delay applied to the message pump will delay all messages destined for the given window handle. That includes mouse events. This is why you saw the mouse behaving sluggishly. With your approach, one can opt for a longer delay time which will reduce the amount of needlessly-rendered frames but will also reduce, among others, cursor updates. Or one can choose for a shorter delay time, which will still cause more frame updates than necessary.

 

My approach, as described above, will yield a smooth play experience while keeping used processor time at a minimum.

 

1 hour ago, Fingolfin said:

I also noticed that considerable time is spent on querying for joysticks -- Jeff, looking at the disassembly again, it seems you are passing SDL_INIT_EVERYTHING tp SDL_Init, which include SDL_INIT_JOYSTICK and SDL_INIT_GAMECONTROLLER. Replacing that with "SDL_INIT_TIMER | SDL_INIT_AUDIO | SDL_INIT_VIDEO | SDL_INIT_EVENTS" might be a good idea.

 

If Fingolfin's findings are accurate, then it is clear that Jeff is not proficient with the framework he is using.
One can also question why SDL would require a "considerable amount of time" to query for a joystick. No wonder I never liked 3rd-party code.

 

*There are some edge-cases where applying a fixed-time delay to the message pump might come in handy; when a timeout is required while querying for messages sent over a network, for instance.

Edited September 17, 2019 by Unbound Servile

[Kelandon]

21 hours ago, Unbound Servile said:

What does matter is that Fingolfin's changes have a rather significant effect on performance, and, that with a couple of minor changes, Jeff Vogel could improve the performance of his games without much ado. I hope Jeff realizes that this issue isn't as much out of his hands as he seems to think.

For me, this is the bottom line. Spiderweb games have been a little energy-intensive in the past, but I've never had a problem like this, and no other game that I've played is this energy-intensive. It's not the computer. It's the game.

 

I know that it's painful to do this because you're worried about breaking everything, but Jeff, could you take a good, solid look at the suggestions here? They seem pretty straightforward and potentially quite useful.

[Spidweb]

I want to be clear that my games having flaws that someone good at programming could fix easily is very upsetting to me. You have convinced me that if I deep dive into the source for SDL2 I could rewrite it to optimize it.

 

As for whether I will do that, for Queen's Wish or for the Geneforge remaster, is still up in the air. It will take time for me to learn how MacOS event-handing works now, make the desired changes, and then more time to test it, and we're really scrambling with everything else for a game release and for all our Kickstarter obligations. It is on our to-do list, and we've bookmarked this thread so we know what to look at. If not for this game, the next one (when we can make sure the engine changes get a full test pass).

 

Sorry I can't give a better answer. In the end, I'm just one guy with one brain, and serious techie stuff is outside my capabilities. If this is a big issue, instead of yelling at me, just write us at support@spiderwebsoftware.com and you can get your refunds no questions asked.

[Fingolfin]

Jeff, first off: nothing could be farther from me than a refund. I love that game far too much, like I love all your other games (like many people here on the forum, I've been with your software since I was a teenager and discovered Exile :-).

 

So, please take our complaining as a wish to see what we love become even better :-). (OK, well, I can of course only speak for myself, but that's genuinely the impression I have from everybody else in this thread).

 

That said: You don't even have to mess with SDL2 itself for an immediate improvement. As I explained: Just locate your call to SDL_PollEvent, and if it returns 0 to indicate that no event was handled, just call SDL_Delay(1).  Of course it may be possible to do something better, but without seeing the code, I can't comment (I would be happy to help with that, though, if desired: If you send me a copy of your main game loop (not the rest of the code), I could possibly make better suggestions based on that. I'd be happy to sign an NDA or anything for that, too.)

 

The other suggested changes also don't require you to mess with SDL internals:

- change your call to `SDL_Init` to use slightly different arguments

- change your call to `SDL_CreateRenderer` to use slightly different arguments

- switch to a recent SDL2 version for Mac, that was compiled with optimizations on; e.g. as available here https://www.libsdl.org/download-2.0.php)

 

 

Unbound Servile has a point though, the design of the SDL2 event handling is not quite the best (in fact, even in Classic MacOS, they started out event polling, using GetNextEvent, and a major improvement over that for the MultiFinder was the addition of WaitNextEvent -- 30 years ago; SDL2 sadly is falling back to the old polling model. In its defense, though this is the result of a compromise between many, many different platforms and what their needs are (I should mention that I used to be a contributor to SDL1, many, many years ago, but it's been a long time since I worked on it or even used it, so some of what I say here may be out dated. I am pretty sure that if one talks to the SDL maintainers, like Sam Lantinga and  Ryan C. Gordon, who are doing an outstanding job on supporting it across many platform, then they would be willing to accept patches to improve the situation regarding event handling, or even work on some improvements themselves. In fact, this was discussed like 2 years ago on their mailing list / forums, so for some background info, you can read up here: https://discourse.libsdl.org/t/blocking-sleep-in-sdl-waiteventtimeout/23255/12 )

 

 

Anyway, back to Queen's Wish: Should you be willing to mess with SDL itself and compile your own version, you could try the following slightly hackish solution *instead* of inserting SDL_Delay calls as I suggested above: In file src/video/cocoa/SDL_cocoaevents.m change line 431 from this:

 

    NSEvent *event = [NSApp nextEventMatchingMask:NSEventMaskAny untilDate:[NSDate distantPast] inMode:NSDefaultRunLoopMode dequeue:YES ];

 

to this:

 

  NSEvent *event = [NSApp nextEventMatchingMask:NSEventMaskAny untilDate:[NSDate dateWithTimeIntervalSinceNow:0.001] inMode:NSDefaultRunLoopMode dequeue:YES ];

 

 

Here we insert a wait of 0.001 seconds = 1 millisecond. You can of course experiment with this and try e.g. 0.01 or 0.005.

[Spidweb]

One of the difficult things about the job is the way it exposes your shattering ignorance to the world. SOmetimes multiple times a day.

 

Anyhoo, you're right, of course. That SDL_Delay command dramatically reduces CPU load without impacting performance as much as I can see. I'll put it in the Mac Windows and iOS versions and see how it goes. It'll probably be a week or two until v102 comes out, so I'll have to ask for your patience until then.

 

I thought to fix the problem I'd have to make and maintain my own SDL2 fork FOREVER, and I really didn't want to do that.

 

Thank you for your extreme patience helping me with this.

[Lattan]

Oh man, thanks for helping with this Fingolfin, and to Jeff for listening because I know it can be really obnoxious for strangers to come in and complain about one's code.

[Fingolfin]

Thank you, Jeff, for making wonderful games 🙂

[Olivil]

Jeff, would it be possible to apply the same improvements to the Avernum remake trilogy? I adore those games but I cannot play them unless my 2019 MacBook Pro is plugged as the battery just disappears with the CPU pegged at 100%... Meanwhile I just finished the Geneforge 1 remake demo on a single charge (and in silence!!)

 

Thanks so much for those games  hope this message finds you.

[TriRodent]

2 hours ago, Olivil said:

 

Thanks so much for those games  hope this message finds you.

 

Jeff usually doesn't come to the forums much unless/until a new game is released & then it's generally only in that specific sub-forum.  However all is not lost, you can always email support@spiderwebsoftware.com to get a message to him. I'd provide a link to this thread so that he knows what you're talking about.  That's the good news.  The bad news is that I doubt that he'll put out a patch/fix for a game/s that have already been out for a few years.  But it can't hurt to ask.  You also may want to wait a couple of months as with the remake of Geneforge 1 coming out in a couple of weeks...he may be a tad distracted with that.  Glad that that one (demo anyway) worked for you with little issues.

 

And finally welcome.  There's a box over yonder that you're supposed to leave your sanity in. However there has been pilferage recently by some of the assorted reprobates around here with their attempts to appear more sane...so...up to you if you drop said sanity off or not...

[Olivil]

Thanks for the input, I have emailed Jeff. We'll see what happens.  

Like the games no matter what so nothing to lose!

 

Edit: Jeff answered, sadly those games don't use SDL so they can't benefit from the improvements.

Edited February 8, 2021 by Olivil