Dolphin Progress Report: July 2015

Since the release of Dolphin 4.0, things have advanced quite a bit. With compatibility rising to their highest levels yet alongside features that seemed impossible and extravagant, users have been asking about the next stable for almost a year now. For previous releases, including Dolphin 4.0, the staff mostly ceased work on new features while crashes and regressions were addressed. In the case of working toward Dolphin 5.0; there were just too many interesting and exciting features on the way to risk stalling out by asking developers to wait. So a release was delayed indefinitely.

So in mid-June, we decided to do things a little differently. Instead of slowing Dolphin down for a release, at 4.0-6727 we forked Dolphin into two branches. The development branch has continued forward as usual with all the bells and whistles without worrying about impeding the next release. Meanwhile, all commits relating to crashes, regressions, and other important fixes would not only be merged to master, but also to stable. This allowed the developers to continue developing the latest and greatest features, while still preparing a stable successor to Dolphin 4.0.

Today, we're happy to announce the first release candidate for Dolphin 5.0! Dolphin 5.0-RC is now on our Downloads page. These builds need to be heavily tested and any bugs; crashes or regressions found in Release Candidate builds should be tagged [RC] when reported to the issue tracker. While we will be switching issue trackers later this month, we intend to transfer all issues to a new tracker to make sure the hard work of our users throughout the years doesn't disappear.

Any future release candidates will be below the development builds on the download's page.

As to be expected until Dolphin 5.0 is complete, any non-essential crash fixes and features from this point forward will not be in the final release. For the latest and greatest features, the development builds are still your best option, and the Progress Report will keep on reporting what's new. With that, please enjoy this month's notable changes!

Notable Changes

4.0-6951 - Linux: Add an evdev based controller backend, to replace SDL by phire

SDL (Simple DirectMedia Layer) was created to be the ultimate hardware interface system. Most hardware interfaces are very specific, such as DirectInput exclusively serving controllers and other human interface devices, but SDL does everything. Controllers and input devices, disc drives, video, audio, and more! By combining every use case into a common open source framework available on almost every operating system, SDL was envisioned as being the final hardware interface tool anyone would ever need. While that dream ended up with the usual result, SDL is still under development and has been a de facto standard on Linux for years.

Unfortunately for Dolphin, the vastness of SDL has caused a number of problems over the years, and has slowly been removed bit by bit. For example, in 4.0-1628 the SDL backend was removed from Dolphin on Windows due to numerous crash reports related to webcams. SDL was reading the webcam as a controller with thousands of buttons, crashing the emulator. While Dolphin was partially at fault for some of these issues, there comes a point where maintenance costs more than the feature is worth. Since Dolphin on OSX has not used SDL since 2011, only Linux was left. Unfortunately, Dolphin still had many issues with SDL even in its ideal environment.

Notable SDL Bugs and Issues in Dolphin

  • Opening the controller menu after running a game would crash the emulator.
  • Running more than one game in a session would crash the emulator.
  • Running a game after opening the controller menu would crash the emulator.
  • Dolphin could not respond to SIGTERM signals.
  • Does not support all axis on various controllers, such as the Dualshock 3/4
  • Generally sporatic release schedule that doesn't align with our own generally sporatic release schedule.

None of these are outright horrible issues that couldn't be hacked around, but the base problem was that Dolphin was relying on a third party program when it really didn't need to be. phire took on the the task of writing an evdev backend that communicates more directly with Linux's controller handling. This allows project members to work on features as needed and address any crashes or bugs that crop up immediately. Linux users should notice a huge rise in stability as well as some nifty new features on certain controllers. Assigning the DS4 touchpad to Wiimote IR not only works, but is very useable and allows users a good alternative to the mouse.

4.0-6970 - IPC_HLE: Don't open/close files for every single file operation by phire

A long time ago during the development of the Wii Network branch it was discovered that a few games and the Wii System Menu would open multiple handles pointing to different locations of the same file. They expected that something written through one file handle could be read instantly from any other handle pointing at the same file. This worked perfectly fine on the Wii's simple operating system, but modern operating systems, such as the ones that run Dolphin, tend to break this assumption.

In order to make things stable, a work-around was implemented into the emulator to prevent problems. Dolphin would now close the file after each read/write and open it again before the next read/write. This forced the operating system to keep the multiple file handles in sync and fixes problem titles such as PokePark Wii and the Wii System Menu. At that time, no performance impact was noted.

Yet, after this commit, reports began to surface that saving was taking forever on certain computers. Despite a lot of research and testing, none of the developers could reproduce the issue and Dolphin 4.0 was released without the issue being tackled.

Super Smash Bros. Brawl

Some users still have anxiety attacks when they see Brawl's saving screen. Sorry!

Earlier this month, phire actually ran into the issue. Being a developer, he refused to let this issue go to rest again and immediately began to debug. Upon looking into it, he noticed that the afflicted games were breaking their writes into small (32 byte) chunks, which when combined with the above fix was resulting in dolphin opening the file, writing 32 bytes to the end and closing it thousands of times until the full 144KB was written. This really shouldn't be a problem as modern operating systems can sustain millions of open/closes of the same file per second, even with a spinning hard drive...

Unless the computer were running anti-virus software. The real-time protection feature which most anti-virus solutions provide involves intercepting every file open call and pausing the application until the file had been scanned for viruses. Combine this with dolphin's creative workaround and a strange problem occurs. This same file ends up being scanned for viruses thousands of times, which unsurprisingly brings any computer to its knees.

Rather than forcing everyone to uninstall their anti-virus software and use Linux, phire created a new workaround which catches attempts to open the same file multiple times and only opens it once and leaves it open. This single file handle now acts exactly as the Wii software expects but the user's anti-virus software will now only scan each file once.

4.0-7034 - Wiimote: Reconnect a disconnected Wiimote when a button is pressed by AdmiralCurtiss

When on the Wii, it's hard to notice how much the Wii will disconnect Wiimotes; any paired Wiimote will reconnect upon any button press on the Wiimote itself. That's why developers will often see complaints about Wiimotes timing out, Wiimotes being disconnected, having to manually reconnect Wiimotes, Player 2's controller not working, random Wiimote disconnects, on and on...

Dolphin doesn't communicate to Wii Remotes in the exact same manner as a real Wii. Dolphin handles all Wiimote connectivity itself, emulated or real. Even if the game disconnects the Wiimote, it remains connected to Dolphin even though the game thinks it shut it off. Unfortunately, the game has no way to know if the Wiimote is on or off; it sent the disconnect command and assumes it is disconnected, displays the disconnection notice and requires the console to send a reconnection notice to proceed. Dolphin on the otherhand assumes the Wiimote is still connected and requires you to manually disconnect and reconnect it before the game will notice it again. Users just had to know to manually disconnect then reconnect the Wiimote with hotkeys whenever this happened. For experienced users, this was annoying, but for people unfamiliar with Dolphin, it was a confusing quirk that led to much frustration.

Metroid Prime Trilogy

Usually the game will tell you when a Wii Remote is disconnected. But Metroid Prime: Trilogy's multiplayer menu notoriously does not, leading to a great deal of confusion

Throughout the Dolphin 4.0 era, Wii Remote support has gotten easier and better. DolphinBar and Toshiba Stack connected real Wiimotes work without issues for the most part. Users can quickly and easily connect a Wiimote and play their games just as they would on a Wii. On the other hand, if they idle for more than five minutes or go through an ES_Launch, or the game decides it wants to be one player for menus in a multiplayer game, Wiimotes will get disconnected and the experience becomes more akin to how it used to be.

AdmiralCurtiss has been pretty busy in the emulation scene. Last month, he starred in the progress report after doing tons of work to make sense of why PAL Wii games were having trouble getting into various video modes in Dolphin, and earlier this month he brought the popular Memory Card Folder feature to PCSX2. Despite all of that, he took up the task of getting emulated Wiimotes to automatically reconnect when they were disconnected by the game.

This task turned out to be not nearly as hard as anyone imagined. The quirk was that Dolphin was already accepting input from disconnected Wiimotes, so the only challenge was to detect when the Wiimotes were being disconnected and reconnect them upon seeing button inputs. And since Real Wiimotes are always connected, it applies to them too! It doesn't work exactly as a real Wii; connected Wii Remotes will always be on regardless of what the game displays, but now if a game disconnects a Wiimote, simply hitting any button on the Wiimote will reconnect it. No more messing with hotkeys or desynced Wiimote states. Happy waggling!

4.0-7056 - Fix Floating Point Compare Reordering in PPCAnalyst by phire

Sometimes the actual bug isn't very hard to fix. Once it was determined what was wrong, it was a pretty simple fix; the entire commit boils down to two lines.

if (a_info->type != OPTYPE_INTEGER)  
    return false;

Yet, this was one of the most difficult bugs to track down in recent memory. Why? Let's look at the entire scenario.

Sonic Unleashed, a relatively popular title for the Wii, had started hanging at some point. The game would be running great and then it would suddenly freeze, with the music playing on as if taunting the player that things were still working fine. Bugs like these are among the hardest to catch; the game itself crashed rather than the emulator. It could be anything from a GPU/CPU desync, CPU emulation inaccuracies, save timing bugs or pretty much anything else!

Thankfully, the user that reported this bug was able to narrow down the issue by quite a bit by testing options. They were able to determine that this only happened on JIT; not JITIL or Interpreter. JITIL hasn't seen the same level of optimization or completion as the JIT; so oftentimes it will fallback to interpreter and avoid various JIT bugs, making it an invaluable tool for quickly testing certain JIT bugs. Still, even with this hint, narrowing down the issue was a huge task that was going to take some serious time and dedication.

Sonic Unleashed

By the time the Sonic Unleashed Hack was finally put to rest, the game had picked up a nasty freezing problem. Poor Sonic Unleashed

When it comes to mind-numbing testing, JMC47 is usually called upon to do the dirty work. He, unfortunately, had no idea what he was getting into with this issue. His initial job was to go through and find out what CPU instruction was causing the crash; as that's normally what happens in this case. After several hours of testing the bug at five to 15 minutes per bisect step, he was unable to determine a culprit and returned with his hands empty. Thankfully, phire had another idea as to what would cause something like this.

Like what was said earlier, Dolphin's JIT is more complete than its JITIL; it emulates more instructions and has more optimizations. But it also has a few things beyond that. There are special optimizations implemented through the PPCAnalyst that allow things to be optimized further. Disabling these special optimizations made Sonic Unleashed about 20% slower, but suddenly the crashes stopped. Simply disabling the optimizations for the game would have been an easy fix, but phire wanted to find the actual bug and fix it, especially since it could cause issues in other games. Fortuitously, this would be proven true.

And so, JMC47 was sent on a second journey of bisecting through instructions, this time disabling instructions from using a specific PPCAnalyst optimization called branch merging. After two days of bisecting (and over 14 hours of testing Sonic Unleashed!) he came to the conclusion that the problem was caused by a PPC instruction for floating point addition. After a 90 minute session in-game with zero hangs, the results were confidently brought back to phire where they were immediately refuted. Phire contested that there was no way that this could cause the issue despite the hangs being worked around. In order to debug it further, he added in even more debug logs and had JMC47 do even more bisecting on the issue. This time, they were looking at what instruction sets were being optimized to figure out if any of the combinations could be remotely dangerous.

It was a lot of instructions (scrollable)

While going through the gargantuan list, phire noticed a potential problem at long last. While Floating CoMPare Ordered (fcmpo) and Floating CoMPare Unordered (fcmpu) were blocked from being reordered into the second slot, they could be reordered into the first slot. Because these two instructions can raise interrupts, reordering them at all is not safe and will cause issues. Once this oversight was taken care of, the crashes in Sonic Unleashed ceased without a huge loss in performance. Thanks to phire's hard work in truly fixing the bug, a second game was discovered days later to be affected the same way, Resident Evil: The Darkside Chronicles.

4.0-7072 - Implement scaled partial texture updates by Tinob

Last month saw the addition of Partial Texture Updates, finally allowing coins in New Super Mario Bros. Wii to rotate with Store EFB Copies to Texture Only for the first time.

But for some people, that was simply not enough. Tinob put his graphics expertise to work and quickly implemented additional features for partial texture updates to support higher internal resolutions. Now all of the coins and dynamic objects that would previous render at native resolution and be displayed at whatever insane settings the user desires. 1080p? Absolutely! 2K? Sure. 4K? If you really need to. 5K? You do realize this is a Wii game, right?

New Super Mario Bros. Wii

No matter your internal resolution, coins were always blurry and blocky

Super Mario Bros. Wii

But now they are crystal clear!

4.0-7088 - Respect Stride of EFB copies to XFB by booto

Virtual and Real External FrameBuffer (XFB) are accuracy options within Dolphin that aren't needed for many games. A quick summary is that the XFB is an area of main memory where a field of output is prepared before being sent to the display device. The Embedded FrameBuffer (EFB) is an area of memory local to the GPU where 99% of field rendering occurs. The Video Interface, which is responsible for sending a field to a display device, cannot access the EFB but can access main memory where XFB is stored. Because most games tend to do all of the work within the EFB, Dolphin's setting of "Disable External FrameBuffer" lets a field be copied to the display device directly from EFB. This completely bypasses the extra copy to main memory yet produces an indistinguishable result in most games.

Real XFB is the most accurate way to emulate XFB in Dolphin, but has the downside of being locked to 1x internal resolution output, much like EFB Copies stored to RAM. Still, these accuracy options are around because they emulate a part of the GameCube and Wii, even if most commercial titles don't use them for much. Several games output videos, loading screen animations, or occasionally games like Chicken Little: Ace in Action write everything to the XFB; meaning you see nothing without it!

As such, it's extremely important for these accuracy options to be, well, accurate when they're needed. Unfortunately, RealXFB was breaking in very strange ways in quite a few games while no XFB emulation was bypassing the bug altogether. Animal Crossing, Nintendo Puzzle Collection, and others were suffering from strange issues once XFB emulation was enabled.

Nintendo Puzzle Collection

With No XFB, Nintendo Puzzle Collection looks fine

Nintendo Puzzle Collection

With RealXFB, a portion of the image was stretched off the screen at higher IRs!

Because these games were seemingly fine without XFB emulation, there wasn't really much motivation to fix this bug. In the end, this bug was inadvertently discovered while working on another bug, so let's change gears to a more apparent problem that was discovered.

Sega Genesis/Megadrive Virtual Console games were very hit and miss in Dolphin. It seemed as though certain titles were doing something strange that was causing Dolphin to output garbage with disabled XFB. Turn on Real XFB and Dolphin outright crashes! As this is something that should never, ever happen, booto's love of accuracy finally got the best of him and he looked into the bug.

Sonic and Knuckles

Sonic & Knuckles without XFB

As can clearly be seen above, Sonic & Knuckles was absolutely not playable without XFB, and since the game crashed when XFB was enabled making it completely unplayable. In fact, most Megadrive/Genesis games were similarly affected; making it the least compatible Virtual Console system remaining in Dolphin. As booto looked into the bug, he came to a startling realization. Each frame was actually constructed of TWO EFB to XFB copies, each with their own stride (which Dolphin promptly was ignoring.) The combination of ignoring the stride and only using the second EFB to XFB copy resulted in the image above - a tiny sliver filling the entire screen! Once this was discovered, it was only a matter of time before the missing feature was implemented.

Sonic and Knuckles

It seems fitting that Sonic & Knuckles' graphics require combining things...

Remember that odd behavior mentioned earlier? Yeah, it's unrelated to Sonic and Knuckles, but while implementing this fix, another bug was discovered dormant within Dolphin's XFB code. Lo and behold! Animal Crossing and Nintendo Puzzle Collection now render at the correct aspect ratio when RealXFB is turned on.

As to be expected from booto, he doesn't seem to leave any stone unturned. Looming in the distance are even bigger changes XFB that will push it much closer to perfection.

4.0-7110 - Android TV UI by sigmabeta

Considering the NVIDIA SHIELD TV running Android TV has the most powerful Android internals available, making an UI for it was a top priority. sigmabeta went above and beyond in creating a slick UI with tons of bells and whistles waiting for users. While it is not complete yet, it is already a very fluid and functional interface. Check it out for yourself!

The New Android TV UI on the Nvidia SHIELD Android TV

4.0-7115 - Emulated Wiimote Support and Configuration for Dolphin Android by sigmabeta

Most emulators these days are spoiled with controllers. The NES/SNES-based control scheme has been standard for so long, many emulators can actually preconfigure controllers so you can just plug in any modern controller and it will immediately have a 1:1 reproduction! Dolphin has never had that luxury. Even the simple GameCube controller rearranged the face buttons and has unique clickable triggers. But that is a piece of cake compared to the Wii Remote. With a completely unique layout, motion controls, IR pointer, multiple orientations, and accessory support, the Wii Remote remains one of the most unique controllers of any modern major console. Dolphin's primary solution to this problem is fairly novel: don't emulate it! The main strategy for good Wii Remote emulation is to use a real Wiimote. If you don't have one or a sensor bar, there is an emulated approximation that works in many titles.

But for all its time booting games, Dolphin on Android has never had Wii Remote support. Ever since Android 4.2.1 Wii Remotes will no longer pair with Android, so Real Wiimote support is impossible. Emulated Wiimote is an option, but the complexities of it on mobile devices have left the developers in knots trying to figure out how to implement it.

For example, consider an emulated Wiimote on a phone. Most games that rely on the pointer rely on it being always active, like a mouse. Touch to point works, but the users finger would have to be on the screen the whole time, blocking the game’s graphics. And how exactly would someone touch the screen for the pointer while using onscreen buttons? Motion controls should be easy right, since phones have accelerometers? But the Wiimote is designed for big movements. Do you really want to swing your $600 phone through the air as hard as you can, all while you take the train to work? And how do you look at the screen while flinging it about?

With all of these complexities no one really wanted to tackle the issue, and when even GameCube games were barely playable, there was no reason to. Now that games are near fullspeed on some devices, sigmabeta decided to give Wii remote support a jumpstart and added some of the basic components of Wiimote emulation and a controller configuration GUI. There is no onscreen button support, no extension support (nunchuk or classic controller), no motion controls, or anything fancy yet, just assigning wiimote buttons to a controller. But even that makes many Wii games playable for the first time!

New Super Mario Bros. Wii on the Tegra X1

Without screen recording, the NVIDIA TV actually runs New Super Mario Bros. Wii about fullspeed aside from a few minor drops. With a few more optimizations, it may run fullspeed without issues at all soon!

4.0-7125 - Cached Interpreter CPU Backend by degasus

Most users stay as far away from the interpreter as they possibly can. Interpreters are an extremely slow, platform independent way to emulate a CPU, and most emulators start with one before developing a recompiler (JIT). Dolphin's CPU interpreter, unlike most emulators, is really too slow to be of much use to users. The PowerPC powering the Wii and GameCube is a lot more powerful than the processors emulated by most current emulators meaning that a fast interpreter is just that much harder.

In an effort to make a faster interpreter, degasus used the JIT interface and made it output machine independent code. This new CPU backend caches the list of interpreted commands and executes them afterwards; hence its name of Cached Interpreter. Because it outputs machine independent code, it can run on any computer capable of running Dolphin, including architectures that Dolphin no longer officially supports, like x86-32 and ARMv7. Don't get too excited, though, it's only about two to five times faster than the interpreter. Only on the fastest of existing processors would any games approach playability, and JIT/JITIL will always be much faster. In truth, it's more or less an experimental backend to make bisecting and working around JIT issues a bit easier. That doesn't mean a little fun can't be had, though!

Cached Interpreter Performance Preview

4.0-7133 - Automatically detect GameCube Aspect Ratios by phire

In another one of those "Finally!" features, phire has implemented a heuristic for Dolphin that allows it to detect when GameCube games switch between 4:3 and 16:9 widescreen. Popular games like F-Zero GX, Tony Hawk (series), Star Fox Adventures, Super Mario Strikers, and tons more have built-in support for 16:9 widescreen output, but until now it's required users to manually override Dolphin's aspect ratio in order to use it. Many TVs require that too! With this enhancement, Dolphin will now detect when games switch between 4:3 and 16:9, allowing users to get the most out of their games without needing to touch the graphics configuration whatsoever. A neat quirk is that it also detects widescreen codes, such as the Wind Waker 16:9 code! It will even switch back into 4:3 in the pause menu where it would normally just be stretched, then back to 16:9 when unpausing!

Remember, unlike Wii games where the system settings handle widescreen, GameCube games that support widescreen need to have the widescreen option enabled in the games' options. As such, this change is disabled for Wii titles, despite games like Super Smash Bros. Brawl allowing players to change the aspect ratio ingame. This is to ensure there are no false positives that cause troubles for users unnecessarily.

Switching between aspect ratios has never been easier!

4.0-7138 - Pixel Aspect Ratio Adjustment / VI Scaling Fix by mirrorbender

For all of Dolphin's 12 years of emulation, an assumption has been a central part of how it displays games - it assumed games were either 4:3 or 16:9. This allowed Dolphin to avoid the nightmarish reality of digital to analog conversion. As the emulator continues to march forward, smaller and smaller issues tend to come to the forefront, and users began to notice that things weren't quite right in certain games.

Super Smash Bros. Melee

The green circle is a perfect circle for comparison

Super Smash Bros. Melee

Now it lines up perfectly!

Subtle issues like this were unfortunately fairly common. Thankfully, most users wouldn't notice these small inaccuracies. It would never be a big issue unless some evil game were to render an absolutely insane aspect ratio. This scenario could only come from an extremely competent developer mimicking the look of a movie...

Star Wars: Rogue Squadron II: Rogue Leader

Most aspect ratio issues are hard to notice. This is not one of those!

Star Wars Rogue Squadron II: Rogue Leader

Rendered correctly, Rogue Leader's cutscenes match the 2.15:1 aspect ratio of the films

Thanks to the work of Progress Report newcomer mirrorbender, now Dolphin renders at the same aspect ratio as the console. However there is a slight side-effect. Since Dolphin no longer assumes 4:3 or 16:9 aspect ratios, if a game renders at some other aspect ratio there will be black bars on the side of the screen, possibly even in certain widescreen games. To address this, Dolphin's Crop function (Graphics > Advanced tab > Crop) has been tuned to crop down to 4:3 or 16:9 without losing any important data. This allows users to not only avoid black bars, but still get the benefits of the correct aspect ratio.

Super Smash Bros. Brawl

You wouldn't even notice that there are black bars in fullscreen

Super Smash Bros. Brawl

But if you want to remove them, you can

Because of the extreme complexity of this issue, a full blog post will be coming later to explain all of the technical details.

Last Month's Contributors...

Special thanks to all of the contributors that incremented Dolphin from 4.0-6921 through to 4.0-7141!

Die Diskussion kann im Forenthread des Artikels fortgeführt werden.

Nächster Beitrag

Vorheriger Beitrag

Ähnliche Beiträge