Merry Multiplatform-mas!

It’s been a while since I wrote anything heavily technical, so here comes some words to fix that!  Black Annex is going to launch for Windows, MacOS and Linux simultaneously now, thanks to some large amounts of progress I’ve made lately.

Let’s start by talking a little about how I make Black Annex.

Black Annex is written in BASIC, it’s almost pure QBASIC, but to make it work on >16bit operating systems, it’s translated to C at compile-time, links SDL for its interface work, and then compiles using GCC. This was, originally, handled by an off-the-shelf product called QB64. It worked really well at first, but I started to find bugs in QB64 as I started to push for higher framerates, modern features (Steam Overlay Support) and slightly unusual input methods (relative mouse movement, for one). QB64 really broke down at a few key points, and I knew I would have to fork my own version of it and change a lot of it to make it work. I know almost zero about programming in C, so the scope of doing this was very scary.

This was the reason I decided to focus on Windows first, and then worry about a custom compiler for Black Annex for MacOS and Linux post-release. Programming in C is scary enough for me, but targeting operating systems that I’ve never worked with would also make the job even bigger.

So, it was obvious that I’d need to use a custom fork of QB64 to compile Black Annex, but I wanted to make sure that the off-the-shelf version of QB64 would still compile it. I wan’t to make sure the game would “work” to some degree if people didn’t have access to my compiler, only the game source.

The first thing I wanted to change about the compiler was the way that the PAUSE key, and the ALT+ENTER keys worked. The PAUSE key, in QBASIC, makes the entire program execution just pause. This really messed with my delta-timing in-game, and I wanted to just disable it so I can control game pausing myself. ALT+ENTER toggles full-screen modes, which crashes the game if you do it while the game is loading in sprite sheets. I wanted to just completely disable that and make a menu option for full-screen.

Changing those two options would be my first adventure in the C source of QB64. QBASIC 4.5 for DOS is basically this library file called qb.lib, which contains all the information for translating your BASIC code into DOS interrupts and such. QB64 works the same, with a file called qbx.lib. All the changes I wanted to make live in a file called qbxlib.cpp, which I can modify, recompile QB64, and then recompile Black Annex to use the new libraries.

I spent many hours exploring qbxlib.cpp, it’s a big file and it’s really messy. There’s no tab-spacing or anything, it’s just a massive flat text document. I tried to get Sublime Text to do the “Auto Spacing” on it to make it a bit more readable, but it didn’t have an amazing effect. I navigated about and found some parts where keyboard input is handled and disabled the ALT+ENTER and PAUSE key behaviors. With that done, I left it at that. That was the only thing I really needed to do at that time.

Six months passed, and I found a bug. The first major change I had to make to QB64 was the way mouse movement is handled.

The bug wasn’t in QB64, or Black Annex, the bug is in SDL 1.2 (or, some would argue, Windows). QB64 uses SDL for keyboard and mouse input. SDL has a bug involving relative mouse movement. When using relative mouse movement with SDL, you’ll typically enable a flag called SDL_GRAB, which forces the mouse cursor to stay inside the program window. The bug is as follows: When you enable SDL_GRAB, then begin holding a key on your keyboard, the mouse movements reported by SDL will skip a few messages, resulting in pulses of “lag” in the mouse movement. Release a key and the issue goes away. Hold two keys and the issue returns, release one key (while still holding another) and the issue goes away. If you disable SDL_GRAB, the issue does not appear at all.

The issue appears because SDL_GRAB works by constantly warping the mouse to the center of the window after reporting mouse movements. It does this warp as fast as it possibly can. Windows can only record so many messages about inputs, and while holding a key, moving the mouse, and warping the mouse, information gets lost (or SDL doesn’t read the messages fast enough, I am not sure).

I was completely unable to fix this issue inside my BASIC code, I had to go back to qbxlib.cpp and write some hacks into the keyboard/mouse handlers there. I tried so many things, sending fake KEYUP messages to SDL, sending fake KEYUP messages to the windows even queue, and all sorts of terribly ugly hacks. Disabling SDL_GRAB and just constantly using SDL_Warp to warp the mouse myself. Using the Windows API to warp the mouse. None of these worked. The issue always returned.

In the end, I had to disable SDL_GRAB, and every time the mouse reaches past the inner 25% of the program window, warp it back to the center myself. This much slower warping (as opposed to warping constantly as fast as possible) made the issue go away. This was about one week’s work, between learning C, learning how to handle events in Windows and SDL, and actually coming up with a solution.

With this work done, I remembered the main roadblock I had originally encountered with the MacOS and Linux builds of Black Annex. Every time you moved the mouse, the entire game would just lock-up. Execution would resume as soon as you stopped moving the mouse. In the past week, I’d learned a lot about how mouse movement works in QB64, and I was confident that maybe I could actually make some headway in the Linux and MacOS forks of QB64 I had planned.

But, I had something else I’d rather work on with my new knowledge of C and the qbxlib.cpp file. I wanted to get the Steam overlay working. The Steam overlay only works if you game renders using OpenGL and DirectX, but QB64 just uses software “Surfaces”, so it was out of the question. I had been talking to Simon Roth a little bit about my issues with mouse movement earlier, and I mentioned the Steam Overlay roadblock. He said to me: “You can just render an SDL surface to an OpenGL texture, that’s what I did with VVVVVV”.

I understood the theory of what he was talking about, but I had zero concept of how to actually do what he was saying. The second major change I had to make to QB64 was the way SDL surfaces end up on your PC screen.

Googling “Render SDL surface to OpenGL texture” comes up with lots of results, and I had to spend a lot of time reading about what these people were doing, and how to make it work. It wasn’t too long because I had completely broken QB64, and ended up with it always just outputting a square window with an OpenGL-rendered triangle on it. Black Annex would be running in the background; you should still hear the music, but the rendering was just an OpenGL loop thingie now.

With loads of help from the goons in #sagamedev, I got the SDL surface that normally appears on-screen in QB64 to instead render to an OpenGL texture in a quad on-screen instead. Black Annex launched via Steam, and the Steam Overlay appeared. I had to do some work as far as communicating to OpenGL what resultions to use versus the size of the SDL surface I had initialized, so I hijacked QBASIC’s “OCT$” function (who even uses that, anyway). By using an existing function, I could ensure my code would still compile using an off-the-shelf compiler, but if OCT$ returned some very specific results, Black Annex would know that OpenGL was available and switch to using that instead. I changed the code for OCT$ in qbxlib.cpp so that it would always return a blank string of 22 spaces no matter what you sent to it, and then it would begin a sequence of initializing OpenGL. Making a call to OCT$ would first turn OpenGL on, then you need to send the width of your screen to OCT$, then the height of the screen. It looks funny to see these unexplained calls to OCT$ in the code for Black Annex, but there’s comments explaining what they do.

So I’d ended up with a build of QB64 that’s very different than the off-the-shelf version, and also supports OpenGL scaling, instead of the very slow scaling method I had used previously. Full-screen Black Annex is a lot faster now.

So next I turned to Linux with my new knowledge of how all this stuff works. I figured “I can probably get Linux working”. I installed Ubuntu on a Dell XPS17 I have spare (it’s not mine, I borrowed it for exhibiting at PAX). The XPS17 has an Optimus video card, so it took two instances of “Whoops, I broke everything, format” for me to get into Ubuntu (I don’t know anything about Linux so it was easy for me to break stuff).

I installed the vanilla QB64 onto my Ubuntu computer and compiled Black Annex. As it had always done, it crashed as soon as you moved the mouse. I jumped straight into the qbxlib_lnx.cpp file and got to work on finding what the hell the mouse was doing that made everything crash. It was a straight-up bug. The fellow who made QB64 must have just never tested relative mouse movement on Linux, because it just had a line that was telling program execution to literally pause whenever the mouse moved. I have no idea what the point of it was meant to be, so I just commented it out. Blam, Black Annex worked on Linux. I don’t fully understand Linux, so I’m not 100% sure how to deploy the game (it has these SDL dependencies), but with the game working, I ported all my OpenGL code over to the Linux libraries and I had Steam Overlay working on Black Annex in Ubuntu. The entire Linux effort took a single afternoon. It was an amazing result.

But then there’s Apple.

I have a really bad time trying to use MacOS. I get frustrated and fumble around and can’t find folders and get confused and end up installing Windows 8 on my iMac and breathe a sigh of relief. That is my relationship with MacOS up to this point. I recall the issues that I had with Black Annex on MacOS.

Firstly, there was the mouse issue, the same as Linux, game would crash the moment you moved the mouse. Then there was the ugly-as-hell terminal that would also run alongside the game reporting some kind of errors or something the whole time the game ran. Then there was this stupid .command file you had to run to actually start the game so that the working-directory would change before running the game so that it could find the SDL framework in its subdirectory. All these issues compounded with that fact that I struggle massively with MacOS made this seem like the scariest endeavor.

So I installed vanilla QB64 on MacOS and compiled Black Annex. It was exactly as I described. I quickly fixed the mouse issue, it was the same bug that had been in the Linux build, so the game worked at this point, but this horrible terminal window and the .command file you needed to use to start it was really ugly.

The reason it had this .command file was because all the SDL framework lives in a subfolder next to Black Annex called “common”, but in MacOS, when you run an executable (unlike in Windows), the program considers ./ to be the hard disk root, as opposed to the executable program’s folder. So when Black Annex looks for ./common, it wouldn’t find them. This .command file performs as chdir to the executable’s folder, and then launches the game so it can find all of its frameworks. I Googled the issue.

I found this program called otool, which you can use to find a list of every framework your executable relies on, and where it expects to find it on the hard disk. As expected, Black Annex was spending a lot of time searching for ./common. A little more searching and I found a program called “install_name_tool” which lets you change the URLs that your program expects to find its dependencies at. In fact, you can use “@executable” to direct these changes to areas on the hard disk relative to where the executable is located! So I wrote a little .command file of my own to change all the dependencies on an executable after it’s been complied by QB64, so the .command file wasn’t needed anymore.

Well, this worked all well-and-good for the dependencies, but what about the actual game content? It lives in a folder called “campaign” next to the Black Annex executable, and there’s no way in QBASIC for me to say CHDIR(“@executable”) or anything like that. I had to go back into qbxlib.cpp and perform a chdir in there by discovering the executable URL manually. This was a real fuckaround, with a lot of conflicting information as to whether it was even possible outside of ObjectiveC. Without delving into actual source code, I eventually found someone who had written a few lines of code to do what I needed, and worked it into my own uses.

So Black Annex would work now without a .command file to start it, the last thing was just this ugly terminal window that always insisted on being there while the game ran. It was just reporting “Warnings” constantly from the C source of Black Annex (which I never touch, considering I deal in the BASIC side of it only), so I wasn’t interested in fixing all the warnings hoping the terminal might go away if I did.

Well, I got distracted by something else…

I wanted to make a .app bundle for Black Annex. You know how on MacOS games are usually just these .app folder things that you just double-click on and the game runs? Well, I packaged up Black Annex into one of those and the terminal window I was arguing with just disappeared completely! I guess it’s just some magical thing where .app bundles just assume you’re not interested in debugging your software and hide that stuff.

I had to make one last change to libqbx.cpp’s new chdir code so that it would change directory to one inside the .app bundle and still find everything, but with that done, Black Annex would launch happily!

I sent the .app bundle to a friend who also has a Mac to test it out. He replied: “It says I need to install x11″. What the fuck is that? I have no idea what this means. It’s something to do with making Linux stuff work on MacOS. But Black Annex just uses SDL for everything and that works on Mac. Why is it using this x11 thing?

I went back to otool and checked the dependancies and saw this “x11″ thing in the list, it was loading something called “freetype”. I guess the default font that QBASIC uses is emulated in QB64 using this stuff. I don’t even use that font, I have my own text renderer I built for Black Annex. Sometimes I use “PRINT” while debugging, but I don’t plan on doing any actual QBASIC development on MacOS. So I went through libqbx.cpp, commented out every line that made reference to “TTF” font stuff, and remove every reference to SDL_TTF in the compilation process, and I was done.

I haven’t ported the OpenGL stuff to MacOS yet, but I now have Black Annex deployable on Windows, Mac and Linux. I still need to re-visit the Linux pipeline to make sure I’m not requiring the user to jump through unrealistic hoops in regards to dependencies, but the major work is done.

Black Annex will launch on all three platforms simultaneously.