Bug Hunting and Refactoring

I’ve had some time to blast through some Woopsi coding over the last week, so there’s quite a bit to write up here. There’s a mix of new features, bugfixes, refactoring and breaking changes.

First up, attempting to flip a screen from one display to the other, or re-organise the depths of screens in the screen stack, is no longer allowed if there is only one screen in existence. Allowing this to occur was a bug introduced in v0.45 when I added a couple of background screens to the Woopsi gadget by default, to ensure the background was always grey and redrawn correctly.

The font class no longer includes a colour member, nor does it distinguish between full-colour and monochrome fonts. It makes no sense for a font to have a state. When trying to draw text, I want to draw text in colour X using font Y. I don’t want to get font Y, make it colour X, then draw with it. Implementing this has involved making changes throughout Woopsi. Text rendering commands now expect to be given a colour to render with. Related to this, the GadgetStyle class now includes a text colour property, whilst the Gadget class has new getTextColour() and setTextColour() methods.

In addition to removing the colour member from the FontBase class, I’ve removed all other members. The FontBase is now a legitimate, data-free interface. All members that are still required, such as height, have been moved into other classes. This will reduce the amount of redundant data inherited by subclasses, such as Lakedaemon’s libfreetype wrappers.

The font change was so successful that I replicated it with bitmaps. BitmapBase no longer includes any data either. Members have been moved into subclasses.

The overload of Gadget::checkCollision() that checks for collisions with other gadgets will not detect collisions with hidden gadgets. I’ve modified it so that it expects to be passed a const gadget pointer instead of a plain gadget pointer. I’ve also added a shortcircuit to the function so that it detects attempts to check for collisions between a gadget and itself and exits early.

I’ve rewritten the Gadget::swapDepth() method. This is used by screens and windows when their depth gadgets are clicked. If swapDepth() is called on a gadget that is not at the front of the subset of sibling gadgets that it collides with, the gadget will be moved to the front of that subset. If the gadget is at the front, it will be sent to the back of the subset.

Previously, the way the method worked was confusing. Most of the logic for the method was actually contained by the parent gadget in its swapGadgetDepth() method. The swapDepth() method just called the parent method to perform the swap. This had some downsides:

  • Children couldn’t determine how they would swap; it was decided by parents, so it was impossible for a new gadget subclass to swap in a different way to (for example) a window.
  • The Gadget, Screen and Woopsi classes all had their own implementations of the swapGadgetDepth() method that were subtly different but basically achieved the same effect.

I’ve renamed swapGadgetDepth() to changeGadgetDepth() and made it considerably more generic - it will now just move a gadget from one index in the child array to another, ensuring the gadget gets erased and redrawn correctly. The swapGadget() method in the child gadget determines which index the child will swap to, so each gadget can decide how it will depth swap. I’ve also added Gadget::getHighestCollidingGadgetIndex() and Gadget::getLowestCollidingGadgetIndex() to help determine these indices. The swapGadgetDepth() overrides in the Screen and Woopsi classes no longer exist.

Gadget::_decorationCount is an s32 instead of a u8. I don’t know why I’d got that set as a u8, especially when the getters/setters were working with s32s.

The RectCache::markRectDamaged() method had a nasty, obscure bug that could lead to the method getting into an infinite loop. It has a couple of nested loops and it seems I’d got the iterator variables confused at some point, and was using the variable from the first loop to index into the iterated-over array of the second loop. Ooops. Thanks to carpfish for spotting the crash and putting together a test so that I could track down the cause.

The todo list that I put together long before Woopsi v1.0 was released included a question: “Is there a bug in the floodfill?” I noticed that there seemed to be a couple of pixels that weren’t filled in the test I wrote for the WoopsiGfx library. Well, there was a bug in the floodfill - it wasn’t filling upwards correctly due to some utterly bizarre typos in the method. Don’t quite know what I thought I was doing when I wrote that function. It’s fixed now.

Related to the floodfill, the stack functions in the Graphics class that it relies on now expect to be passed a reference to a stack object to work with rather than a pointer.

For some reason, three members of the Woopsi class (the vertical blank count that stores the number of frames elapsed since the app started running, the deleted gadget list and another one that I can’t presently remember) were static. This made no sense, so they are no longer static.

I’ve tested empty ListBox gadgets for the first time and encountered a couple of problems. First of all, the scrollbar went crazy and suggested that there were hundreds of options to choose from; that’s fixed. Clicking the empty space within the ListBox caused it to try to locate a non-existent option in its option list and dereference a null pointer. That’s fixed too.

I’ve stripped out some redundant code. The GadgetFlags enum included a “GADGET_NO_RAISE_EVENTS” flag that wasn’t used anywhere; it’s now been removed. I’ve also removed the Woopsi::goModal() method that only existed because the Woopsi::handleClick() method was badly designed. Redesigning that let me get rid of the goModal() override.

Gadgets included a concept of “close type” which existed solely to allow the close gadgets on windows to close, hide or shelve the window as appropriate. It wasn’t useful in any other situation and wasn’t really useful for windows either, so it’s gone too.

Another pair of useless features were the AmigaScreenFlags and AmigaWindowFlags enums. They were used to send gadget-specific flags to the AmigaScreen and AmigaWindow constructors. However, as there were only two flags in each enum, and as typing the enum value names took longer than just passing true or false a couple of times, I’ve stripped them out and replaced them with booleans instead. This change will break user code, but fixing it just a matter of changing a couple of values when creating AmigaScreen and AmigaWindow objects.


Woopsi 0.99.2 Released

Woopsi 0.99.2 is now out. Grab it from SourceForge, as usual:


The font system has had some minor changes made. I finally got around to ripping out the Font and MonoFont classes, as the PackedFont classes perform the same job with far more efficiency. They’ve been moved to the “extras” directory in the SVN repository in case anyone should ever need a reference for creating new font classes.

All of the monochrome fonts that ship as part of the library are now PackedFont1 classes instead of PackedFont16. This highlighted a bug in the bmp2font .NET program that is included in Woopsi’s “tools” directory - it was allocating relatively huge bitmaps for the font data instead of the tiny bitmaps that were really necessary. I’ve deleted the “CourierMono” font as it was an exact duplicate of the “Courier” font. The GlyphFont is a PackedFont16-based class instead of a Font-based class.

This changes have reduced Woopsi’s footprint quite significantly:

  • 1.5MB from the size of the Woopsi library;
  • 2MB from the sourcecode;
  • 400K from the packedfont example ROM;
  • 60K from the helloworld example ROM.

A big saving for very little work.

Other changes include replacing the x, y, width and height members of the Gadget class with an instance of the Rect class (itself converted from a struct). A lot of the clipping work can now done be within the rect instance rather than throughout the codebase.

The FileListBox and FileRequester classes include a getPath() method that allows a developer to determine which directory the file requester is pointing at.

Other miscellaneous changes are detailed in the changelog.


FreeType Support

Woopsi now has FreeType support. That means it will load TrueType, OpenType, FNT and nearly a dozen other types of font.

Many thanks to Lakedaemon for his hard work implementing this. All I had to do was tidy up the code a bit, split the classes into three separate cpp/h file combos and make a couple of minor alterations to the build files.

Lakedaemon, if you’re interested in getting hold of the latest version, pull down the Woopsi branches folder from SVN and take a look at the “woopsi_with_freetype” branch. To compile with FreeType support:

  • Deploy libwoopsi as usual to your devkitPro folder.
  • Copy “libfreetype” to your devkitPro folder from the “libs” directory within the “libwoopsi” folder.

Here’s your FreeType demo altered to use the new structure. It’s identical except for two more includes.

I need to do some testing and see if I can get your suggested monochrome version working. I’d also like to try and use the pre-ported version of FreeType hosted by the devkitPro guys instead of a custom build, but their instructions for how to use the port libraries don’t seem to be terribly accurate. Once I’m happy with it I’ll merge the branch into the trunk and we’ll have another release to play with.

I’m not sure how this is going to work with the SDL version yet.


A Bugfix Bonanza

If you have been following the comments for the previous few posts, or the forum, you’ll have seen work progressing at a frantic pace. Most of the thanks for this goes to Lakedaemon, who has been zapping bugs throughout Woopsi’s text rendering system. Working so quickly and not regularly blogging about it means that I’ve probably got a lot of changes to list here. Let’s see if I can collate them into some sort of order…

The WoopsiString class has seen a lot of changes. I’ve added indexOf(), lastIndexOf() and subString() methods. The first will find the first index of a particular character in the string and return its index. The second does the same thing, but in reverse - it finds the last index of a character. The last will return a pointer to a WoopsiString containing a subsection of the original string.

WoopsiStrings are no longer null-terminated. They already included a member variable that stores the string length, so appending a terminator was a waste of a byte.

The Text class’ wrap() function has had some long-standing bugs fixes. The strangest was probably an invalid comparison between a line index and a char index that was supposed to prevent the function from wrapping the entire text, but actually just screwed up the function most of the time, and made it slower the rest of the time. Not sure how that stayed in there for so long.

The next most significant bugfix was to the GraphicsPort’s clipScroll() method. This is used primarily in the ScrollingPanel gadget. It performs two functions. If it is possible to copy any of the existing visible region to the new location, the function copies it. This prevents redrawing, which is slow, if the required visual data is already available. Secondly, it remembers any rectangular regions that require redrawing and passes the list back to its caller. The ScrollingPanel uses the list to redraw any non-copied regions of itself.

The clipScroll() method worked perfectly if scrolling in just one plane, but as soon as horizontal and vertical movement was attempted simultaneously, it fell apart. Rectangular regions were missed out or copied to the wrong locations, particularly in the corners.

The function was originally tackling the problem backwards. It calculated the size of the source and destination rectangles then clipped those rectangles. However, this introduces two problems:

  • The rectangles could become different sizes;
  • The rectangles could acquire different offsets from their original locations.

Neither problem is too difficult to solve, but it does mean that if either problem arises, the function will not be able to copy the regions and must therefore fall back on the much slower redraw option.

The “correct” approach to the problem is to clip before calculating the sizes of the source and destination rectangles. That will ensure that neither of the two highlighted issues arises, which means that the copy can be used in preference to redrawing much more frequently.

I’ve finally got around to removing the glyphs from the Topaz and NewTopaz fonts. As the glyphs are now an entirely separate font, keeping that data in the bitmaps was a waste of space. Neither font requires colour now that the glyphs are gone, to I’ve converted Topaz from a Font to a MonoFont, and converted NewTopaz from a PackedFont16 to a PackedFont1. This has made both fonts considerably smaller.

Related to this, PackedFontBase::isCharBlank() returns the correct value (false) if the requested character is outside the range of the available bitmap data.

Dragging a background screen below the level of the topmost screen no longer crashes the system. The Screen class was attempting to copy a region with a negative height, which the DMA hardware really doesn’t like.

The FileRequester has had yet more fixes since the last post. When running in SDL mode, it now shows a dummy list of files and directories. I had intended to get it to show a real list of directories, but I ran into The Windows Problem - every major operating system but Windows has POSIX-compliant file system access. Unfortunately, of those major operating systems Windows is the most prevalent. I really couldn’t be bothered to write the POSIX code if most Woopsi developers are using Windows, and I couldn’t be bothered to write Windows code when I only run the SDL version in OSX. The compromise, which will work on all platforms, POSIX or not, was just to create the dummy list.

Finally, both the MultiLineTextBox and the standard TextBox respond to d-pad repeats. Holding down left or right on the d-pad when either of these two gadgets has focus and a visible cursor will cause the cursor to move repeatedly until the d-pad is released.


Unicode Complete

Woopsi is now 100% unicode. Except for anything in the “bonus” folder, which I’ll get around to at some point.

I’ve tidied up the WoopsiString, removing redundant methods and sorting out which should be public and which shouldn’t. To cater for iteration, rather than trying to redesign the string, I’ve added a new StringIterator class. The WoopsiString can produce instances of these (in the same way that Gadgets produce instances of the GraphicsPort class) that can iterate over the string. It has some cunning optimisations so that it will always choose the optimal way of moving to a new character index.

On top of that, I’ve moved the FileRequester and associated classes into the main codebase. Lakedaemon pointed out that, unless you use any libfat features, it isn’t included in the final ROM. A quick test revealed that this was indeed the case, so there’s really no reason not to enable libfat in the makefile by default and include the FileRequester in the main library.

I’ve fixed a few problems in the FileRequester. Its background was transparent in places, which was caused by one of the gadgets in its children not having a draw() method. The sorting was broken - I thought I’d fixed this - so although the items were sorted alphabetically, directories were intermingled with files. Lastly, it doesn’t try to draw before the rest of Woopsi has been drawn, a problem which caused some graphical corruption.

The Bmp2Font .NET utility for converting BMP files to Woopsi fonts now creates working Font subclasses. I’ve removed four fonts from Woopsi as they didn’t seem to be converted properly by Jeff’s original process - they either had glyphs with missing sections or the vertical positioning of some of the glyphs was off.


Unicode News

Unicode support in Woopsi is coming along nicely. Most of the gadgets now expect to receive WoopsiString objects instead of chars or char arrays when dealing with text. As the WoopsiString now works with UTF-8, that means most of the gadgets now also support UTF-8.

I’ve overloaded the WoopsiString’s “=” operator for WoopsiString objects, char arrays and u32s, which means that any parameter that should be a WoopsiString can accept char arrays or u32s instead. Converting from string literals to WoopsiStrings is therefore seamless and automatic.

The glyphs are now stored in a separate “GlyphFont” font. The default glyph font is stored in the defaultGadgetStyle global variable, in the same way that the default system font is stored. Converting this unearthed a couple of bugs in the Bmp2Font program - it doesn’t write out the resultant font constuctor properly (it misses out the transparent colour and uses the raw bitmap data instead of the bitmap wrapper object). I still need to fix those.

The most problematic changes were convincing buttons that displayed glyphs to centre their text properly (turns out that they were centring based on the wrong font) and getting the MultiLineTextBox to redraw at a reasonable speed. After some cunning hacking about with the WoopsiString and the MultiLineTextBox I’ve got it back up to something approaching its old speed.

Regarding speed, unicode definitely has a negative impact. Having to scan through an entire string to locate a codepoint is a horrible performance killer. WoopsiString::getCharAt() is definitely not a function you should use if you’re reading out sequential characters, as it looks for the codepoint from the start of the string each time it is called. It’s much more advisable to use getToken() to get a pointer to the first token, read out the codepoint, then increase the pointer by the size of each character as it is read. Before I went back to optimise things, the MultiLineTextBox was using getCharAt() when drawing and so was the PackedFontBase, meaning the string was being scanned through twice for each character being printed. Xcode’s “Shark” profiling utility noted that the MultiLineTextBox was using 31% of the total CPU time when printing a line of text solely because of all the UTF-8 parsing. Yikes.

There are still a lot of gadgets to convert over, and probably a hefty set of bugfixes and optimisations to make.


UTF-8 and TrueType Fonts

If you haven’t been following this thread lately, you won’t be aware of Woopsi’s latest contributor, Lakedaemon. He’s doing a fantastic job of introducing two new, much sought-after features to Woopsi - unicode (UTF-8) and TrueType font support.

Both features seem to work well. Both of these features have pros and cons, which I’ve been contemplating a lot lately. Unicode support, for example, has a very negative impact upon random access within strings. Unicode is a variable width encoding system, so there is no way to locate a random character within a string without first parsing all of the bytes that come before the character. However, Woopsi doesn’t really do much random string access, so this is not a huge problem.

The real issue with supporting extra character sets is the amount of memory consumed by each string. As a variable width system, UTF-8 uses single bytes for the standard ASCII set and so the problem is a non-issue.

In order to have system-wide support for UTF-8, all gadgets that have previously worked with char arrays must be altered to work with the WoopsiString instead. This implies some overhead, which is unavoidable, but as support for non-ASCII character sets is one of the most requested features for Woopsi, I think we can live with it.

Supporting TrueType fonts is more complex. Lakedaemon has used the FreeType library. It looks like a solid project and is dual licenced under the GPL and MIT licences. The MIT licence is basically the BSD licence with a different name, so there are no legal problems with using it. However, it does make the build process more complicated, and it adds substantial bulk to the size of a Woopsi ROM file. The TrueType font must be loaded from the flash disk, so the ROM must be linked with libfat, further adding to the ROM size.

I haven’t tried out a Woopsi GUI that uses TrueType fonts yet, but I’m guessing that it will be slower than a simple bitmap font-based GUI. FreeType is designed for embedded devices, however, which may mean that the difference in performance isn’t noticeable.

Bearing these points in mind, I think I’ll add TrueType support as an optional extra that can be enabled in the makefile. I’m not sure how that’s going to work yet; it may be that I split Woopsi into two pre-compiled libraries, one of which is lightweight (no libfat/freetype) and one of which is heavyweight (libfat and freetype enabled).

Someone using a TrueType font as their system font will soon come across an interesting problem - the glyphs on their buttons (close gadgets, screen flip gadgets, radio buttons, etc) will be drawn as characters, not images. The reason for this is that the glyphs are stored in the extended ASCII portion of the bitmap fonts. At this point, it has clearly become a bad idea, and I’ll need to split the glyphs into a separate glyph font. It is unlikely that anyone will want two or more glyph fonts active at once, so I’ll make the glyph font a global variable initialised in the woopsifuncs files. It’ll be user-definable, just like the system font.

Aside from that, I’ve made a couple of other fixes. I’ve removed the “raisesEvents” flag from the Gadget class, as that is now handled by the GadgetEventHandlerList class. The setRaisesEvents() and raisesEvents() functions interact with the GadgetEventHandlerList instance correctly, which has fixed a glitch in the ScrollingPanel. The ScrollingPanel::raiseScrollEvent() method refers to the GadgetEventHandler’s isEnabled() function to see if events should be enabled or not.

Finally, the FontBase class now includes a getCharHeight() method, which should hopefully help out Lakedaemon’s desire to support variable-height rows of text.


Woopsi 0.43 Released

Another release escapes from the Simian Zombie factory.

In addition to the changes discussed recently, which mainly concern new fonts, new .NET utilities for working with fonts, and many bugfixes, this release moves event handler management out of the Gadget class and into a new GadgetEventHandlerList class. This is now responsible for maintaining the list of event handlers added to a gadget and raising any events to those handlers. This is an under- the-hood change that won’t affect the majority of user code. If you have created custom gadgets, you may find that you now need to call _gadgetEventHandlers->raiseXXXEvent() instead of just raiseXXXEvent().

Download it from SourceForge as usual:

Additionally, I have zipped up some pre-built example ROMs. Download them below.


New Font Utilities

I’ve extended the font creation utility I discussed a few posts ago so that it now creates Font subclasses in addition to the other three. The bmp2font utility is now capable of producing any of the official Woopsi font classes.

This is great, except actually creating one of the bitmaps is a real pain. I have no intention of mucking about with glyph maps unless I really have to, so I set about creating a utility that would convert a Windows font to a bitmap.

Doing so was slightly tricky. Requesting a particular font size in .NET doesn’t guarantee you a font of that size. Due to the spacing around fonts (which .NET doesn’t provide a way of changing) and the way it calculates font sizes, what you get if you use the built-in APIs is a bitmap with a lot of empty space and not a lot of font. This is clearly no use.

Instead, I came up with a solution in which I render each glyph to its own bitmap, calculate the exact size of each glyph and the overall size of the font, then blit all of the bitmaps to a single master bitmap. Each of the glyphs is left-aligned and no space is wasted. This new “font2bmp” utility can turn any Windows font into BMP file with the minimum of hassle and dimensions.

At this point, I realised that I had a utility to convert from a Windows font to a bitmap, and from a bitmap to a Woopsi font, and that it would probably be a good idea to have a utility that could cut out the middleman and go straight from one font format to the other. Refactoring the two projects produced two DLLs that could be used to achieve this.

A short time later and I’ve got the third utility in the font trilogy, “font2font”. This will convert straight from a Windows font to a Woopsi font.

All three utilities are available in the Subversion repository, and will be in the next release. One minor bugfix - Lakedaemon spotted Woopsi allocating heap memory for no real reason when it converted single chars to strings when setting string values. This was rather wasteful. Woopsi now uses the stack instead.


Font Dilemma

Jeff’s PackedFont classes can do everything that my Font and MonoFont classes can do. They store their data more efficiently and require far less processing time to render their glyphs than my fonts. Why not just remove my fonts and simplify things?

I think I’ll do just that unless I can think of a good reason not to.