2013-12-24

Putty Squad

Putty Squad for the Amiga finally got released. Who’d have guessed? I loved the demo version that Amiga Power gave away as a coverdisk, and was hugely disappointed when the full version never appeared. The SNES version wasn’t a great substitute; it didn’t feel right.

I’ve got it running in FS-UAE. Download it from the System 3 website:

Putty Squad

2013-09-07

Auto-Disable Gobble Comments

Here’s a neat new feature added to Gobble a couple of weeks ago: comments can be automatically disabled when a post reaches a certain age. I’ve enabled it in this blog and spam has dropped from a few dozen comments a day to zero.

I need to tidy it up a little. At present, it looks like comments are still enabled until you click through to the post itself. I couldn’t get my initial plan working, probably because I’m pushing the templating system too far. Also, I switched to a singleton for the config settings, which might not have been the best way to implement it.

2013-08-21

Gobble Moves to GitHub

Gobble is now stored in a Git repository on GitHib:

Other recent changes to Gobble include the addition of support for per-theme favicon.ico and robots.txt files.

It’s been just over a month since I decided to gradually switch from Mercurial to Git. Thus far the change has been surprisingly enjoyable. Git’s extra flexibility turned out to be a good thing. Somewhat disturbingly, I’ve already forgotten how to perform basic operations with Mercurial. Travis CI is next on my list of new Git-powered toys to investigate.

In other Gobble-related news, I’ve removed Google Analytics from this site. GoAccess is a neat web server log analyser that does everything I want, and it keeps Google’s grubby little paws off my website.

2013-07-10

Git

When I first made the switch from Subversion to Mercurial, I tried out most of the available DVCS options: Git, Mercurial, Fossil, Monotone, Darcs and Bazaar. I wanted something that was easy to use, powerful and cross-platform. Only Mercurial and Fossil met the criteria. Fossil’s weird file system handling and lack of built-in Creole or Markdown support in its wiki eventually killed off my enthusiasm for its unique feature set, leaving Mercurial.

Mercurial is possibly the greatest development tool ever created. It completely altered the way I write code and made me immeasurably more productive. Its command line UI is a work of art. Every command is obvious and works exactly as you’d expect. Any chance I get I promote it, especially to the folks who still think that TFS is a really neat idea.

When I was evaluating DVCSes, Git had two main failings:

  • Windows support was second-rate;
  • The UI was designed by Linux programmers and made absolutely no sense.

Though the official Windows client is still an afterthought, Windows support is getting better. Microsoft themselves are adding Git support to TFS and Visual Studio; considering their attitude to Linus Torvald’s other work, this is deeply ironic.

The UI hasn’t improved in the last few years. It still has counter-intuitive commands that are overloaded to perform multiple actions that have little relation to each other. Users are still required to be experts in Git’s data structures in order to use the tool effectively.

Despite these failings, Git has become the de-facto DVCS. It is vastly more popular than any of the others I looked at, including the dreamlike Mercurial.

Microsoft aren’t the only company to have integrated Git into their tools. Xcode 4 uses Git as its default source control system, and Xcode 5 looks to be basing a number of very enticing new features around it. Though I don’t want to, it’s time to add Git to my toolbox.

Here’s my initial Git session:

git init test
vim test.md
git add .
git commit -m "Initial commit."
vim test.md
git commit -m "Edited text."

Working well so far. Now let’s try branching.

git checkout HEAD~1

Hmm. You are in ‘detached HEAD’ state.

vim test.md
git add .
git commit -m "Branch edit."
git checkout master

Warning: you are leaving 1 commit behind, not connected to any of your branches. That’s unexpected. I have no idea what it means. Mercurial allows multiple heads on the same branch; doesn’t Git?

git log --oneline

e48b1bf Edited text. 6739f7f Initial commit.

Two commits. I made three. Yikes.

At this point I decided that the only way I was going to learn to use Git properly was if I forgot everything I knew about Mercurial and approach Git as an entirely new tool. I’m up to chapter 3.6 of the Git book and it’s making more sense. The command aliasing function goes some way to alleviating some of the pain of the insane UI, it has some exceptionally cool features, and suddenly I find that I’m looking forward to using it.

As a test run, I’ve migrated the Super Foul Egg iOS source code to GitHub:

2013-06-21

Super Foul Egg for iPad Released

Finally, after months of procrastination:

This one got straight into the App Store.

Obvious changes from the OSX version are:

  • Two player mode removed (still got a bug in the garbage transmission code);
  • Controls designed for the iPad touch screen;
  • Controls displayed via icons on the menu screens;
  • Pause and exit buttons whist in-game.

Less obvious changes are:

  • Multithreaded AI;
  • All code refactored.

I’ll open up the source code soon.

Give the game a try and let me know what you think!

EDIT:

Source code can be found here:

Apologies for its current scrappiness; adding networking support has meant changes all over the place. I remember reading an article years ago by a developer who suggested that networking should be built into a game from the start rather than added later as it’s a huge pain. I definitely agree with him.

2013-06-14

OSX Diff Tools

I’ve been happily using SourceGear’s DiffMerge as a diff tool for years. It’s a great cross-platform tool with an ugly UI. Every now and then I check out the competition to see if there’s anything better, but I haven’t found anything that includes a similar feature set at a comparable price point. It’s hard to beat free.

Today all of that changed. DiffMerge 4 is nagware and costs $39. This isn’t necessarily a bad thing - it’s only fair that SourceGear be compensated for their work - but it means that the app loses its main advantage over the competition and gives me an excuse to shop around for a replacement.

There aren’t that many diff tools available for the Mac. Many of them can be culled immediately from the set of contenders for making basic mistakes.

Mistake #1 is using cross-platform, un-Maclike UIs. Violators are DiffMerge, Meld, P4Merge and KDiff3. Meld is the worst offender as it requires the installation of XQuartz, so I couldn’t even try that one out.

Mistake #2 is not supporting 3-way merges, which makes the tools useless with Mercurial and Git. Apple’s FileMerge is the only tool I looked at that doesn’t support this essential functionality.

Mistake #3 is a lack of new releases. If I’m going to have to invest money and time in a new diff tool I want something that is still under active development. Changes hasn’t seen any significant updates since 2009, despite being taken over by a new company. DiffFork hasn’t been updated since 2011.

Mistake #4 is stupid pricing. Most of the tools are priced at $39 or cheaper. Araxis Merge is three times that price for the cut-down version and six times for the full version.

Mistake #5, which I’m editing in at a later date, is Java. I don’t like Java for reasons far too numerous to go into here. Out goes DeltaWalker

By default, I’m left with just one choice: Kaleidoscope. I’ve mentioned it before and complained that it couldn’t “compete on price, features and speed” with DiffMerge. At $69 it’s more expensive than DiffMerge, but the difference is now between $39 and $69, rather than “free” and “not free”. Kaleidoscope offers a vastly superior UI, and having tried out many of the other options, I realise that it’s actually one of the faster diff tools out there.

It has a number of features that I immediately appreciated:

  • It can jump directly to each change in a file.
  • It shows the list of changed files in a pane of the diff window, rather than a separate window.
  • It shows new files in addition to changed files.
  • Each change is highlighted in a different colour.
  • It has a dark theme.
  • Double-clicking on a folder or file opens it in a new tab.

I’m a little annoyed that I missed out on version 2’s introductory discount price, but I downloaded it from the AppStore anyway.

So long, DiffMerge.

2013-05-31

Patching NSManagedObjects

Previous posts discussed how to create a diff of an NSManagedObject pair. Now we’ll look at how to apply that diff as a patch.

Creating a diff gives us a data structure that looks like this:

(
    {
        attributes: {
            new: {
                city: "Sometown"
            },
            old: {
                city: "Townsville"
            }
        },
        entityName = "Address",
        guid: "123"
    }
)

The diff tells us that the Address entity with the GUID “123” changed its city attribute from “Townsville” to “Sometown”.

The patcher has to be able to do a number of things with a patch like this:

  • It must update the CoreData entity’s attributes to match the new values for each change in the patch.
  • It must be able to traverse both the nested patch structure and the CoreData graph and apply patches to sub-entities.
  • It must be able to apply patch arrays to sets of sub-entities (one-to-many relationships).
  • It must be able to delete entities from the CoreData graph if the patch indicates a entity was deleted.
  • It must be able to create new entities within the CoreData graph if the patch indicates a new entity was created.

The last requirement is tricky. Creating an entity is rarely a case of just calling insertNewObjectForEntityForName: inManagedObjectContext:. Typically, each entity will have associated sub-entities that should be created at the same time, so the patcher will inevitably use the delegate pattern to call back to an external object that can create new entity instances.

It would probably be handy if the patcher could allow manual patching for certain entity types, so that the developer could override its default behaviour for complex or unusual object types. For example, we might receive a patch for a Distance entity that includes two attributes: value and units. If we respect the semantics of the type we can’t update either attribute in isolation. We would expect the patcher to only overwrite one attribute if only one attribute has changed, but we should really overwrite both values to maintain the integrity of the data. (In contrast to the earlier posts about diffing, the current version of the differ in SZManagedObject emits the entire set of old attributes if it detects a change, precisely to cater for this situation.) Allowing for the ability to manually patch objects and override the patcher’s default behaviour lets us do this.

Something else I’m going to bake in is the ability to manually convert types for date and binary attributes. I want to be able to send diffs over the wire as JSON, but JSON doesn’t like date and binary data. They typically get sent as ISO strings or JavaScript date object constructors (for dates) and base64-encoded strings (for binary). A patch will therefore not necessarily have the correct data types for these two attribute types. Whenever they are encountered in the entity I’ll check the type of data in the patch, and if they disagree I’ll call out to a delegate to perform a conversion process.

Patching an entity requires two steps: attributes then relationships. Patching attributes is easy:

  • If there are old values but no new values, the entity has been deleted.
  • Loop through keys in the new dictionary and apply the values to the entity.

Patching relationships is really just a matter of creating new entities if necessary and using recursion to patch the entire graph:

  • If the relationship is one-to-many, patch the relationship as a set.
  • If not:
    • If no sub-entity with the required identifier can be found, call out to the delegate to create it.
    • Recursively patch the entity’s attributes and relationships.

Eventually we’ll hit a point where there are no more sub-patches, which means we have finished patching.

Patching one-to-many relationships is a little more complex, but it isn’t too hard:

  • Loop through sub-patches.
  • If the matching sub-entity does not exist and there is no old data in the patch, call out to the delegate to create it.
  • If there are no new values in the patch, delete the sub-entity.
  • If there are new values, old values and an object to update, recursively update it.

We now have a fully patched object graph.

Again, check out the SZManagedObject project on BitBucket for the sourcecode.

2013-05-30

Gobble Updates

Gobble (and therefore Simian Zombie) has received a few updates:

  • The RSS feed now works and implements the Atom standard.
  • The current search term is displayed as a placeholder in the “search” box.
  • Comments can be disabled on a per-post basis.

The post workflow could be made simpler by adding an admin section, but Gobble is holding up well so far. It compiled first time with the new Go 1.1 compiler, which was nice.

2013-05-29

SZManagedObject

As promised, here’s the BitBucket repository that contains the full NSManagedObject diff/patch functionality discussed in the last post:

The repository doesn’t include a demo yet, but the comments are extensive and the APIs are simple. I’ll revisit this soon with a post that discusses the patching functionality.

2013-05-28

Diffing NSManagedObjects Part 2

When I last discussed comparing two NSManagedObjects and producing a diff, I said that I’d come back to a few issues later. It turned out to be much later than expected; this post has been floating around on my hard disk for 6 months.

This time around I’ll deal with two of those issues:

  • What do you do if you want to include a related object in the diff if it has its deletion rule set to nullify?
  • What do you do if you want to exclude an attribute or relationship from the diff?

The answers are “whitelist” and “blacklist”.

In the previous post, we used deletion rules to signify object ownership. This works well, but means that we can’t automatically include crucial information from a relationship if the deletion rule indicates a lack of ownership.

For example, imagine we’re a dogfood company and have a Dog class and a Person class:

  • Person (User info: @{ @“id”: @“guid” })

    • Attributes:
      • guid
      • name
      • age
    • Relationships:
      • dog (Dog) (Delete: cascade) (To one)
  • Dog (User info: @{ @“id”: @“guid” })

    • Attributes:
      • guid
      • name
    • Relationships:
      • owner (Person) (Delete: nullify) (To one)

If we delete a person we’ll also delete his dog. We don’t care about the dog if we no longer track the person, presumably because we can’t sell anything directly to a dog. On the other hand, if we delete the dog we don’t necessarily want to delete the person. The person could get a new dog.

If a dog changes owner, it’d be handy if we could represent that information in a diff. Unfortunately, the deletion rule will prevent the automated diff methods from examining this relationship. We need to whitelist this relationship to ensure that it is included in the diff.

The whitelist looks like this:

NSDictionary *whitelistedRelationships = @{
@"Dog": @[ @"owner" ]
};

Each key in the dictionary represents the name of a class. Each value represents an array of properties of that class. In this case, the whitelist says, “If the diff creator encounters an instance of the Dog class, include the owner relationship in the diff regardless of the deletion rule”.

There are two issues here. Firstly, we’re hard-coding information that should really be metadata in the data model. Unfortunately, there’s not much we can do about that if we want to be able to produce a variety of different diffs that can be started in multiple places within the object graph.

The second issue is that we’ve re-introduced a cycle into the graph. The diff creator will repeatedly walk from the Dog to its owner and back, eventually causing a stack overflow. To get around that, we need to introduce the concept of a blacklist. Anything in a blacklist will not be examined. Here’s what the blacklist would look like in this situation:

NSDictionary *blacklistedProperties = @{
@"Owner": @[ @"dog" ]
};

The dictionary format is that same as the blacklist: each key is the name of a class, whilst each value is an array of attribute and relationship names that should be ignored by the diff creator. This particular blacklist says, “Do not traverse the dog relationship of any Owner instances”. This would be counter-productive if we were starting the diff from the owner, due to the direction of the relationship established by the deletion rule, but if we start from the dog it is very useful.

Now that we’ve decided upon workable solutions, we just need to implement them. We’ll start off with a method that can retrieve all of the property names for an NSManagedObject instance that exist in a blacklist or whitelist:

+ (NSArray *)propertyNamesForManagedObject:(NSManagedObject *)managedObject inList:(NSDictionary *)list {

    NSMutableArray *output = [NSMutableArray array];

    for (NSString *className in [list allKeys]) {
        Class class = NSClassFromString(className);

        if ([managedObject isKindOfClass:class]) {
            [output addObjectsFromArray:list[className]];
        }
    }

    return output;
}

We’ve introduced a new metaprogramming technique here: getting a pointer to a metaclass using the NSClassFromString() function. As the name suggests, we can get a class definition from a string representing its name. We can use that to determine the properties of the class.

This function will eventually give us a list of all property names of object managedObject that match a name in the list dictionary. For example, if we passed it an Owner object and the blacklist defined above, we’d get the following output:

@[ @"dog" ]

We know that the dog property of our Owner object is blacklisted and shouldn’t be traversed.

Next we’ll add a little helper method to determine if an array contains a string:

+ (BOOL)isString:(NSString *)string inList:(NSArray *)list {

    for (NSString *item in list) {
        if ([string isEqualToString:item]) {
            return YES;
        }
    }

    return NO;
}

Perhaps a set would have been a better data structure here; I might revisit that.

Now we can modify the diff method so that it checks the blacklist before attempting to examine an attribute or relationship:

...

NSArray *attributeNames = newObject == nil ? [[[oldObject entity] attributesByName] allKeys] : [[[newObject entity] attributesByName] allKeys];

NSArray *blacklist = [self propertyNamesForManagedObject:newObject ? newObject : oldObject inList:blacklistedProperties];

for (NSString *name in attributeNames) {

    if ([self isString:name inList:blacklist]) continue;

    ...

}

We do something similar to ensure that we traverse whitelisted relationships:

...

NSArray *whitelist = [self propertyNamesForManagedObject:newObject ? newObject : oldObject inList:whitelistedRelationships];

for (NSString *name in relationships) {

    NSRelationshipDescription *relationship = [relationships objectForKey:name];

    if (![self isString:name inList:whitelist]) {
        if (relationship.deleteRule != NSCascadeDeleteRule) continue;
    }

    ...
}

It’s a simple check - if the relationship is whitelisted, ignore the deletion rule.

At this point, the code is far too big to post here. I’ll create a BitBucket repository with the diff/patching system as a library with a demo project at some point. The explanation of the patching code will come later.