Aidan Delaney’s weblog

A very nice person, by the name of Allen B. Downey, has written several textbooks. All of which are available under free licences. So you can have the source and tinker with them.

A few years ago, I adopted his free-content C++ book as my standard course text. Since then, I’ve made a few updates every year. The latest version is downloadable from Google Docs.

I have an update to “my” OpenGL book in the pipeline too.

Two of my students so far have released their (in development) final year project code as Free Software using Canonical’s Launchpad system. I hope a few more of my students will follow suit but, of course, I’m not insisting that they do so. The students are

• https://launchpad.net/~cwanstall, and
• https://code.launchpad.net/~ollie-petch.

The project have not been submitted yet so I won’t make any value judgement on them. But, as the code exists, it is obvious that some progress has been made.

I finally sat down for a day to write an application I’ve been wanting to write for a while. I’m on holidays, so I can do fun things like this now. I choose TurboGears as the framework to achieve this. I could have chosen other frameworks such as Django, Ruby on Rails, PHP Cake, etc…. But I didn’t. I choose TurboGears because it’s a Python framework and I feel that I need to develop my Python skills. I choose TurboGears over Django because the Fedora guys have written some very neat applications koji and bodhi. I didn’t choose it because it’s necessarily better, but because I’ve seen some good results achieved using TG and because I have a penchant for build systems which’ll make me want to dig into koji at some later date.

TG is based on the model-view-controller design pattern. If you’re a student of mine you certainly know this pattern For the model it uses SQLObject as an object relational mapper. The controller is CherryPy and to construct an AJAXified view one uses the Kid template engine and Mochikit. TG essentially integrates these four independent projects to make them easier to develop for and deploy.

I now like developing in TG as I’ve figured out how to do it. I did find it extremely frustrating to figure out how to develop in TG. This is down, I think, to two issues;

1. me expecting the SQLObject ORM to do more than it does, and
2. the documentation in each of the five projects being written in a different manner and to differing standards.

I found the TG screencasts great for giving an overview of the project and explaining that the separate components work together. I found the screencasts gave a poor explanation of how to make the components work together. Most of the tutorials/screencasts I found concentrated on “Using SQLObject with TG” or “CherryPy controllers in TG” but I’ve not yet found an example of a very simple application taken all the way from start to finish using TG.

Enter Listy. Listy is the most simple application I could think of. It allows you to Create, Read, Update and Destroy list enteries. That’s it. It’s really dumb but exercises all aspects of TG in a very simple manner. I’m going to assume that you know how to use tg-admin to set up a default project and that you’ve read the TG overview documentation.

The model is simple. A list contains items which are Unicode strings:

class ListItems(SQLObject):
  item_text = UnicodeCol()

There are no one-to-one relations or many-to-one etc… If you want to see examples of more complex models have a look at the TG book. I don’t really like SQLObject. It’s too SQL’y for me particularly when it comes to creating relationships between data. One of the gotcha’s for me is that if I create a new ListItems object and modify an item the results are not persisted to the DB eg:

l = ListItems(item_text="foo")
l.item_text = "bar"

I found that, unless you change some of your project configuration, you have to use the set call.

l = ListItems(item_text="foo")
l.set(item_text="bar")

This was very counter intuitive for me. As ListItems derives from SQLObject there are some nice select functions defined and ListItems will also have an “id” field as it’s primary key (you can change this).

In the contoller I expose four simple methods. The returns the entire list as a dictionary via the “welcome.kid” template….we’ll look at that later. The latter three methods expose themselves as JSON calls. Take the modify(self, item_id, item_text) call. TG will expose this as either “/modity?item_id=1&item_text=foo” or the more RESTful “/modify/1/foo”. I like the RESTful way of doing things.


class Root(controllers.RootController):
  @expose(template="listy.templates.welcome")
  def index(self):
    return dict(list=ListItems.select())

  @expose(format="json")
  @validate(validators={'item_id': validators.Int()})
  def delete(self, item_id):
    ListItems.get(item_id).destroySelf()
    return dict(result=True)

The validators in the above code are nice. They ensure that the passed in values are validated that, say item_text is simply plain text and not some SQL injection string like ';drop all; nuke world;.

The view was where I got a little confused. Mainly because if you want to use Mochikit with TG you have to add tg.include_widgets = ['turbogears.mochikit'] to your app.conf. I don’t understand this. I know there are other AJAX technologies (Eg: jQuery) that may be used with TG but it would be nice to have the default one turned on by default. Anyway…. I have a very simple page defined in welcome.kid. It simply takes the list returned in the dictionary from index in the controller and prints it to screen.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:py="http://purl.org/kid/ns#"
    py:extends="'master.kid'">
<head>

<script type="text/javascript">
// intentionally blank for the moment.
<script>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type" py:replace="''"/>
<title>Listy
</head>
<body>
  <h1>A List

  <ul id="list">
    <li py:for="l in list" id="${l.id}"><input type="text" id="input${l.id}" value="${l.item_text}"
    onChange="modify_item(${l.id})" /> <a href="#" onclick="delete_item(${l.id})">(delete)</a></li>
  </ul>
<a href="#" onclick="add_item()">Add list item
</body>
</html>

This produces the list as returned by index() and displays each entry in a textbox. Each textbox has a unique id of the form input10. The onChange event for each textbox is an ECMAScript function called modify_item(id). There’s a delete link on each line whose ECMAScript onClick event is hooked up to delete_item(id). Finally there’s a link to add an item to a list whose onClick event fires the add_item() method.

Let’s have a look at the modify_item method first. It’s the most straightforward.

function modify_item(id) {
r = doSimpleXMLHttpRequest("/modify/" + id + "/" + $('input'+id).value);
}

The call doSimpleXMLHttpRequest() is provided by Mochikit and will work on all major browsers (at least). It simply calls the URL /modify/id/value which was exposed by the controller earlier. The $('input'+id) is Mochikit shorthand for “get me the DOM element with the following id: input10″ (assuming the value of id=10).

The code for add_item() is a little more interesting.

function add_item() {
r = doSimpleXMLHttpRequest("/add");
r.addCallback(add_item_handler);
}

It calls the previously exposed “/add” URL. And then it registers a callback. It is at this stage that we realise the Asynchronous aspect of AJAX. Our simple HTTP request may take a while. Maybe a second or maybe a lot longer if the server is under heavy load. The callback is executed when the results are returned. Here’s the code for the callback

function add_item_handler(result) {
item = evalJSONRequest(result);
id = item.list.id;
node = LI({'id':id});
input = INPUT({'type':'text', 'onchange': 'modify_item('+ id +')','id':'input' + id, 'value':item.list.item_text})
node.appendChild(input)
node.appendChild(SPAN(" "));
node.appendChild(A({'href':'#', 'onclick':'delete_item('+id+')'}, "(delete)"));
(document.getElementById("list")).appendChild(node);
}

The callback takes the result and turns it into an ECMAScript object. Then it creates a new XML node which contains a list item (<li>) which has three children; an input field, a span field and a hypertext link. Given this list item we find the list in the document and append it as a child of the list. This is all pretty standard DOM manipulation. I like the ease with which Mochikit provides the asynchronous callback.

Just to complete the picture, the code for delete_item() is

function delete_item(id) {
elem = document.getElementById(id);
hideElement(elem);
doSimpleXMLHttpRequest("/delete/" + id);
}

where the function hideElement is a Mochikit special. Essentially I tell the controller to delete the specified object and then hide it in the view. I should only really hide the element in the callback for successful completion….but I’m lazy. You can also register callbacks to handle the condition where the doSimpleXMLHttpRequest fails. I don’t. I’m lazy.

Anyway. There’s a very simple TG application from back to front. It looks like the following picture

IBM have Cell development tools…ummm…..obviously. Their tools are GCC And Jochen at IBM has been quite keen to push packaged versions of these tools into Fedora. I have a need for them on Fedora 9 (on PS3). Interested parties can follow the development here. I’ve got every confidence that their GCC package will follow soon.

This is great stuff for a few reasons. One; being totally self-centred, I need a spu-gcc package. I have my own internal one but it’d be nicer to have stuff upstream. Two; IBM previously offered their Cell SDK on RedHat Enterprise Linux. This shows that they are quite happy to engage with the wider community. I know they’ve engaged in lots of different areas but there is a qualitative difference between supporting Linux by providing a download and actively working with the community.

It’s quite novel to be talking to IBM (actually Jochen specifically) and for them to take on board my requirements and use them to inform internal development. It would be hard to buy that level of support from most companies and, at the moment, I’m getting it for free from Big Blue.

Maybe they didn’t Maybe they’re just doing what they were going to do anyway. But at least it’s “The Right Thing ™” and I’m astroturfing them.

The spu-binutils package is available now.

I’m building a toolchain for the Cell processor. This is the processor found in the PS3 and IBM blade servers. Others have built the toolchain before notably IBM, the Barcelona Supercomputing group and YellowDog Linux. However, these sets of packages do not move at the speed that Fedora does. They are currently GCC 4.1 based whereas Fedora 9 includes GCC 4.3. Furthermore, the other implementations of a Cell toolchain do not include the Fedora patches to GCC and binutils. Thus my packages are closer to the Fedora packages than the IBM ones. This makes maintenance and debugging easier from the system integrators point of view (i.e. my point of view).

This toolchain is also part of my evil plan to replace Mesa on the Cell architecture with Gallium3d. Gallium3d is an implementation of OpenGL (amongst other things) using the Cell SPUs. This will allow developers to develop OpenGL applications on PS3 Linux.

In order to do this I need to build the following packages

• binutils for the SPU,
• gcc for the SPU,
• libspe2 in order to control the SPUs,
• newlib for the SPU, and
• gallium3d using the toolchain developed in the previous packages.

My libspe2 package is based on the IBM package but recompiled for Fedora 9, the binutils package is based on the binutils in Fedora 9. Both of these packages are complete. I’m currently arguing with GCC. Then I’ll tackle newlib (a much easier issue) and then Gallium3d.

Test’em and beat me if they don’t work.