Since this site is partly to give me practice with web development I have updated the entire site to HTML5.
Anyway, some of the improvements to HTML5, apart from the most famous video and audio tags, are the semantic tags that give structure to a document. This structure allows computers in particular to identify parts of the page with greater precision and determine the relationships of content. If you want to learn more about this you should read the excellent Dive into HTML by Mark Pilgrim. In particular the chapter on the semantic changes was very useful.
There are probably a few broken styles here and there, but now that the structural conversion is done I will probably start modifying the look of the site. So don't be surprised if it looks totally different next time.