Another quick post that I came across relating to linux filesystem comparisons. It effectively draws the same conclusion that I have personally started to formulate, that XFS is a pretty decent filesystem.
http://www.debian-administration.org/articles/388
I have recently changed to XFS on a large USB drive and it definitely seems to have helped speed things up!