Of course there is some difference between even correct spelling across the varieties of English. American spelling tends to be simpler than the British, but then it has to be (I kid my dear cousins, I kid). Frankly, living as I do in Japan, a non-English speaking country where American English is the most common form of the language, I find myself looking at my extra Us and the additional "-me" on the end of "programme" and thinking it would be a great deal easier if all forms of English agreed to use Daniel Webster's version. But then I catch myself, like Luke realising he has one black glove.
Horobin makes the good point that English spelling, while infuriating at times, has value because it captures the history of the language. Too true - we Brits at least have all those superfluous Us to remind us of our Norman overlords, and can draw on sturdy Anglo-Saxon vocabulary when referring to our current rulers.
In my view, a common orthography, shared at least among users of each version of English, is needed for the same reason that a shared grammar and even a shared literary canon are needed: they provide common ground for us to communicate.
As for the apocryphal end to a job application letter, "I look forward to herring from you shorty", why assume it was a spelling mistake? Just reply "We're out of herring fatty; will you accept haddock?"