In column A - list of cities.

Insert into cell B2

http://maps.google.com/maps/geo?output=csv&q=

Into Cell C2

=ImportData(CONCATENATE(B2,A2))

http://apitricks.blogspot.com/2008/10/geocoding-by-google-spreadsheets.html

[[Pamela Fox's approach|http://blog.pamelafox.org/2008/11/geocoding-with-google-spreadsheets-and.html]]

Tutorial how to create a web map from a spreadsheet via @johntynan

There is now a mapping gadget.