When you are going to have multiple language set up on your Drupal site, it's important to set the default language appropriately before creating content. Once that is set, content will normally be set to be in that language, and any translations made on the site will be assumed to be from the default language as the source. So changing it is not a good idea, as there's no way to differentiate between translations made before and after the switch in Drupal 6 or 7! (This has been resolved in Drupal 8.)
So, once you've thought first about what is necessary for your multilingual site, the next step is to pick the right default language, ideally before setting up anything else, as everything is 'in' a language in some way. It's usually an obvious choice, but did you know that the Drupal software itself and associated modules (i.e. the codebase, referred to as the 'interface') is all written in U.S. English (as per the coding standards)?
This means that the translation interface for all text that comes from the codebase only allows you to translate from U.S. English. Any text entered from elsewhere (e.g. content entered into nodes by editors) will be either be in the default language that is configured, or any language that is specified when setting the text (e.g. the node language). As the interface is in U.S. English, it can be confusing to have content configured by default to be in a different language - especially as the line between content and interface is blurred (e.g. text within a view that is provided, in code, by a module). So it can help to deliberately set your site's default language to be U.S. English, so that all translation is consistently from that into other languages. Of course that doesn't make sense for many use cases - but it's just worth considering when deciding which language to make a default.
For example, I have built sites where we decided to make British English the default language, as the European audience was prioritised. But as it's so similar to U.S. English, it's easy to get confused between which text should be entered in each version of English. Given that translating went through agencies too, it could have been helpful to have a single workflow for all types of translations - as it shouldn't have to matter whether the text comes from code or an editor.
Text in the interface can only be translated into languages other than U.S. English, so use the String Overrides module for overriding rather than translating that text. (We have to use it with this patch.) This module will be useful if you want to customise any interface text on your site, regardless of language, but it is particularly handy for allowing existing translations of a certain piece of text to remain as they are whilst changing the 'original' text. Changing the 'source' text would otherwise mean new translations are required. Look out for these cases where you may want to make changes in future, and use string overrides on the source text rather than changing it directly - your editors, translators, and users of other languages will appreciate it!