While I'm looking through the web.config settings, I saw an interesting setting which took my mind to past.
Few months back, one of my previous workplace colleague told me that a trouble he went through while trying to delete an language in Sitecore. After he had deleted a language in Sitecore and looking into the items in content editor, item versions related to that language has also been deleted from Sitecore. They had to work hard to make everything Okay in the system.
The setting which remind me the above scenario is,
<!-- LANGUAGES AUTO REMOVE ITEM DATA
Indicates if item data is automatically removed from a database when a language is deleted.
Default value: true
-->
<setting name="Languages.AutoRemoveItemData" value="true" />
So, if you have set the above property to false, and then deleted the language, items will remain with Sitecore.
But, off-course, there should be steps taken to make this removal sensible, since there is no point of having items of Non-existing language :-D