From the desk of Cabel Sasser, regarding Transmit, Coda, and Unison in the Mac App Store:
The Mac App Store may show software bought from us previously as “Installed”, even though they’re two different licenses. You will not get Mac App Store auto-updates unless you purchase from the Mac App Store. To re-enable the “Purchase” button in the Mac App Store, just drag the app to the trash. Your preferences/sites will not be affected.
So the App Store recognizes that you already own the app, but it doesn’t treat it as if you bought it from the App Store. Meaning, on the App’s page within the App Store you’ll see “Installed” instead of the purchase price. However, you won’t see it in your list of apps underneath the “Purchases” tab and you won’t get updates for that app via the Mac App Store.
The Mac App Store won’t let you buy apps that are already “Installed”. According to Cabel, if you want to purchase an app through the Mac App Store that you already own, you have to drag your current app into the trash.
Update: According the Daniel Jalkut (MardEdit maestro), apps you already own will only show up as “Installed” in the Mac App Store if the developer used the exact same Bundle ID in the App Store as with the self-sold app you bought.