If it was just Greece and a one time problem, I'd agree that bailing them out would be simpler and cheaper than tossing them overboard.
Instead, Greece is just the worst off of the countries violating all the rules to be part of the Eurozone. If it's impossible to get tossed out, why bother to follow the rules. Sooner or later, you can just cancel 50% of your bond debt (like Greece) or resorting to legalized bank robbery (like Cyprus), all while begging other EZ members for more money to reorganize debts (Greece, Cyprus, Spain, etc., etc.).
Imagine if a partner in a business enterprise acted this way. That person would be fired and would face lawsuits. Criminal charged might even result.
The Eurozone needs to set clear standards and set up a formal procedure to remove countries that refuse to follow the rules. Otherwise, the Euro will end up failing sooner or later.