The problem of governments’ over-indebtedness is one of the most important challenges for today’s EMU governance. As numbers suggest, the problem of extensive deficits has appeared in the EMU long before the burst of the global financial crisis. We suspect that the membership in a currency area might be partially blamed for such progression of indebtedness. This paper examines the determinants of government risk premiums in the EU Member States to answer if the risk premium assigned by the market may give currency area Member States additional incentives for profligacy. Controlling other factors, we investigate the pattern in which fiscal deficits and GDP growth affect the yield of 10-year-maturity government bonds in the euro area and the non-euro area EU Member States. Our results are straightforward. The market penalizes EU countries that do not belong to the euro area for bad economic performance and extensive deficits from 4 to 7 times stronger. Our estimates confirm the strong impact of the common credibility problem in the EMU but also support the key role of financial stress in determining the cost of government debt.