If the bond is taxable, the annual accruals will be reported to you on a 1099-OID form and you have to pay income tax on them. That's why many people opt to hold taxable zero-coupon bonds in tax-deferred accounts. If the bond is a tax-free municipal, the accruals aren't taxable and they won't be reported to you. Simple as that.
Where it Gets Complicated
It gets more complicated when a zero-coupon bond is either bought in the secondary market (as opposed to being bought as a new issue), sold before maturity, or both.
Let's start with the sold-before-maturity scenario.
If you sell a zero-coupon bond before it matures, you may incur a capital gain or loss. That depends on how the price at which you sell compares to the adjusted issue price, which you've been calculating each year for tax purposes (see the example above). If you sell for more than the adjusted issue price, you have a capital gain. If you sell for less, you have a loss. It doesn't matter whether the bond is taxable or tax exempt.
On to buying in the secondary market, the subject of your question.
If you buy a zero-coupon bond in the secondary market, and the price you pay for it happens to be the adjusted issue price, you're golden. You just follow the same rule you would have followed had you bought it new.
But in all likelihood, the price you paid will be either lower or higher than the adjusted issue price.
If it was lower, you've got what's called a market discount. That's the difference between the adjusted issue price and the price you paid. Eventually, you'll have to pay ordinary income tax on it, whether the bond is taxable or tax exempt -- but not until the bond matures or you sell it.
In the meantime, you accrue the market discount using the same formula used to accrue the original issue discount, described above. Or you can use a so-called straight-line formula. Under the straight-line formula, if you have 10 price points of market discount and 10 years till maturity, you would accrue one point of market discount a year.
If you hold the bond to maturity, you pay income tax on the full amount of market discount at that point, whether the bond is taxable or tax-exempt.