The home would have been a good investment if you found 4% an acceptable return. Generally, that would mean a bigger return than you could have earned on other investments of similar risk.
But wait, you had the use of the home for shelter. Yes, and that value can be measured separately, as the total of all the other costs -- mortgage interest, property tax, homeowner's insurance, routine repairs and maintenance. (For interest and real estate tax use an after-tax figure to account for the federal income tax deductions on those expenses.)
These are unrecoverable expenses that come on top of the investment. That means they are expenses you would not have with other types of investments, such as mutual funds. These costs are comparable to rent, which is not recoverable. They are what you pay to be able to use your home, just as you'd pay rent for the right to use an apartment.
If these costs exceed what you'd pay to rent a comparable property, you're paying too much. They're not part of the investment, which was already analyzed separately.
Keep in mind that as the years go by the interest portion of your monthly mortgage payment will get smaller while the principal portion will get larger. In other words, your shelter cost of housing you own -- equivalent to rent -- will get smaller, while the value of your investment will rise. That's why owning makes financial sense if you keep the property long enough. Note that other shelter costs, such as taxes, insurance and maintenance, will rise with inflation, offsetting some of the progress you make from the shrinking interest cost. But if you rented, that cost would probably rise with inflation, too.
The ideal homeowning situation is one in which you are happy with the investment return and have a shelter cost below
what you'd pay for rent
If the investment return will be exceptionally high, you might feel it's OK to pay more for shelter. But if the investment return will be low, a high shelter cost would just make things worse.
A final thing to note: For this analysis, it doesn't matter whether the home is expensive or cheap. Either way, it can be a good investment or a poor one, depending on how fast you expect home prices to rise -- 3% to 4% is the long-term average. But if you buy a cheaper home, the consequences won't be as bad if you bet wrong.