Not all wealthy types are elitists, and sometimes when a person gets the feeling "he doesn't belong" has absolutely nothing to do with hoity-toity attitudes from others or a lack of income compared to others.
Private club members... sure - there are some clubs in the area where there is a high degree of elitist "privilege" in certain circles. But I've played a lot of local private clubs over the past decade through reciprocal agreements between our club and other clubs and I couldn't have felt more welcomed there.
So I think the private club stigma is unwarranted a lot of the time. In fact - I've seen an elitist attitude on display at just as many public courses over the years as I have during the 10 years I've been a private club member who's been fortunate to play other private clubs.
People don't have to spend 500 bucks for a driver, or 50 bucks for a dozen balls. They're not required to spend $100 on a collared polo or comfortable golf shorts/pants. eBay has all kinds of great deals for those looking to save money. Every big box store that I've ever been to has run deep discounts on older-model equipment, or a used club section.
But there is no denying that it takes a lot of time to get good at this game, a lot of patience, and for sure - daily greens fees have gone up. And sometimes there's not really a lesser-expensive option that's convenient enough distance-wise from where one works or lives to maintain the hobby at a reasonable level. Expense does factor in, but most of the time it has nothing to do with $500 drivers.
Let's face it - golf just ain't for everyone, for lots of reasons.