Most small farmers are part-timers, relying on income other than their farming. From what I've read, that's the norm. But there are circumstances where you can get around that, but these surely won't apply to most people. In my area of Hawaii, there are many mini and small farmers that are surviving. Maybe not with a lifestyle that would appeal to you, but they manage to live.
This first thing that usually comes to mind when I talk about off farm income is getting a job in town, something that is really scarce around here. But that's not the only means. Some people take the occasional temp job when they come available....helping putting up fencing, weedwacking someone else's property, picking up macnuts, doing minor house repairs, house sitting (called caretaking here), caring for livestock when the owner is away, etc. Some people actively seek handyman jobs to fill in between farm chores. Others develop a small side business, such as small engine repair, to fill in the income gap. Mini-farmers are often eligible for SNAP benefits (food stamps), which can add a monthly "income" that really helps. If of retirement age, Social Security or pension benefits can make the difference between surviving or having to quit.
The small and mini farmers around here tend to have two things in common. One, they work long hours, far more than 40 a week. Two, they live a more minimalistic lifestyle than most people. By government definition, they are in the poverty class. Never hungry, never without shelter or clothes, but certainly lacking. All own their farms and have no major debts. Most cannot afford health insurance. Medical expenses (insurance, deductibles, co-pays, costs not covered) can easily be $10,000-$12,000 a year for a husband/wife here. That could be a mini-farmer's entire yearly income! And most cannot afford homeowners insurance let alone farm liability insurance. So most "fly naked". So those who want insurance need to have off farm income for sure.
A small farmer not far from me just started offering group retreats. He supplies the venue, the housing and meals, and picks participants up at the airport. A friend of his conducts the retreat and they share the income.
A mini-farmer set aside a wooded acre for individuals retreats, set up with funky tropical settings, meditation spots, workout areas, platforms with vistas. Popular with mainlanders who need a spot to get away for a week and be catered to.
Another mini-farmer conducts tours. He does car and ATV tours plus hiking tours.
I have seen many people in the last ten years try to be small farmers and fail. They needed more money than they could generate. Some were not interested in working so many hours. Some seemed to lack the passion needed. All were not willing to adjust to a more minimum lifestyle. They decided that the life of being a poor farmer was not for them. I don't know of any rich small farmers here. Many are comfortable, some are less comfortable than others. Most struggle to some degree or other.
So why be a mini-farmer or small farmer? For me it's a lifelong passion being fulfilled. For a friend of mine, it's the need to be independent and have some control on what he does each day. Another friend told me that it's for the freedom. Yet another said its for the peace.