Stress
Fishing, like most other jobs, has its great days and its not so great days; how you cope with the not great days all depends on how patient you can be. Sometimes this means waiting out a storm on land that never seems to end, or ending up in a fishery undergoing a severe dry spell. Nets break, machinery rusts, and boats occasionally run out of gas.
Couple that to the fact that your income is tied directly to how many edible aquatic life forms you catch, and you can see how this job can turn even the brightest young sailor into a grizzled and grumpy old curmudgeon.
Patience is not a virtue that solely belongs to the people on the water. Unless you're nineteen and basically the seaborne version of an intern, there's likely someone back on shore waiting for you to come home. Not only will the time pass for your family without you; they'll often have no way of knowing what's happening to you at sea.
Of course, your job is tough, but understanding is a two-way street. If you're planning on subjecting your significant other to this kind of lifestyle, let them have at least a couple hours of "shut-up-and-listen" time when you get back.