Stress
This is one of the least stressful of horticultural industries. The stress of incoming products and material is about as exciting as it would be for any other retail store, and shouldn't be too difficult to manage. The finer points of business expenses can likewise be stressful, but that comes with the territory of managing your own venture.
Unlike grocery stores or other retail outlets, nurseries can be subject to the ravages of nature. Plants can become infested with insects or diseases. The summer sun can kill them, as can unexpected freezes. Snow that's not deep enough to keep the nursery closed for the day is probably enough to keep customers from coming in, and can make it unpleasant to work outside. Making snow angels gets old after a while.
Regardless, many people take retail nursery jobs because they don't want to deal with the stress of other work. Or they do it when the last of their freelance catering gigs has dried up. Either way, there are a lot of pleasant, stress-free parts of working in a plant nursery.