Physical Danger
Well, when you work for the Mafia, there's always an inherent danger. You never know when a deal's gonna go south and suddenly someone's trying to feed you a hearty lunch of lead salad and a knuckle sandwich.
Wait, sorry—that's the danger if you're a loan shark. As a loan officer, there's literally no danger. You don't actually have any of the money with you, so you won't really be a target for robbery. People may get angry at you for denying their loan, but typically they'll just storm out in an overdramatic huff. Keep your office at street level and out of the underground and you should do fine.