Stress
Matters of national security and keeping the country safe don't stress you out, do they? Because if they do, this might not be the job for you.
Not only do you have to be on your game every time you put on the badge, you also have to trust your team to bring the same amount of game, roughly. When you're tracking a killer wanted for a spree in four states, the slightest smudge of dirt or piece of hair at a crime scene could determine whether you catch the perp or allow someone else to wind up on the evening news.
There's also the severe lifestyle-cramping involved in being a federal agent. You essentially lose any and all rights to personal privacy when you join the FBI, since you can bet they're monitoring your phone calls and emails. And you thought your boss at the Container Store watched you like a hawk.
If you're in a shootout—yes, we're going zero-to-sixty really quickly here—you have to be able to keep your cool long enough to save yourself and keep those around you safe. And let's not even get into the counterintelligence and counterterrorism stuff. A leaked schematic or misinterpreted intercepted message could cost countless lives.
Despite the obstacles, FBI Special Agents get the job done. They're the ones responsible for catching everyone from John Gotti and John Dillinger to the Oklahoma City Bomber to President Richard Nixon during the Watergate scandal.
You try investigating your President. Talk about stressful.