Physical Danger
Thirty-five to forty percent of psychologists are at risk of being attacked by their patients at some time during their career (source). Granted, most children can't do much damage (unless you count ankle-biting), but those who work with teens or specialize in abnormal psychology do face the possibility of assault.
You're not very likely to end up in the hospital, but it's scary for sure. There are some easy things you can do to keep yourself safe, like positioning your chair close to the door, removing any possible weapons from your office, and having a coworker on hand and visible during the session.
Dangerous patients are a possibility, but brain drain is much more likely. If you stress too much about work, and let it get to you, your body might start to notice. Stress can interrupt your sleep patterns and make you spacier than normal. Getting too into your work can also result in risky behaviors like substance abuse or self-harm. So make time in your busy schedule for some yoga, and get a non-lethal zen garden for your office desk.