WHEN YOU SEE a police car in your rearview mirror, lights flashing, which one of the following best describes your reaction?

2. You flush with adrenaline, convinced you’ve done something wrong.

3. You calmly drive on, assured that somebody else has screwed up.

4. You start preparing an alibi about your broken speedometer.

For many of us, the first response is the most likely. We feel guilty even if we’re not. And this same emotion can haunt us at many other times. One of the most common places to feel unwarranted guilt is on the job, especially around departure time each night. The dilemma: When can you leave work without seeming like you’re in too big a hurry? Many variables are at play. Are your colleagues still around? Is the coast clear of supervisors and busybodies? When did you leave yesterday? Was the boss in a good mood?

Many of us burn too much time and energy playing this game on the job, robbing the rest of our lives in order to avoid feeling guilty about not working hard enough. The work we do may not even need the extra time, but we stay on the job anyway—desperate to score points, fit in, jockey for position, and generally avoid looking like we’ve got a life beyond the workplace. To show you just how absurdly pervasive the guilt reflex can be, the top executive of one large company told me that he goes through the same furtive rituals as his staff. “I’m waiting for my managers to leave, so I can leave, and they’re waiting for me to leave, so they can leave,” he confided. “It’s a standoff.”

It turns out we don’t have to be slaves to our jobs; all we have to do is put guilt in its proper place. Not all guilt is bad, but irrational guilt can drain our energy, efficiency, and spirit. In the long run, it actually hurts the work we do. The first step toward pleading not guilty is to understand what guilt is. Only then can we begin to get out from under the worst taskmaster most of us will face in life—ourselves.

The unnecessary guilt we feel on the job is often in our heads, but it’s real enough that we fail to stick up for ourselves when the demands of work threaten to overwhelm our time with family and friends, community projects and favorite pastimes.