When working on a feature/story, it's pretty common to feel pressure to just get it done. The source of this pressure might be external, from iteration deadlines and managers breathing down your neck, or it might be internal, such as just wanting to the satisfaction of finishing something. Invariably, in the process we come across a few things that we would like to clean up but we decide to put off. "Let's refactor that later", we say to ourselves, "once we're done". This is fine, especially as we don't want to interrupt our flow, as long as "later" happens.
The question is: when is "later"? In the drive to get stuff done, to get the card signed off and demonstrate progress, it's tempting to think: "let's just get this card tested and signed off and then we'll refactor". This might be fine on a codebase where there are extensive unit and acceptance tests, but if your project is anything like mine, this is a dangerous proposition. On my current project (and pretty much every project I've seen so far), we still rely on manual and exploratory testing to validate that everything works properly. Refactoring after the fact is a way of potentially introducing bugs into the system. If we do refactor after sign off, we could go back and ask for the card to be reverified; however, the customer (tester/analyst) is probably not going to be very happy about having to do this work again and our manager probably isn't going to be very happy about it either ("what are you doing refactoring this card? you're supposed to be working on this card now..."). Basically, when we hear the words "refactor later", a little alarm should go off in our heads.
I think that part of the reason that we procrastinate about refactoring is because of a lack of courage: a lack of courage to take it on and a fear that we might not finish in time if we do. As a coach, I think that one of my primary responsibilities is to help give courage to my pair. So when I hear "refactor later", I try to encourage: "Let's do it now", "Let's try it out". If they want to proceed, that's fine, but I try to make them agree to come back and finish it off before we can say we're done.