It's the same design, so why should I pay a fee to re-use them?

I've run into this scenario a few times.  Actually more than a few - let's try multiple times.

A prospect recently asked me what it would cost to re-use permit ready drawings I had drawn for another client (we're talking about two separate parties here) because this new prospect really likes the design. 

What they are really asking, without knowing it of course, is "What's the licensing fee for me to use a design you've created?"

Here is the conversation, paraphrasing has been employed to protect the parties involved.

I lead the conversation with "Well, if you are not making any changes what so ever to the design, then you pay the license fee for the plans."

"Great! So is that, like, a couple of hundred bucks?" is the next question.

"Actually, that would be more like a couple of thousand" is the reply.
 
Insert pregnant pause...

"Well", the reply comes back, "you've already been paid for the drawings, I just want to re-use them".

My reply, "So, do you pay the painter for every home they paint or ask for a freebie if it's a repeat?"

So now the prospect's decision hangs in the balance. 

Do they commit to licensing my design or do they find a low ball estimate to basically re-draft my drawings with the necessary changes made to the title block?

My question is, why is licensing a set of drawings such a show stopper for builders?