“Sorry, I already have plans after after work.”

There is no need to mention that your plan is to go home and relax.