If someone has a bad memory, do you tell them?

My Dad and Granny have horrible memories. They both tell the same stories every time you talk to them. They get stories so twisted, and just seem generally confused no matter how many times you try to tell them how something actually occurred. For instance, I told my dad that my cousin "Heather" told me something which gave away a surprise party for me and the next time I talk to him, he says my friend "Tiffany" is the one that told me, and is insistant upon it, and has even told my Granny and others that "Tiffany" blew the surprise for me. He goes on talking about how bad my Granny's memory is and says "Tell me when mine is that bad" BUT IT ALREADY IS! It's so frustrating talking to them because they go on for hours with the same exact stories every time we talk. I think it's rude to tell him that his memory IS that bad, if not worse than Granny's. Should I tell him that he does have memory problems??