I hear you. I get the same terrible moments when I can't remember which is the Canadian and which is the Scottish. And sometimes it goes so far that I can't tell which accent is which because they both sound so normal.