A cannibal father and son were out looking for food one day. They
were hiding in the woods watching for people to walk by. First, a really
skinny man walked by and sond son asked, "How about him Pa?" "No
boy, he's too skinny." said Pa. Then a Really fat woman walked by.
The son asked, "How about her Pa?" "No boy, She's too fatty." said
Pa. Then a really good looking young girl walked by and the son asked
"How about her Pa?" "Yep, we'll take her home and eat yer mother."
 
 

In possibility # 1, the second  missionary would see you wearing a red hat.  If she used the same logic we just did, she'd know that missionary number 3 could not possibly be looking at two blue hats but could be looking at two red ones.

Possibility # 1 

In possibility # 2, the second  missionary would see you wearing a blue hat. Again, using the same logic we just did, she'd know that missionary number 3 could be seeing no more than one blue hat.  So if missionary number two saw a blue hat, she'd know that she couldn't possibly also be wearing one or missionary number three would have already called out red.  So missionary number two would call out red.
 
 

But she didn't!

Which means that missionary number two does not see a blue hat on your head.  The only possibility left over is that she sees a red hat on your head!

So the longer you don't hear either of your companions calling out, the more certain you are that your hat is red.  So go ahead and pick the red hat and see what happens!  Or if you think my logic is faulty, pick the blue hat.