Lucky for me there was a theme to this party. Something about army guys/GI Joes, something. The idea was to come dressed in camouflage attire. I don't have any camo shirts that fit anymore, but I still have some shorts (and a hat). I showed up to a house where I only knew one of the people who lived there and nobody else. I made my way to the keg and grabbed a beer without incident. I mingled for a while then inquired if anyone had seen John. I tracked him down after a while and said hi. A little more random mingling, then headed home (on my bike people...I don't drink & drive).
I'm pretty sure that people would have started asking questions of me if it hadn't been a themed party. Nobody else knew who I was (OK, a few other people had seen me at other parties and we recognized each other's faces but didn't know names or anything). Even though nobody knew me, everyone knew that I belonged. I must have been invited and got the memo that it was a camo party because I was dressed the part.
One of my friends used this themed party signaling device after some people had crashed her parties in Orlando. She started telling people to dress according to the theme and that way they were able to identify the party crashers and get rid of them. It's a brilliant and simple idea. You can identify any interlopers (or bums who don't dress up). At that point, enforcement becomes pretty easy.
That's why your next party should be themed.