Play Rock Paper Scissors Lizard Spock against Sheldon Cooper

Dude, that’s mega-awesome! Anything with pop culture references gets me excited, and this doubles up on that in the best possible way! Fantastic work!

I’m curious about the gestures. I’d guess HaGRID detects more than the five gestures used here? Are you using a high threshold on the gestures you want, as a way to filter out the gestures you don’t? I notice that it didn’t detect any others in the video, so accuracy is great! Just wondered if non-game gestures were an issue at all?

there's a lot the model picks up that the game doesn't use.

For the game I map only specific classes to moves:

• Rock: fist
• Paper: palm, stop, stop_inverted
• Scissors: peace, peace_inverted
• Lizard: grip, grabbing
• Spock: two_up, two_up_inverted, four

Any other detected gesture (like point, ok, call, etc.) is just ignored — the game only acts when it sees one of the mapped gestures held steady for ~0.5 seconds. There's also a no_gesture class (class 33) which the model outputs when it doesn't see a clear gesture, so it handles the "hand in frame but not doing anything specific" case natively.

The confidence threshold is set at 0.55, which is fairly standard for YOLO — not particularly aggressive. Honestly the HaGRID dataset is just solid. It has over 1 million images across those 34 classes with a wide variety of backgrounds, skin tones, and lighting conditions, so the model generalises well out of the box. I didn't need to do much filtering at all — non-game gestures just get classified correctly as something other than the five game moves and get ignored.

i am thinking of adding small voice clips too

Noice! And how is Sheldon’s respond handled? Just a randomised selection of one of the five gestures?

yes, random generator. could be extended to 1v1 game with friends(fellow Axeleraties?) 

I was just going to ask that - whether you could make it into a two player game 😄 Or potentially more! Just like Sheldon expanded on the traditional rock, paper, scissors, maybe there’s a way to make this a three (or more) player game? 😄

This is so much fun! Thank you for posting this!

I’m just dropping this comment to keep informed on any (if any) updates here to hopefully try it out myself in the future!