Yeah tbh I do recommend using this alongside a privacy screen for best protection. Privacy screens also suffer from the fact that they won’t block someone directly behind you from seeing the screen, so both methods have issues.
Any tips on improving accuracy? A lot of it might be due to lack of diverse images + labelling errors as I did it all manually.
I dunno, my only idea is that maybe if you use traditional face detection to find the face/eyes and then do classification (assuming you aren't doing that already?).
Right now that's pretty much what I do. I use YuNet to get faces, crop them out and run detection. It's probably a factor of a not enough data/poor model choice.
Sorry, I haven't gotten around to gathering examples yet. I ran the models on some example videos which is where the accuracy stats come.
Perhaps I have been jaded by the Mac webcam, I agree on most old webcams it wont be great but on newer webcams I have had success.
I did try a calibration approach but it's simply too fragile for in the wild deployment, calibration works great if you only care about one user but when you start looking at other people it doesn't work so well.
Good idea, it may be more fruitful to do that. At least then for the primary user we can be much more certain.
This models supports the EyesOff application which will prevent information extraction by either having a popup, switching to another app, or a notification(you can define the behaviour in a few different ways).
Privacy screens are still useful and I recommend people to use EyesOff and the screen protector. A privacy screen won't stop someone shoulder surfing from directly behind you etc.
Others are already trying to do this https://arxiv.org/abs/2504.06237. I haven't seen anyone take the approach I tried though, as most uses cases focus on tracking the main user rather than others around.
I'm working on EyesOff[1] - v2.0.0 should be out soon.
It's a simple (currently macOS) application which aims to target shoulder surfing by using a locally running neural network to detect those looking at your screen.
I think the best thing I realised was to post what and when I want, with little expectations regarding the number of readers. This way I produce the best content and any reader who does stumble across the posts will get the best read. I suppose my content is sort of nice too, so if I focused on readers my content would inevitably stray from what it is at the moment (sucking the joy out of it for me along the way)
EyesOff a privacy focused macOS app (other platform support coming soon) - https://www.eyesoff.app
Built using python the app alerts you to when someone is looking at your screen, using locally running deep learning models + your webcam.
I developed the app when I felt uncomfortable with working in public spaces, I wanted EyesOff to give an extra barrier of security to shoulder surfing scenarios
Thanks for the feedback! And yeah I can see that, this was why the model being ran locally was a necessary for EyesOff. Also, I hope the source allows people to trust I dont send any data outside the app
This model lets you upload a reference face and then matches to those in images. I think this allows for the "approved faces" function you mentioned.
I suppose a difficulty may arise when we run a few models at the same time, however there is probably a lot of room for efficiency on the table and thanks to the small model size we are already in a good place.
Any tips on improving accuracy? A lot of it might be due to lack of diverse images + labelling errors as I did it all manually.