Does anyone know of a raspberry pi (or similar) webcam system that can upload an encrypted video stream to a server? Preferably a (any) plain file host rather than hosting some custom service, but I'm interested in either option.
I want to have an off-site backup of the security cam but preferably only access my camera data myself. That's what I'm always missing in these projects. There are dozens, perhaps hundreds of raspberry pi (security) camera setup tutorials and scripts with motion detection and all sorts of features, but I haven't found any that implement any sort of encryption, if they do uploading at all...
A lot of UVC webcams will output MJPEG, basically a sequence of pre-compressed JPEG frames, which would relieve pressure from the Pi's CPU.
With some work I suspect you could combine tools like ffmpeg, tar, GPG (with private key stored off-site), and rsync to: 1. capture and extract JPEG frames, 2. bundle them up in roughly equal 1MB-ish tarballs (to prevent leaking information about how compressible the frames were), 3. encrypt them, 4. store locally in case the network fails, and 5. ship them off to be reassembed later.
Either pipe it all through the shell (if possible) or maybe use a RAM disk as a scratch space, and beware of swapping or disable it altogether.
I'm not super familiar with GPG, but bonus points if it can generate a new random symmetric key for each bundle. That way if someone steals the device, they can only decrypt a few of the most recent frames using whatever key is still in RAM.
Depending on your setup you could restream (as in reencapsulate) your RTSP stream to HLS and and restream it through ffmpeg to another container at the end of the chain.
Doesn't http progressive download (over https) achieve this? I could be wrong, but I think if you just configure nginx with SSL and have the webcam dump to some sort of a raw file format, you should be able to download the file and since you're using SSL now it's encrypted. It won't be a true streaming server, but should be close enough.
You may need to have the client send a byte range header with each request to ensure you always start near the end of the stream so it's closer to realtime.
This doesn't satisfy either requirement: "encrypted video stream" most importantly, but also "[using a] plain file host rather than hosting some custom service" since a plain file host won't pull my https stream, I'll need to setup another service to do that. Or did I misunderstand what you meant? Either way, thanks for helping think of solutions!
Depends what you mean by "plain file host". To some that means a server I control that has a simple file share configuration. To others that means Google Drive. In the case of the former just mount the storage securely on the pi and save "locally" to it. Could be as simple as mounting the remote server via sshfs and selecting that mount directory to save the file.
In the case of the latter usually a similar concept but depends on the service/protocol.
> Could be as simple as mounting the remote server via sshfs
Right but that's not encrypted and requires me to maintain another server. With unattended upgrades it isn't much, but it adds up if you have ten other things already running and you're actually trying to downsize this time commitment (mainly for the major version upgrades which take days out of holidays every few years, upgrading and re-testing everything again, sometimes fixing dependency issues, reinstalling things...).
Storage can be an "as a service" thing these days, indeed Google Drive / OVH NAS / hosted NextCloud / HiDrive / etc. is what I meant, basically anything that could interface with a pi in a way that it uploads the encrypted stream, so you capture whoever unplugs it -- at least that's how I imagine a regular security camera would work. But then, the only things I find online start at 4 figures and none of them are open source so who knows what encryption (if any) they really apply... Is there really no one on this planet running open source encrypted cameras, are all these systems that consumers use (some of whom are hackers) 100% proprietary, plaintext, or only stored on-site?
I'm not sure I follow how sshfs is unencrypted, it's literally just "I have a server with ssh/sftp enabled and I'm going to mount it's storage that way". Most certainly encrypted.
If you mean the individual video files post upload then it's probably easier to just encrypt the drive on your storage server rather than every individual file you upload to it.
encrypted video stream? you mean to encrypt the stream itself?
raspbian has support and packages for everything you could do with openvpn on a normal debian system, so you can totally build a point to point openvpn tunnel for traffic if that will suit your needs.
if all you want to do is upload image files you can script it with ssh public/private key auth and something like rsync/sftp/scp
And indeed Debian (raspbian) has a ton of components that I can string together, but it doesn't appear entirely trivial to make a low-powered system encode (h264 is expensive by itself) and encrypt a video stream of a decent frame rate and resolution and upload it to a standard file host, all close to realtime (perhaps up to ~4 seconds delay, that should be fast enough even if the intruder knows about the cam (Kerckhoff's principle) and immediately moves to pull the plug).
I don't think it can be done with entirely off-the-shelf components actually, it probably requires quite a bit of glue to make the components work together, cleanup files (locally and remotely), etc., but I'd be happy to be told I'm wrong :)
Edit: did you edit this in?
> if all you want to do is upload image files
Well kinda, but multiple images per second and preferably with some delta algorithm so they're not 100k×86400×fps = something on the order of 50GiB/day. A h264-compressed stream is an order of magnitude smaller. (Sunrise is the worst, increases the stream size by ~3×.)
The CPU horsepower may be a limiting factor, but your best best for putting it together from common raspbian/debian software pieces would be using ffmpeg as an RTSP video stream server, and a separate way of encrypting the network traffic end to end (openvpn, wireguard, a more traditional ipsec tunnel, mounting a remote mount-point over sshfs and continually writing small temporary files to the destination, etc). Then at the destination end, have your choice of RTSP client/stream-receiving software pulling the video feed.
Some of the back end software pieces to do it are not very dissimilar from a typical 'zoneminder' install on amd64 debian. Yeah I edited that in after I forgot to include it before hitting the post button...
on something like a raspberry pi 3b+ or 4 you might get good results by using a video frame rate around 5fps. Or even if you went down to 2-3fps that would be functionally equivalent to a constant stream of still images, depending on what the stream is needed for. If it were a security camera that needs to see peoples' faces the frame rate could be too low, a moving person might pass through the frame without any good shots of their face.
If you have lots of network bandwidth you can do a great deal with ffmpeg by not transcoding the video from the format it's acquired from, which is very helpful on systems with limited CPU power.
ZoneMinder doesn't do encryption, not sure if it would allow adding in gpg as a plugin somewhere.
Network bandwidth is generally not metered on home connections, so that's indeed not an issue, but storage isn't infinite. Transcoding the video would be a plus and can't happen remotely due to encryption. Raw mjpeg is also an option though, storage isn't that expensive (or throw more CPU at the encoding system; a raspberry pi is just an example).
> encrypting the network traffic end to end
but the video stream? The idea is that the storage server (Google, for that matter) doesn't need access to my video stream. Not sure if end to end encryption is too much to ask but I did figure since it's 2020, it should be doable.
Thanks for your responses by the way, it's helpful to have a second pair of eyes on finding a simple solution. Since there doesn't appear to be one, I will start experimenting with some ideas, feeding ffmpeg to gpg and checking cpu vs. compression ratio etc. Next step will be seeing which protocol to use for local + remote writing of the encrypted file (not sure if sshfs does incremental writes / appends). Let's see how hard this is! :)
> Transcoding the video would be a plus and can't happen remotely due to encryption.
why couldn't it? Unless the remote system (where the video is being received) is untrustworthy. In my scenario I was describing network traffic encryption, for a theoretical where the entire network between the video-sending and video-receiving linux systems could be 'hostile', but not encryption of the video itself, I was assuming that the location where the video is being sent to is trustworthy.
If the video needs to go to a remote storage location where you want files to be written to disk, but nobody else to be able to read them, that's a somewhat more rare problem to solve. Possibly something involving a virtual machine with a dm-crypt/LUKS implementation for FDE.
> Unless the remote system (where the video is being received) is untrustworthy
Well, that's part of the threat model. There is no need to trust the remote end fully, especially since I'm hoping that I can host at Google / OVH / Hetzner / whatever. I trust them as a backup and not to collude with a burglar, but why give them plaintext data?
... To answer my own question, not adding encryption makes life easy I suppose... but that's not very satisfactory imo :)
I want to have an off-site backup of the security cam but preferably only access my camera data myself. That's what I'm always missing in these projects. There are dozens, perhaps hundreds of raspberry pi (security) camera setup tutorials and scripts with motion detection and all sorts of features, but I haven't found any that implement any sort of encryption, if they do uploading at all...