This is a pretty impressive project, though the results (https://www.youtube.com/watch?v=dwXs9RB6uD0) are somewhat disappointing. From what I see toward the end of the video, the image is blurry with poor colour rendition, although it is fantastically low latency.

I suspect you're currently limited by the optics you're using, lack of white-balance and low CRI lighting. Looking forward to seeing further tests!