I wonder what sort of memory related tech it would take to pack nearly all of the internet in a small space, and have it incrementally update(the internet!) and yet write it in available memory.

Besides any contact with outside world would need communication. So you can't have an entirely standalone gadget.