I just reread the first chapter of Penrose's _The Emperor's New Mind_ so this is based off of it.
Let's simplify the problem to just conversation, for the sake of simplicity. Suppose we have a program that can pass the Turing test... But the way it is constructing it's responses is only a more capable version of the emacs psychiatrist. Does it have consciousness?
I posted a link to ENM in another post. Maybe look at the first chapter. It hits the nail on the head, IMHO.
EDIT: Or even just read the first few pages... It seems to introduce it decently, if your short on time.
Let's simplify the problem to just conversation, for the sake of simplicity. Suppose we have a program that can pass the Turing test... But the way it is constructing it's responses is only a more capable version of the emacs psychiatrist. Does it have consciousness?
I posted a link to ENM in another post. Maybe look at the first chapter. It hits the nail on the head, IMHO.
EDIT: Or even just read the first few pages... It seems to introduce it decently, if your short on time.