One can, to be sure, program a digital machine in such a way as to be
able to carry on a conversation with it, as if with an intelligent
partner. The machine will employ, as the need arises, the pronoun
"I" and all its grammatical inflections. This, however, is
a hoax! The machine will still be closer to a billion chattering
parrots—howsoever brilliantly trained the parrots be—than
to the simplest, most stupid man. It mimics the behavior of a man on
the purely linguistic plane and nothing more. Nothing will amuse such
a machine, or surprise it, or confuse it, or alarm it, or distress it,
because it is psychologically and individually No One. It is a Voice
giving utterance to matters, supplying answers to questions; it is a
Logic capable of defeating the best chess player; it is—or,
rather, it can become—a consummate imitator of everything, an
actor, if you will, brought to the pinnacle of perfection, performing
any programmed role—but an actor and an imitator that is,
within, completely empty.
-- Stanisław Lem,
"Non Serviam" (chapter from the book
A Perfect Vacuum (1971)).