"It is off-putting to try and read about this apparently amazing language, and have large portions of every (hyperbole) article saying effectively that if people were just { smarter || better || less sheep like } they would use Lisp."

Perhaps you missed the wave of Perl hate that Python and Ruby rode to popularity on. This anti-Perl snobbery hasn't seemed to have hurt them any.

Or how about the snobbery of C users against COBOL and FORTRAN? Once again, C got adopted despite (or perhaps even because) of it.

Or the religious wars over static vs dynamic typing, functional vs imperative, OO vs procedural, and which editor or OS is superior.

This kind of elitism is really endemic in the programming field, is by no means restricted to Lisp, and is probably not very relevant to the lack of adoption of Lisp in the general programming community.

Rather, I'd chalk it up to more of a matter of marketing. Lisp is still viewed as more of a (difficult) academic language, rather than a practical one. It's also rather old by now, and doesn't have the shiny/new sparkle that helped Java and Perl (in their day), and Ruby and Python (more recently). Also, how many killer apps (that a typical programmer cares about) have been written in Lisp? Where is Lisp's Ruby on Rails?

That being said, there is something of a difference between hating Perl and saying that Lisp loses because it's too good and you have to be too smart to understand it. In short, Perl fundamentally sucks in ways that impact your life in big ways on a day-to-day basis. Python does have its problems, and it sucks in some respects, but I would claim that it doesn't reach nearly the level of fundamental suckage on an absolute scale that Perl does.

And, moreover, while Lisp fails to suck in certain ways that Python sucks, it ruins your life in other ways that Python doesn't. For example, I use lots of numpy/scipy/matlab/what-have-you style code. I also use lots of Cairo/GTK. In Python, all the stupid crap to take care of doing that in a Python-idiomatic way has been done already; it's known as pygtk and pycairo, and it's available in every package manager I care about.

In Lisp, all that stupid crap has not been done, or if someone has tried, they have not achieved remotely the level of polish of the Python version. So if I want to use Lisp, I have to spend a pretty big quantity of time becoming the maintainer of the equivalents of pygtk, pycairo, etc., and at least last I checked, the amount of crap and slowdown involved there was much greater than the amount of crap and slowdown that resulted from Python's various sucky bits (slowness, lack of threading, lack of macros, shitty lambdas).

And, moreover, Lisp's lack of syntax is a big win in some kinds of code, but in other kinds of code, it's pretty horrible; for example, with Numpy, I can say

myarray[where(other_array > 0)] = foo(bar(5))

or

myarray[:, baz(bum(4)):baz(bum(5))][bum(baz(6)):, :]

and that's going to be several times more succinct than writing my own slicing code and then having to say 'aref' or equivalent for each node in the syntax tree where there's some array indexing going on. In other words, syntax makes the common case fast for some very key common cases.

So saying that a key part of why Lisp fails is because it's too good and people have to be too smart to understand it is mischaracterizing people who decide to use Python because they don't want to deal with the library problems and lack of common-case syntactic convenience.

(I'm not wedded to being one of the latter people, but I do think that there are some strong points in that direction.)

... which is ironic for students of language design who realize that none of (for example) Ruby, Python, and PHP get lexical scoping anything close to correct, that none of them do object systems particularly well, none of them do an extension system particularly well, and only Python comes close to the amount of available and accurate documentation or the robust test coverage of the language and its features.

You can discount a lot of the rest of the criticism as "I didn't bother to learn what makes the language unique before writing a bunch of awful code" and "I don't like sigils but am embarrassed to sound that much a snob."

(Yes, its imitation of Smalltalk has a few seams showing — like the fact that blocks are not normally objects — but overall it's a pretty good if imperfect system. I certainly don't know of any language that gets closer.)

CLOS, Smalltalk, Perl 6, and Perl 5 with Moose have better object systems. Mixins suffer similar method resolution flaws to multiple inheritance, and the same problems with unconstrained monkeypatching show themselves with unconstrained duck typing in large systems.

(To mean that it is very much different from smalltalk/ruby/java paradigm of single inheritance, encapsulation, and message passing).

Take the evolution of C, for example. First, you had ALGOL, but it had problems. So someone created CPL. But that had problems, so then we had BCPL. Then B. Then C. That’s five separate languages! People on the imperative side of the language divide were willing to throw stuff out and start from scratch, over and over, until they got it right.

Lisp advocates were not.

Fortunately, the creation of Clojure suggests that a willingness to change is finally making its way into some areas of the greater Lisp community.