AFAIK, that's how it starts. Top end unit sales fall, top end unit prices fall, and this starts a domino effect. That 3 million dollar condo now sells for 2.5M. The condo that previously sold for 2.5 now looks overpriced, it falls to 2M. The condo that was previously 2M now has to sell for 1.5M, and so on. Eventually, your 1 million dollar condo can't sell for a million anymore, because you can buy something nicer for one million.
The pool of potential buyers increases as you move down market. So prices may fall 25% for the absolute top of the market without affecting the median price at all.