I'm weighing in not because I'm mentioned in the thread (although thank you for that, guys!) but because this issue is one that I find really hard to get out of my head when I'm reading. If I read a book that isn't a bestseller and I think SHOULD be, it will drive me crazy until I can figure out what went awry. MOST of the time, I can figure it out. For instance, I adored KETURAH AND LORD DEATH, but it's not going to be a book everyone will adore. I would not give it to my mom and my dad and my sister and my vet. Not because I didn't love it enough, but because I'm sure they won't have as passionate a response as I would. Likewise, I loved SHIP BREAKER, and I did try to give it to everyone . . . and was incredibly disappointed when some of them found it too dark, some of them found it too slow, some of them found it too heavy on the world-building, whatever. I also adored The Monstrumologist, but it was too gory for a lot of people. I love lots of books, but only some of them are bestsellers. The rest are just too specific to my particular tastes for them to be loved by enough people to hit the bestseller list.
A bestseller is not the world's best book. It's not even the world's favorite books. It's merely a book that most everyone can find something to like, just a little. It's a book that has enough universal appeal that you can give it to your mom or your therapy group or your cat walker and be pretty sure they won't be offended, bored, or too distressed by it (generally). That's why there's no "bestseller formula." Because that's the only common denominator -- that a lot of people found it worth buying. Not worth loving.
Personally, I'm okay with this. There are so many other versions of success that don't involve the bestseller list — it's such an arbitrary thing, anyway, changing numbers from week to week, so a bestseller in January is a much weaker seller, for instance, than a bestseller in October. And it's not even truly a bestseller list. If you want to have a good time, have a friend with Bookscan show you the top 100 selling childrens' books for the week. Bookscan tracks actual books sold, and you can see how the list is not even real — we're all being outsold by titles that have "grandfathered" off the bestseller list because they've been on there for so long . . . like A Wrinkle in Time.
For me, the thing that gets my panties in a twist is when books aren't doing well at a much more basic level because of some sort of mishandling — bad cover, no ARCs at BEA, bad timing, crummy placement. There are those sorts of injustices going on all the time in the business, but that's a bit less flashy looking than "WHY IS THIS BOOK on the bestseller list?"
(and by the way, THE SCORPIO RACES has not hit the bestseller list, unlike my other books — I knew when I wrote it it was for a narrower niche than my Shiver books, and I was totally okay with that, I wrote it for me. I'm pleasantly surprised by how well it IS doing, and if I told you the numbers on it versus my bestseller list books, you'd laugh and shake your head . . . because they're nearly the same. Really, truly, the list is not a measure of a book's commercial success).