Building off of JB’s answer it can also be attributed to the atmosphere and politics surrounding rocketry over the last half century.
Until the last decade or so aside from the select few corporations that are apart of the MIC there were not many organizations that manufacture launch vehicles or satellites. The only demand for those satellites were for military purposes and for communication(Still similar to now though).
There was no push from either the manufacturers or the customers to reduce costs and that large upfront development cost is a huge barrier to entry for spaceflight. When you have such a huge upfront cost decisions were made to spend it on what works rather than try something risky like any sort of deviation from the 2-3 stage recipe that still is dominant today.
What we had with ULA was a legitimate monopoly on US government launches and they also performed pretty poorly worldwide in the commercial market due to their large costs, but that didn’t matter because they could sell Atlas V’s and Delta IVs for 250-500 million a pop to government on top of a $1 billion a year subsidy for launch readiness. There was no competition to promote innovation.
If the current trends of falling costs to orbit and increased interest(anecdotal) in spaceflight we seem to be heading in a direction where, soon enough, companies will begin investing in these risky new launch vehicles(ex: BFR and Skylon).
The reduction of cost is what has allowed the vast majority of what we’ve seen to happen. You can go on rocketlabs website right now and book a rideshare for a cubesat with a fancy gui. You can do a lot with a 12U cubesat and it “only” costs a million to launch it. Though it may only be 20-30 kg which is ~ $50k/kg putting a complete satellite into orbit for that little is unreal. As more and more players get into the game competition will brew to drive down costs and find new technologies that could give you an edge. That’s the hope anyways. The key to all of this is that the cost to orbit needs to be driven down.
On top of that there is still a lot of research to be done for engines that can propel craft up to Mach 5 let alone past it. Mach 5-6 is about the speed limit for ramjets and to go faster you’d need a scramjet. I don’t know much about either of them other than their operating speeds.