If you look at this building collapse
https://www.metabunk.org/files/pull-verinage.gif you can roughly see an initial acceleration as the support which was holding it up is removed, then as it impacts the structure below and crushes through it starts to decelerate. It does not continue to accelerate through the structure which is capable of supporting it's weight. Every single example of a building collapse if you measure the rate of acceleration you will see an initial acceleration followed by a deceleration in accordance with Newtons laws. Every experiment you can conduct no matter what the scale a structure which is capable of holding another structures weight when static will cause the crushing structure to decelerate as it crushes through it in accordance with Newtons universal laws of motion.
You are still making the incorrect assumptions the energy required to "crush" the supports under a floor is greater than the energy gained by the above floors falling on it.
Here's a simplification:
Consider the upper part of the building starts at rest, with a velocity 0.
It then falls a floor, hits the floor below it, and must then either be:
A) Stopped, because it could not destroy the floor below it, or
B) Moving downwards at V, less the energy needed to crush the supports, say V-C
If it were, A, then we'd just see the top of the building drop a floor and then stop. Clearly it is B.
So, it's now starting out falling both heavier than it was before, and moving faster than it was before. When it hits the next floor, it will hence be faster again. And again and again. Hence it's accelerating, just at less than g.
This is, or course, a simplification. Just less of a simplification than your analogy.