At night, you're only going to see a rocket when it's in one of it's boost phases. That's when it's actively burning propellant and you'll probably see the glow or trail.
The rest of the flight is unpowered and the rocket has a ballistic trajectory (it may have fin control for additional pointing), and it won't be emitting any other light source. So totally invisible in the night time.
For most of these rockets, I'm guessing they have only a single boost phase at launch, or perhaps a few stages, but all immediately during the ascent phase only. So that's what you're seeing in the videos.
Now, let's say you had a pretty wide angle video of the whole area during the events. At some point, the rocket that hit the hospital should be visible. So it's either a. the rocket originated outside of the available videos and did all of its boost beforehand, or b. the rocket IS in the video, but we don't know which one because it hit during an unpowered phase of flight.