Contrast is more important than size. A plane at 40,000 feet is only lit by the sun when the sky is as well, meaning it's in a low contrast visual field and thus hard to see. But at night, you can very easily see it's external lights, because they're bright and the sky is dark, giving high contrast. During the day, some of those lights are still on, but even if you can see the plane you'll never be able to make out the lights because of the low contrast.
The ISS is most visible when it's still lit by the sun but the sky isn't, and because it has a large and highly reflective surface area it's in a very high contrast visual field and thus visible. During the day, you'll never see it without some pretty serious optics (still, nothing you can't go down to the hobby store and drop a couple hundred bucks on - the hardware available to amateur astronomers off-the-shelf these days is pretty incredible) for the same reason it's hard to see the plane.
Now, as for speed, what you actually need is angular speed, not absolute. It's much, much faster than a plane, but it's slow in the sky because of distance. It takes a bit under 8 seconds to cover the width of a full moon, which is slower than an overhead airplane but still high enough to be plainly visible, which makes it more visible, not less.