Antarctica has a permanent population of about 1,000 and runs over 4,000 during the summer months. And it's not always the same 4,000, tens of thousands of people have been to Antarctica, and hundreds of thousands have been either on waters or in the air nearby. So there's quite a few thousands there.
Let's not forget that you can just slap down some money and buy a trip to several of the outlying stations like McMurdo. Only a few hundred people have done this, but it's something anyone can do. Just googling Trip to Antarcita I found a half dozen travel agents in the US who have special landed packages, and virtually every cruise line has a route that goes within sight of the ice.
There's also an immense fishing industry in Antarctic waters, including one of the largest illegal fishing industries and the bulk of the world's remaining whaling.