This is often referred to as the Sapir-Whorf hypothesis*, a type of linguistic relativity
(Wikipedia,
https://en.wikipedia.org/wiki/Linguistic_relativity)- the idea that language can directly influence our perception,
e.g. people who use a language that has the same word for yellow and orange can't perceptually distinguish between yellow and orange.
As with describing many qualia (the subjective taste of a strawberry, the scent of a rose) describing a specific shade of colour if the speaker and listener don't have a term for that shade, or shared prior experience of something of that shade, is difficult.
Using the example above, someone might see a yellow frog, tell their friend they've seen a frog using the term for (yellow and orange), the listener might imagine an orange frog.
A strong interpretation of Sapir-Whorf might be that these people only
perceive (yellow and/or orange), one colour, not (yellow) and (orange).
To cut a very long story short, it's broadly accepted that while there is linguistic influence on cognition, humans generally have similar abilities to differentiate between different colours with which they're made familiar, and notice colours which they're not familiar with. For example, if you spray-painted the leaves and grasses around a Himba village bright blue, they'd notice.
*Arguably an incorrect title, but still much used and understood in psychology and linguistics