We had, not too many years ago, the concept of "politically correct" speech. Much as it was vilified by some, it meant that a person's racism/misogyny/homophobia (etc) was best kept to oneself in polite society, and for the most part people realized that the explicit hatred in those attitudes was not socially acceptable. That's gone now, unfortunately, and in the USA and other countries people at the highest ranks of government and society feel empowered to "let it all hang out" and spout even the nastiest of comments in public spaces. That has allowed those attitudes to be mainstreamed. I desperately hope that hate speech is not the "point of no return" which you mention, but I fear it is.
The "point of no return" is more a cumulative set of factors related to cognitive, psychological, and behavioral impact from widely shared experiences within the ingroup. Hate speech and various dynamics related to it (specific words, conduct, the persons individual impact from prior life experiences, etc) all are part of it's not a single thing. Since this exact concept isn't really termed or studied this way as far as I'm aware, there's no real backed indicators for it. Although, just from personal ideating, I don't think you could make cookie cutter indicators from this due to how many factors are involved vs creating one per society referenced.
Here in the US, slightly adjusting from a polarization indicator itself, I think a good one reflecting being near/at that point would be "politically correct" speech being dropped
by its proponents, rather than by adopters. For example on why, a lot of RW politicians may only adopt it for reputational reasons and readily drop it when the environment wont be as negatively impactful, using them as the root for the indicator would end up making it a poor indicator.
Another good one would be using a mix of data analytics, populace polling, and a few different supporting applied sciences (like social psych) to monitor either isolated conflict between two discernable ingroups which make up a predominate part of society, or alternatively as more of a cascade across many discernable ingroups within society. Then you'd specifically look for those features of increasing beliefs (aligned with behavior) reflecting segregation-ism, you could look for marked increases in exaggerated metaperceptions with perceived threat outcomes alongside willingness to commit violence, etc. There's actually frameworks and stuff that happens in this regard already but it's usually narrowed to things like supporting NGOs or governments/militaries in understanding conflict zones they're operating in, so, adjusting the indicators themselves isn't entirely baseless in context.
Also important part on the point-of-no-return, hard to lay out theories like this in short so maybe framed a bit bad above. This doesn't always mean it's a society ender or anything, just brought up in that context. It can happen in isolated contexts between a specific ingroup-outgroup, although that does leave social and political system support for reconciliation pathways. The actual society wide context would be more relevant when this is either, or both, discernable across two groups which make up a predominate part of society, or alternatively when the conflicting is escalating on a societal wide scale vs isolated between two groups (going back to the indicator ref above). The social and political systems within the society also become fractured and/or degraded to the point those reconciliation pathways are not supported at that level.
This goes into those 2 paths I mentioned. The 1 with "good successes" is a bit odd to digest sometimes. It's unification. Unfortunately, ingroups and their outgroups will always conflict. Our natural human instinct will be to conflict over lowest denominator, which will first up usually be things readily observable such as race, gender, physical appearance, political affiliation in some societies, etc. Unless there is intentional human intervention in this dynamic, it generally will lead to actual open conflict (eg violence, eugenics, slavery or other forms of disenfranchisement and abuse, etc). We have likely
all experienced this already. How often do we hear about generic school bullies stopping when no one intervenes and the person being bullied does not also?
What "unification" actually means is hard to swallow though. Outside of being insanely difficult since it has to be actively controlled constantly (why it largely fails, especially the more successive administrations/presidencies/regimes/etc it goes through), this also generally means subconsciously influencing large parts of society to orient focus around shared problems, and combatting those shared problems. You then forge new ingroups through this mutually beneficial engagement, where you actively chip away those readily-available factors and offer new ones up on a platter. This then creates a new unified identity which you can forge into an ingroup of its own, although you can't stop there or it will naturally split again. This is true from the kingdoms in Abrahamic religious scripture, through ancient China, empires throughout the Middle East, Rome and the Byzantine Empire; all the way through things such as
only partly the USSR and modern China in concept theory.
They all largely had/have developed concepts for this with their own cultural twists to them based off their respective experiences. People do dislike this though because, for example, it would require controlling and influencing certain discourse, even if reasonable, that may cause unaccountable imbalances. Concept theory is important here because, obviously, some of those examples did horrible things path 2 covers in practicality, although their actual theory behind it isn't necessarily intended that way (as an external example, plenty of examples where communism nearly always goes wrong in practice but is fair to point out it's not explicitly held in theory, thus there's debates on it amongst communists even).
For folks who may know some things about countering radicalization, or programs to rehabilitate folks away from racism and etc - and even influence operations. This is a strategy used in all of those with tiny twists for respective objectives.
You also need to go all hands on actually fixing the underlying problems also, they will be a massive, if not impassable barrier to trying to achieve that unified identity even, let alone the ability to formulate it into a discernable ingroup.
The 2nd form we see is when it takes a darker turn and decision makers, based upon their respective ingroup stances and/or personal desires, begin to project their own specific perceptions as policy to correct this issue. This is how you get things like China's forced attempts at cultural identity changes to 'force' people into the unified national identity. This is wrong, this also never works, not worth considering anything related to that path. Keeping to the US as an example, if we were to get past that point socially (not just at it but past it), this would sadly be the result we see because the ideas to enable the other path are largely considered bad. We just, don't really consider the opposite it forces. Historically there's yet to be a medium, and US ability to implement influence concepts at all is far poorer than the public may presume (even if this admin did magically become the best rose in the garden, if it was too late "fixing" the implementation issue there it'd be too late).
We see some light indicators of shared support for this, although, noting again its more from not recognizing the overall dynamic vs knowing and choosing. As an example, support for fact checking, rapid response, and removing posts that're manipulative or issued is very high. These in fact aggravate grievances with one ingroup, to correct in favor of another. Generally we don't do anything to account for this besides further message things that aggravate their grievances. If you fuse intentional narratives into a national identity, all of those 3 are irrelevant because the impact will be preemptively neutralized by the pre-set narratives. They already hold predominancy and the manipulative content would not be able to take footing. This is also one of the reasons objective study tends to find these don't work most of the time outside very specific circumstances that are not the most common use (such as how military PA teams may tactically use it). Leaving out a right leaning example on that one since it's very observable unfortunately.
As a side point, we can totally draw up indicators of which path one may be going down there. You could largely consider things like leaderships political orientation and their personal biases, and how readily they project these biases. You could also take some of the specific psychographic data like willingness to do X and specifically narrow to decision makers. You can look at predominate social forces in society also with the same factors, with specific consideration on how exactly, and what gravity (if at all) they influence the decision makers.
