The problem now is that we're (for specific values of "we") literally building systems to outsmart ourselves. It's the worst, stupidest form of Singularity.
And it's because the standard is not, "produce goods and services for our society, species, ecosystem, and planet," but "produce value for the shareholders."
That evolutionary pressure is bad fucking news. Because you know what actually controls shares?
Hint: it's not humans.