If you want a seat at the table then your AI must prove itself capable of winning some competition against the incumbents. How about a nice game of chess?
The article revolves a lot around the "how" and "why" but does not really answer it's own title. We thought we have the answer right there: in democracies, the people delegates working out the rules to a group of elected people. The rules then are enforced by another, independent group of people. Currently this seems to be broken in (not only) the USA.
The answer is: historically the incumbents define the rules and use safety as an excuse to build their moats by gaining exclusive rights and they end up being cartel/monopolies.
(And it is not even bounded. Some years ago, people argued that "it's already broken so voting in the crazy guy who wants to shake things up will flip the switch from broken to non-broken". But they were wrong. As McCarthy wrote, There are no absolutes in human misery and things can always get worse)
Cohere's disagreement seems mostly about who makes the rules. From the essay (which does seem significantly written by AI, sadly):
> The rules around AI are getting written either way. What's still open is whether they get written by a group anyone can join and with evidence anyone can check, or by a handful of companies in a room with the door shut.
Of course none of the AI companies will suggest that they should be accountable for the illegal actions taken by agents they host, but I feel like that could be a very effective "pacing mechanism", and addresses the concerns brought up by this essay about fairness and transparency.
> Of course none of the AI companies will suggest that they should be accountable for the illegal actions taken by agents they host, but I feel like that could be a very effective "pacing mechanism"...
The problem is that the potential liabilities arguably vastly exceed what these companies can absorb. It would be interesting to see if any major insurer wanted to step up and insure them.
On the plus side, in the worst case scenarios, we won't have to worry about damages because nobody will be left to pursue them.
There will be many scientists, myself included, that would band together to break the cartel if it forms. It's just a matter of money. It is absurd.
Any logic shows that this is really about their valuations and being tired of innovating instead of having any safety concerns. What would happen if the US did this? Chinese and EU labs would eventually win. And then either the US becomes a backwater or they have to eventually spend the money to keep up anyway. But in the meantime they will have bought themselves a few years at insane valuations.
Also, look at the stark choice they want to offer "we'll kill everyone or you give us the power to do whatever we want with the most important tech around". That's it. If we aren't in an oligarchy now, we will be if this goes through. They're going to end up stealing trillions of dollars from the rest of society.
RSI scenario is not scientist-limited though. So, you'd better get cracking if you want to catch up before it's impossible. Will first need to raise some serious capital too...
It’s a rhetorical question.
(And it is not even bounded. Some years ago, people argued that "it's already broken so voting in the crazy guy who wants to shake things up will flip the switch from broken to non-broken". But they were wrong. As McCarthy wrote, There are no absolutes in human misery and things can always get worse)
Cohere's disagreement seems mostly about who makes the rules. From the essay (which does seem significantly written by AI, sadly):
> The rules around AI are getting written either way. What's still open is whether they get written by a group anyone can join and with evidence anyone can check, or by a handful of companies in a room with the door shut.
Of course none of the AI companies will suggest that they should be accountable for the illegal actions taken by agents they host, but I feel like that could be a very effective "pacing mechanism", and addresses the concerns brought up by this essay about fairness and transparency.
The problem is that the potential liabilities arguably vastly exceed what these companies can absorb. It would be interesting to see if any major insurer wanted to step up and insure them.
On the plus side, in the worst case scenarios, we won't have to worry about damages because nobody will be left to pursue them.
And there is no need for rules, this doomerism probability is not based on science.
Any logic shows that this is really about their valuations and being tired of innovating instead of having any safety concerns. What would happen if the US did this? Chinese and EU labs would eventually win. And then either the US becomes a backwater or they have to eventually spend the money to keep up anyway. But in the meantime they will have bought themselves a few years at insane valuations.
Also, look at the stark choice they want to offer "we'll kill everyone or you give us the power to do whatever we want with the most important tech around". That's it. If we aren't in an oligarchy now, we will be if this goes through. They're going to end up stealing trillions of dollars from the rest of society.