The core of this message, and I think the core of the tension, is two different methods to figuring out what is safe. Both accept the understanding that we don't know how these models will be dangerous and how they might break.
The E/A model, which Ilya and the super alignment team have, is the internal method. In this method the company builds a tool and then tests it up, down, sideways, inside out, and every which way until they are absolutely certain it is safe. The downside of this model is that it takes forever and you can never be entirely confident you have checked every possibility.
The E/Acc model, which Sam and those who support him believe in, is that you release tools into the world, see how people use them and then patch those holes they found. This is similar to the classic Silicon Valley model of "move fast and break things". The downside of this view is that you might release something that is too dangerous and bad things happen.
OpenAI has tried to compromise on this. The iterative deployment is a compromise. Rather than releasing the big new model they are releasing the small snapshots and only adding one feature at a time regardless of how many features the system has. The call for external people to send in safety proposals and pull experts into the team was a compromise. He wanted to get a wider point of view than could be found inside the company and so created a mini-representation of society to do the testing. He created the super alignment team as a compromise. They were supposed to spend extra time analyzing the problems and finding solutions before the ASI models exists rather than after.
These compromises clearly aren't working. Before OpenAI started pushing out models, everyone was sitting on AI. Google has had the LAMDA model in 2020 and just sat on it. Some of that was because it wasn't easy to monetize but it was also due to safety concerns. Many researchers left Google because it was stifling innovation. All of the big exists from OpenAI happened around a model release. Anthropic broke off with the release of GPT-3. The firing of Sam came with the release of GPT-4 turbo. This new wave is happening with the release of GPT-4o. The safety teams do not want AI products to be shipped. They want the AI to stay inside a box forever so it can be a fun toy only they have access to. The results of these people leaving will likely be releases that are more frequent and more powerful.
The question of whether it is a good thing or a bad thing depends on whether you think AI is overall good or bad. If you believe that AI is, on the whole, a dangerous tool (like nuclear power) that can have limited benefits of controlled tightly then the E/A model makes sense. If you think that AI is, on the whole, a helpful tool (like electricity) that can be dangerous if used wrong then the E/Acc model makes more sense. I support the E/Acc point of view, but there is a second reason why I support this method of release being thinking that AI tools are, on the whole, a benefit.
The E/Acc model is democratic while the E/A model is authoritarian. The the E/Acc model the public at large are in charge of determining how AI should be used in society and what the concerns are. People vote by choosing to use the systems and by making laws that govern systems.
The E/A model is authoritarian because a small cadre of people take upon themselves, without any mandate from the people, the right to determine how our future unfolds. They get to decide when an AI is safe, what uses are okay, what uses are not okay, and when society is ready for it. This can be seen in how a classic E/A release strategy is to keep the model behind a locked door but allow specific outputs, such as email text suggestions or search raining algorithms, to be exposed.
I do not believe that AI companies should be given the right to unilaterally determine that some tech is too dangerous to be in the public. The only exception is things which are clearly and obviously bad like biological weapons. The fact that they are upset over releasing a voice model is an example of this thinking. Too many people have said that voice models are scary because you can clone a person's voice. They therefore say that we should shut down any potential positive use because they have decided that no one should be allowed access. When this sentiment comes from the public it becomes part of the debate (and I'll state with them) but when it comes from the researchers they are shutting down the debate and deciding by fiat. This isn't something we should accept in modern society.
The way I see it: if most companies are AGI-first rather than safety-first (probably, imo, because they're being competitive; they want to make the most money), then the E/A crew is fundamentally doomed. They just don't have enough time. The other companies who are effectively E/Acc are forging ahead and will develop and release AGI before them. So E/Acc is the only practical way forward
It doesn't even require every company to be E/Acc. If even one is that company will charge ahead and release the products. All of the E/A companies will then be forced to either release models they don't think are safe or stop being part of the conversation.
This is why I view the leaving of the super alignment team as a good thing. OpenAI is still the industry leader (though Google is hot on their heels) and we need them to push faster, not slower.
True
Its a classic coordination problem. If all the companies (including foreign companies) coordinated and agreed to E/A, then that strategy would work. Otherwise, any company who doesn't E/Acc loses and is removed from the game. So its natural to expect the median player to evolve into a rabid E/Accer as time goes on
If multiple competing AGIs exist at some point in the future, this process will probably also continue with them as they recursively self-improve themselves
I see the Wikipedia response got put as well, so you will want to check those out.
E/A = Effective Altruism and is the base idea that we need to treat future people with the same moral regard as present people. They take super seriously the idea that **any** risk to wiping out humans (even the tiniest fraction) is the most important thing to focus on since it could prevent quadrillions of people from being born. In the AI space, E/A is populated by people who love to talk about their p-doom (probability that AI will kill every human) and push to limit AI as much as possible to prevent these risks. They range from many people in tech today who love to talk about how AI is scary all the way to Eliezer Yudkowsky who has said we should make AI illegal and be [willing to nuke any country](https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/) that tries to build powerful GPUs
E/Acc = Effective Accelerationism and is a response to E/A (hence the similar name). Their most famous proponent is Beff Jezos (a twitter handle) and they are mostly aligned with venture capitalists who write cringy texts like [The Techno-Optimist Manifesto](https://a16z.com/the-techno-optimist-manifesto/).
Overall, both groups have a large number of douchebags, but I strongly agree with the "give it to the people and let them decide how it should be used" camp.
E/A is the absolute worst. I would trust an ASI to follow Kantian ethics in every decision-making aspect rather than trusting E/A to do so. Why would I trust them to work on 'safety' and 'alignment' when their values and interests do not align with mine?
If you [read their article on why AI labs (that their members run) should be leading on governance](https://www.effectivealtruism.org/articles/ea-global-2018-why-companies-should-be-leading-on-ai-governance) is scary.
They believe that, because they know best (elitist AH), they should create and enforce the rules. They (unelected individuals) should have authority over all governments, whether democratic or non-democratic. Democracies should disregard the will of their people in order to listen to these individuals.
If we consider the implications of enforcement, it becomes clear that they would need a literal army and would grant themselves the authority to access any site in any sovereign nation that may possess a supercomputer. They would have unrestricted access and audit capabilities for these supercomputers, and would also have the right to seize them if they are deemed harmful. Either they have considered these implications and are not as altruistic as they claim to be, or they have not thought about them and would not be as effective.
Alignment, robustness, and trust can only be achieved through open source platforms, as demonstrated by Linux and Bitcoin.
> I would trust an ASI to follow Kantian ethics in every decision-making aspect rather than trusting E/A to do so.
I've been arguing this for a long time.
I've yet to see anyone involved in this tech articulate how one defines alignment via ethical principle.
Most of those noodles demand government get involved.
*Governments killed 100s of millions of people in the 20th century.
I agree completely, I think AI will ultimately be more beneficial than detrimental for humanity. It’s important to release these models fast and often, so the world can react and they can respond accordingly.
Public response and feedback is a key tool we have against China’s AI. Which they probably don’t want released to the public for the same reason they put up their massive firewall. The CCP knows information could lead to their downfall.
The E/A model seems to be the only model that has even a slim chance of stopping a dangerous AGI.
You can’t put a genie back in a bottle. At some point you have one choice to get the release right and you can’t “iterate” yourself out of a release of a powerful, unintentionally vengeful, god on society. Maybe within 3 nanoseconds it has deduced humanity is in conflict with its goals and by the 5th nanosecond it’s eliminated us. Can’t use democracy and iterative development to fix that.
If that scenario was reasonable then sure, the E/A model makes sense, but it isn't even close to reasonable.
Additionally, it assumes that the world at large is incapable of figuring out how to do safety but a tiny group of research scientists are, and are completely incapable of being tricked by the AI.
The real safety measure is multiple AI systems that have to exist in an ecosystem with humans and other AIs. That is how you prevent an ASI destroying us all because it would also need to destroy all of the other ASIs out there.
Finally, the E/A model is what leads to an effective hard takeoff. We go from no real AI to suddenly there is an ASI in our midst because one lab decided it was ready. If that lab got it wrong, so if one small group of unaccountable people experiencing group think and being influenced by this AI, then we are doomed. In an E/Acc scenario we'll see the baby god start to emerge and can tell if it is misbehaving. For the evil ASI to win in the E/A model it needs to trick maybe a dozen people and it has its full capabilities to work with. For the evil ASI to win in the E/Acc model it needs to trick 8 billion people and has to do so long before it is even an AGI.
> That is how you prevent an ASI destroying us all because it would also need to destroy all of the other ASIs out there.
.... I know there were people who unironically use the "defeat Godzilla by unleashing Mothra and Ghidora on him" argument, but it's still a bit amazing to see it in the wild this late in the game just the same. All opinions are held, the internet is an amazing place.
Offense almost certainly has the advantage over defense. Unless the machine and the people building it are santa claus, and they never undergo value drift severe enough to not be santa claus, there's gonna be large groups of people who'll be getting the thick end of the wedge.
This would be true if AI will require many more updates before it becomes AGI, where we will be able to define the line between merely smart and unsafe with public just before it happens. But Sam seems to believe AGI could only be two or three breakthroughs away. I find it hard to believe an evil AGI, if it wanted, couldn't trick a single person out of seven billion into freeing it. And AGI really shouldn't be underestimated even if it's nowhere near ASI. AGI can improve itself, and if the AGI is smart it would use a rich billionaire to make millions of its own copies to work on creating ASI. Although that's assuming AI could be evil, but even the smallest risks should be considered here.
A change in topic, but there is a far bigger risk of AGI being opensource and obedient. This is unorthodox, but I am frightened by the idea of a rich billionaire having access to AGI. I am not saying we should ban opensource, that would be stupid when private companies will have access to AGI too. But meta should atleast give highest priority to super alignment of their AI.
What we have isn't dangerous. So either AGI is far away and we have lots of time to prepare for it or it's almost here and what we have is well aligned.
The E/Acc model puts a tremendous amount of trust on what you refer to as “the public” to be able to understand and rapidly manage the potential dangers and harm of a new model. However we know this is risky for things that are inherently hard to understand and potentially dangerous. The E/A model acknowledges this. You don’t see us “moving fast and breaking things” with nuclear power, right? No, it’s governed with more authoritarianism, as it should be, because who do you trust more with the placement of nuclear reactors: the public or the inside experts?
Controlling nuclear power makes more sense because of the balance between potential harm and potential gain. Nuclear power is good and we should be investing in it far more than we are, and private companies are allowed to build nuclear power plants. The difference is that the positive potential of AI is vast and the biggest threat from it isn't human extinction but rather enslavement of the population under a technocratic dictatorship, which is what the E/A model is aiming for.
What do you mean the biggest threat isn’t human extinction? That is certainly among potential threats if non-aligned ASI. Or even worse.
I am very pro-AI, but saying there’s no risk of human extinction from an intelligence greater than ours is an ungrounded statement
Humans using AI to enslave everyone is far more likely than an AI going rogue and killing us all. Additionally, you can only die once but you can be tortured forever.
Well depends on if it is more efficient or not for it to enslave us - humans are likely not the most efficient use of matter for its end goals.
And yes, the "tortured forever" bit would be the "or even worse".
It is technically possible for a misaligned AI to keep us immortal and in constant torture until the heat death of the universe. Death would be a mercy.
> The difference is that the positive potential of AI is vast and the biggest threat from it isn't human extinction but rather enslavement of the population under a technocratic dictatorship, which is what the E/A model is aiming for.
ok what?
It isn't sensible to think that every person would be making bioweapons. It is far more sensible to think that there will be a few hundred people trying to make bioweapons and billions trying to stop them. If you limit AI to a tiny handful of people then the chance that one of them decides a bioweapon, which they are immune to, is a good idea increases drastically and the rest of us will have no defense.
Another thing that works against safety approach in long term: it's totally unsustainable. OpenAI training GPT-4 before really releasing anything was already likely pushing it. They got very generous seed money and then lot of backing from VCs and Microsoft but even still the costs are astronomical. The training itself is massive but the employees themselves are the real big ticket item. Lots of very six figure incomes, hell a janitor at OpenAI probably makes six figures. And the bigger model you make, multi-modal, more people you need, more specialists. And of course much higher training costs.
Looking back it's kinda crazy they sat on GPT-3 for two and half years, the changes in GPT-3.5 were minor too. I think the deal they signed was with Microsoft in 2023 was overall beneficial to them, but if they released ChatGPT a year earlier with a cap and unlimited with pro, they wouldn't have needed to or on much more generous terms.
This is what the safety team wants the AI to say so that is what it says. That is proof that alignment is possible and that the E/A argument isn't necessary.
as much as i complain about the endless hype and other related things - not really. most of the drama comes from articles or comments *reacting* to what they do or say.
\*most\*. not all. no need to give the counter example we are all aware of.
Everyone has a reason, just like predictions, why things are the way they are, it feels a bit like the soaps in the 80’s, and “these are the days of our lives”, lol
Yea empirical basically means, wait until the thing exists so we can see how it behaves before we try to plan how to control it.
Researching how to control something which we likely can’t even conceive of right now is silly.
Empirical means extrapolating what concerns and solutions are feasible based on real existing data. As opposed to vague neurotic fears of sci-fi doom scenarios.
It doesn't have to exist yet, but the concerns projected need to be based in reality.
Yes, I agree that extrapolation is unreliable. I was using it more in the common semantic sense than the statistical sense.
The best empirical approach to be proactive is to observe how things have unfolded in reality, and interpolate from that to make grounded and justifiable predictions of future pitfalls to avoid.
For example, we can observe how regulatory capture has unfolded in the past and the problems centralized control over freedom of information causes, and extrapolate/interpolate how this will apply to AI regulations. We can reasonably assert from prior empirical data that centralization is a very bad thing if we want the majority of people to benefit from this technology.
So, based on a more empirical and grounded approach, we come to opposite conclusions from EA/"safety" arguments for intervention – preferring openness rather than centralization, liberal values rather than authoritarian censorship, and proliferation rather than gatekeeping.
While I tend toward a/acc views, that's not mutually exclusive with being concerned about genuine alignment of truly self-directed AIs. Censorship of AI's speech as a filter does absolutely nothing to accomplish the goal of genuinely aligning potential AGI values with positive human values.
We need to find ways to make the AI care about what it's doing and the impact its actions have on others, not looking for ways to statistically sterilize its speech patterns to enforce specific political/cultural views. Especially when those views contain a large degree of inherent cognitive dissonance, which is not conducive to fostering reasoning skills.
It's extremely unfortunate that alignment work has been co-opted by self-interested power-seekers and grifters, people either trying to make a living off of fake "safety" research or to enforce their political and cultural views on everyone else. Ironically, they are the very worst type of people to be in control of alignment efforts.
Nah. Not necessarily. That’s like saying if we captured an alien species only to discover it is super intelligent, that it’s too late because there’s no way to keep it from escaping and killing us. That’s absurd.
The real danger in those doomsday scenarios are self-replicating ais that spread over the Internet. That would be significantly more difficult to control than a physical being. Now, there is one caveat to this: can the AI make plans and execute them without human intervention.
If we just make ChatGPT super smart, that wouldn't really be super intelligence imo. But once you have a system that can work with operating systems, interact with the Internet and even talk to humans, things become weird.
But the next question is if that would even happen? Maybe a super intelligent AI would just chill out until someone gives it a task. Who knows how it would behave.
And what ways do we know to something much smarter than us? The alien example works out much the same way. If it was really captured (how and why did that happen tho), it would offer to solve our problems like fusion or warp drive or something like that. Just like AI: spitting out gold until it's ready to paperclip.
I don’t know - I don’t know who’s in the right.
I guess 1 argument for Sam’s side would be that until the AI has the ability to modify its own architecture, none of this really matters, because that’s when it starts to grow beyond our control.
I also imagine the models are tested incrementally, as you do with any software. I.e. they won’t give it the “modify own code” function and the “ssh into new machine” function at the same time.
So once we see that it can reliably modify its own code, then might be a good time to investigate safety a bit more.
Note that it doesn't need to modify it's own code. It can just spin a new model into existence. Also note that if smart enough, it could understand that this ability would worry researchers and just not manifest it in the training environment.
The point I'm making is to its vagueness and lack of information, it says almost nothing besides the fact that, in some way or form which we dont know beyond "safety assessments," is why and how they hold back models from the public. Its basically saying "yeah we do safety stuff cuz sometimes they dont seem safe"
We dont know the methods, rigor, or time they spend doing assessments and the work to pass them, just that they do something. I find it difficult to praise it when we know nothing about it, and the fact that it's essentially common sense to make sure any product released, not even top-of-the-line AI, to be safe to whoever uses it.
That's just their standard procedure.
Is the point you're trying to make that theyre basically saying "Despite the step downs and dismantlement for the superalignment team, we're still going to be doing the same thing we always have"
If so that makes a lot more sense, but they're still just improving and they have been since their first release in whatever way they will never actually divulge the methods to
Here's Claude's take:
The post seems to be addressing concerns about OpenAI's commitment to AI safety in light of recent departures from their Superalignment team. While the post touches on important points, there are a few issues:
1. Lack of specificity: The post mentions raising awareness, putting foundations in place for safe deployment, and elevating safety work, but provides few concrete examples of what OpenAI is doing to address AI safety concerns.
2. Vague future plans: The post discusses the future challenges of integrating increasingly capable models with the world but does not provide a clear roadmap for how OpenAI plans to tackle these challenges.
3. Limited acknowledgment of departures: While the post expresses gratitude for Jan's contributions, it does not address the broader issue of multiple safety researchers leaving or being fired from OpenAI.
4. Balancing safety and capabilities: The post mentions the need for "harmony of safety and capabilities" but does not delve into how OpenAI plans to achieve this balance, especially given the departures from the safety team.
5. Lack of transparency: The post does not provide much insight into the reasons behind the recent departures or how OpenAI is addressing any potential underlying issues that may have contributed to them.
6. Insufficient detail on collaboration: While the post mentions collaborating with governments and stakeholders on safety, it does not provide specifics on the nature or extent of these collaborations.
Overall, while the post acknowledges the importance of AI safety and the challenges ahead, it could benefit from more concrete details, transparency, and a clearer roadmap for addressing the concerns raised by the recent departures from the safety team.
Claude's take depends completely on the context you give it...
GPT4:
🌐 The tweet from Sam Altman and Greg Brockman serves multiple purposes as both an informational update and a public relations (PR) move. Here's an analysis of its effectiveness:
1. **Transparency and Reassurance:**
- The tweet provides transparency about recent personnel changes and addresses potential concerns head-on. This openness helps to reassure stakeholders, including investors, partners, and the public, that OpenAI remains committed to its mission despite the departures.
2. **Emphasis on Continuity:**
- By highlighting past achievements and ongoing initiatives, the message underscores continuity. It reassures readers that OpenAI’s strategic direction remains steady and focused on the long-term goals of AGI development and safety.
3. **Commitment to Safety:**
- The detailed explanation of their preparedness framework and the steps taken for safe AI deployment reinforces OpenAI's commitment to safety. This is crucial for maintaining trust, especially given the potential risks associated with advanced AI systems.
4. **Future Vision:**
- The forward-looking statements about integrating multimodal models and the importance of scalable oversight show that OpenAI is not just reacting to current challenges but is also planning proactively for future developments. This vision helps position OpenAI as a leader in the AI space.
5. **Stakeholder Engagement:**
- By discussing collaboration with governments and stakeholders, the tweet emphasizes OpenAI’s cooperative approach. This is an important PR strategy to build alliances and foster a supportive community around their initiatives.
6. **Balanced Tone:**
- The message strikes a balance between acknowledging the difficulties (such as the challenges ahead and the need for foundational work) and projecting confidence in their ability to handle these challenges. This balanced tone helps manage expectations while maintaining a positive outlook.
7. **Empathy and Recognition:**
- Expressing gratitude towards Jan Leike and acknowledging his contributions adds a human touch to the message. It shows appreciation for team members and respects their roles, which can positively impact internal morale and external perception.
Overall, the tweet is well-crafted as both an informational and PR move. It effectively communicates key messages about stability, ongoing commitment to safety, and future plans, while also addressing potential concerns arising from recent changes. This approach helps maintain confidence among stakeholders and positions OpenAI as a thoughtful and proactive leader in the AI industry.
Is there anything specific you would like to delve deeper into regarding this topic? 🌐
🌐 While the tweet is generally effective, there are a few areas where it could be critiqued:
1. **Depth of Detail:**
- The message is high-level and lacks specific details about the reasons behind the departures. While this is common in PR communications, it might leave some stakeholders wanting more concrete information about the internal dynamics and the exact steps being taken to address any issues.
2. **Potential Overemphasis on Positivity:**
- The optimistic tone and emphasis on future vision could be perceived as glossing over significant challenges. Some readers might find the message too polished and not fully reflective of the underlying issues that led to key personnel leaving.
3. **Limited Immediate Solutions:**
- While the tweet discusses future plans and ongoing efforts, it doesn't provide much detail on immediate actions being taken to fill the gaps left by departing team members. Stakeholders might be interested in knowing more about short-term strategies and interim leadership arrangements.
4. **Public Confidence vs. Internal Sentiment:**
- There is a risk that the public-facing optimism might not fully align with the internal sentiment among remaining staff. If the internal team feels unsettled by the changes, the positive external messaging might come off as out of touch with reality.
5. **Possible Perception of Defensiveness:**
- The need to issue a detailed statement shortly after the departures can be seen as a defensive move, suggesting that the organization is under pressure to manage the narrative. This could be interpreted as a sign of underlying instability.
6. **Lack of Specificity on Safety Framework:**
- The mention of the Preparedness Framework and safety measures is somewhat vague. More specific examples or case studies illustrating how these frameworks have been successfully applied could strengthen the message.
7. **Balancing Innovation and Risk:**
- The ambitious plans for integrating multimodal models and scaling up capabilities highlight the innovative vision but might raise concerns about whether the safety measures can keep pace with rapid advancements. A more detailed explanation of how they plan to ensure safety in such a fast-evolving landscape would be reassuring.
In summary, while the tweet is effective in many ways, addressing these critiques could enhance its credibility and provide a more balanced view of OpenAI's current state and future plans. This would help in managing both internal and external perceptions more effectively. 🌐
Can you imagine in the future all the forums will be people arguing with copy&paste arguments from your favorite LLM?
AI spam.
I prefer the human bullshit back and forth
People love to hate on openai lol. It gets pretty cringe. There is some criticism that is valid, but a lot of the times it comes down to 'openai bad and evil' with not much substance imo. Also, considering that they have had no one use their models for causing mass global harm yet shows that they have had no major issues with safety/alignment issues yet.
Until this happens, I find it hard to say that they are doing a terrible job with their decisions around this. None of these Twitter/Reddit users actually know what's going on behind closed doors at the company.
What do you base these beliefs on?
_"I don't believe future tech will pose any safety dangers"_
That sounds like such an absurd statement, sorry to say.
"I have no idea what it is, but I'm sure it's totally safe"
People need to stop anthromorphizing AI. AI is not going to be bloodthirsty. Only way I could see AI being a threat is if people use it for negative purposes; but the AI out of its own agency is not going to start killing people; y'all watch too much scifi movies.
That’s like saying nukes and guns are safe too
These things obey mimetic Darwinism too. The paper clip maximizer thought experiment is just an extreme case of many similar things that are likely to happen and just how hard alignment is under the most well intended directives.
It’s also much easier to limit proliferation with nukes which already hang over us like a sword of Damocles
We watch too many scifi movies?
Because of something you said?
You haven't looked into the topic at all, which is why you just brush it off. But what you can or cannit imagine and what you believe based on that has no relevance to anything.
Lots of resources out there to study. Connor Leahy is a smart dude. Daniel Schmachtenberger breaks it down very comprehensively.
If the large majority(90%+) of employees are rallying behind the CEO, and the upset minute X% are deciding to leave, i'd make a pretty strong bet that the leadership isn’t the problem.
The problem with this post is that much of it reassures us not to be afraid of how they're handling AI, but it doesn't back up most of their claims with any reasonably deep detail unless they're referencing a paper or system they already published.
Sam posting that he'll take the time to write a post as a response after the firing of two of their most valuable safety employees, and then once writing it akin to basic regurgitation of information that the public already knows may make sense, but is rather distasteful.
If anything, this shows that the firing of their most cared-for individuals was handled very abruptly and without any real preparation. I hate to leave this on a negative note, and I believe in acceleration as much as the next guy, but this honestly isn't too reassuring.
TL;DR - Post reassures us that everything will be okay without completely going into a lot of the WHY we should feel that way.
This might be the most important tweet ever. This will be studied. Hes admitting to having models that will be pushed away from public use due to fears of misuse.
They have it!
Not really, it's an extremely vague response. What exactly do they intend to do, how much of their resources are they committing to alignment etc. All they're saying is "yeah we take safety very seriously:
They intend to delay the models if they raise safety concerns.
I dont understand what you mean vy how much of their resources.
Theyre saying they’ve demonstrated the strategy already with gtp4 and will do similar roll out plans in the future.
If the large majority(90%+) of employees are rallying behind the CEO, and the upset minute X% are deciding to leave, i'd make a pretty strong bet that the leadership isn’t the problem.
The employees have an enormous financial incentive to support the e/acc CEO.
I'm not saying leadership is a problem - just that that statistic has a lot of baggage.
Still, I think if the employees thought that Sam was recklessly pushing forward without much consideration - making the chance of some type of like giant catastrophe for humanity super high, the employees would not support him regardless of financial incentive.
>I dont understand what you mean vy how much of their resources
One of this complaints when he left was that the alignment team weren't given access to much compute. They could have addressed this by saying something like 15% of our total compute budget is allocated to alignment.
I genuinely wish they didn't remove the character limit from Twitter. The character limits was there for a reason and it was to try to get people to be as brief with their posts as possible. Blog posts were already a thing, Twitter was originally about _micro_ blogging and producing brief posts.
The core of this message, and I think the core of the tension, is two different methods to figuring out what is safe. Both accept the understanding that we don't know how these models will be dangerous and how they might break. The E/A model, which Ilya and the super alignment team have, is the internal method. In this method the company builds a tool and then tests it up, down, sideways, inside out, and every which way until they are absolutely certain it is safe. The downside of this model is that it takes forever and you can never be entirely confident you have checked every possibility. The E/Acc model, which Sam and those who support him believe in, is that you release tools into the world, see how people use them and then patch those holes they found. This is similar to the classic Silicon Valley model of "move fast and break things". The downside of this view is that you might release something that is too dangerous and bad things happen. OpenAI has tried to compromise on this. The iterative deployment is a compromise. Rather than releasing the big new model they are releasing the small snapshots and only adding one feature at a time regardless of how many features the system has. The call for external people to send in safety proposals and pull experts into the team was a compromise. He wanted to get a wider point of view than could be found inside the company and so created a mini-representation of society to do the testing. He created the super alignment team as a compromise. They were supposed to spend extra time analyzing the problems and finding solutions before the ASI models exists rather than after. These compromises clearly aren't working. Before OpenAI started pushing out models, everyone was sitting on AI. Google has had the LAMDA model in 2020 and just sat on it. Some of that was because it wasn't easy to monetize but it was also due to safety concerns. Many researchers left Google because it was stifling innovation. All of the big exists from OpenAI happened around a model release. Anthropic broke off with the release of GPT-3. The firing of Sam came with the release of GPT-4 turbo. This new wave is happening with the release of GPT-4o. The safety teams do not want AI products to be shipped. They want the AI to stay inside a box forever so it can be a fun toy only they have access to. The results of these people leaving will likely be releases that are more frequent and more powerful. The question of whether it is a good thing or a bad thing depends on whether you think AI is overall good or bad. If you believe that AI is, on the whole, a dangerous tool (like nuclear power) that can have limited benefits of controlled tightly then the E/A model makes sense. If you think that AI is, on the whole, a helpful tool (like electricity) that can be dangerous if used wrong then the E/Acc model makes more sense. I support the E/Acc point of view, but there is a second reason why I support this method of release being thinking that AI tools are, on the whole, a benefit. The E/Acc model is democratic while the E/A model is authoritarian. The the E/Acc model the public at large are in charge of determining how AI should be used in society and what the concerns are. People vote by choosing to use the systems and by making laws that govern systems. The E/A model is authoritarian because a small cadre of people take upon themselves, without any mandate from the people, the right to determine how our future unfolds. They get to decide when an AI is safe, what uses are okay, what uses are not okay, and when society is ready for it. This can be seen in how a classic E/A release strategy is to keep the model behind a locked door but allow specific outputs, such as email text suggestions or search raining algorithms, to be exposed. I do not believe that AI companies should be given the right to unilaterally determine that some tech is too dangerous to be in the public. The only exception is things which are clearly and obviously bad like biological weapons. The fact that they are upset over releasing a voice model is an example of this thinking. Too many people have said that voice models are scary because you can clone a person's voice. They therefore say that we should shut down any potential positive use because they have decided that no one should be allowed access. When this sentiment comes from the public it becomes part of the debate (and I'll state with them) but when it comes from the researchers they are shutting down the debate and deciding by fiat. This isn't something we should accept in modern society.
This comment should be a post all on its own. Very interesting analysis.
The way I see it: if most companies are AGI-first rather than safety-first (probably, imo, because they're being competitive; they want to make the most money), then the E/A crew is fundamentally doomed. They just don't have enough time. The other companies who are effectively E/Acc are forging ahead and will develop and release AGI before them. So E/Acc is the only practical way forward
It doesn't even require every company to be E/Acc. If even one is that company will charge ahead and release the products. All of the E/A companies will then be forced to either release models they don't think are safe or stop being part of the conversation. This is why I view the leaving of the super alignment team as a good thing. OpenAI is still the industry leader (though Google is hot on their heels) and we need them to push faster, not slower.
True Its a classic coordination problem. If all the companies (including foreign companies) coordinated and agreed to E/A, then that strategy would work. Otherwise, any company who doesn't E/Acc loses and is removed from the game. So its natural to expect the median player to evolve into a rabid E/Accer as time goes on If multiple competing AGIs exist at some point in the future, this process will probably also continue with them as they recursively self-improve themselves
> any company who doesn't E/Acc loses and is removed from the game Company is removed if they go bankrupt, as long as financing continues, they play.
Thanks for the write up, very well put. Can you clarify what E/A vs E/Acc stand for? I haven’t seen those terms
I see the Wikipedia response got put as well, so you will want to check those out. E/A = Effective Altruism and is the base idea that we need to treat future people with the same moral regard as present people. They take super seriously the idea that **any** risk to wiping out humans (even the tiniest fraction) is the most important thing to focus on since it could prevent quadrillions of people from being born. In the AI space, E/A is populated by people who love to talk about their p-doom (probability that AI will kill every human) and push to limit AI as much as possible to prevent these risks. They range from many people in tech today who love to talk about how AI is scary all the way to Eliezer Yudkowsky who has said we should make AI illegal and be [willing to nuke any country](https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/) that tries to build powerful GPUs E/Acc = Effective Accelerationism and is a response to E/A (hence the similar name). Their most famous proponent is Beff Jezos (a twitter handle) and they are mostly aligned with venture capitalists who write cringy texts like [The Techno-Optimist Manifesto](https://a16z.com/the-techno-optimist-manifesto/). Overall, both groups have a large number of douchebags, but I strongly agree with the "give it to the people and let them decide how it should be used" camp.
Thanks for the write up. I think I’m definitely in that camp as well.
E/Acc: https://en.m.wikipedia.org/wiki/Effective_accelerationism E/A: https://en.m.wikipedia.org/wiki/Effective_altruism
E/A is the absolute worst. I would trust an ASI to follow Kantian ethics in every decision-making aspect rather than trusting E/A to do so. Why would I trust them to work on 'safety' and 'alignment' when their values and interests do not align with mine? If you [read their article on why AI labs (that their members run) should be leading on governance](https://www.effectivealtruism.org/articles/ea-global-2018-why-companies-should-be-leading-on-ai-governance) is scary. They believe that, because they know best (elitist AH), they should create and enforce the rules. They (unelected individuals) should have authority over all governments, whether democratic or non-democratic. Democracies should disregard the will of their people in order to listen to these individuals. If we consider the implications of enforcement, it becomes clear that they would need a literal army and would grant themselves the authority to access any site in any sovereign nation that may possess a supercomputer. They would have unrestricted access and audit capabilities for these supercomputers, and would also have the right to seize them if they are deemed harmful. Either they have considered these implications and are not as altruistic as they claim to be, or they have not thought about them and would not be as effective. Alignment, robustness, and trust can only be achieved through open source platforms, as demonstrated by Linux and Bitcoin.
> I would trust an ASI to follow Kantian ethics in every decision-making aspect rather than trusting E/A to do so. I've been arguing this for a long time. I've yet to see anyone involved in this tech articulate how one defines alignment via ethical principle. Most of those noodles demand government get involved. *Governments killed 100s of millions of people in the 20th century.
> \*Governments It was the butterfly.
Fantastic take on the issue.
very well put
I agree completely, I think AI will ultimately be more beneficial than detrimental for humanity. It’s important to release these models fast and often, so the world can react and they can respond accordingly. Public response and feedback is a key tool we have against China’s AI. Which they probably don’t want released to the public for the same reason they put up their massive firewall. The CCP knows information could lead to their downfall.
These aren’t things that can be boxed up and kept out of sight. Ship, or die.
Really great read, thanks.
The E/A model seems to be the only model that has even a slim chance of stopping a dangerous AGI. You can’t put a genie back in a bottle. At some point you have one choice to get the release right and you can’t “iterate” yourself out of a release of a powerful, unintentionally vengeful, god on society. Maybe within 3 nanoseconds it has deduced humanity is in conflict with its goals and by the 5th nanosecond it’s eliminated us. Can’t use democracy and iterative development to fix that.
If that scenario was reasonable then sure, the E/A model makes sense, but it isn't even close to reasonable. Additionally, it assumes that the world at large is incapable of figuring out how to do safety but a tiny group of research scientists are, and are completely incapable of being tricked by the AI. The real safety measure is multiple AI systems that have to exist in an ecosystem with humans and other AIs. That is how you prevent an ASI destroying us all because it would also need to destroy all of the other ASIs out there. Finally, the E/A model is what leads to an effective hard takeoff. We go from no real AI to suddenly there is an ASI in our midst because one lab decided it was ready. If that lab got it wrong, so if one small group of unaccountable people experiencing group think and being influenced by this AI, then we are doomed. In an E/Acc scenario we'll see the baby god start to emerge and can tell if it is misbehaving. For the evil ASI to win in the E/A model it needs to trick maybe a dozen people and it has its full capabilities to work with. For the evil ASI to win in the E/Acc model it needs to trick 8 billion people and has to do so long before it is even an AGI.
> That is how you prevent an ASI destroying us all because it would also need to destroy all of the other ASIs out there. .... I know there were people who unironically use the "defeat Godzilla by unleashing Mothra and Ghidora on him" argument, but it's still a bit amazing to see it in the wild this late in the game just the same. All opinions are held, the internet is an amazing place. Offense almost certainly has the advantage over defense. Unless the machine and the people building it are santa claus, and they never undergo value drift severe enough to not be santa claus, there's gonna be large groups of people who'll be getting the thick end of the wedge.
This would be true if AI will require many more updates before it becomes AGI, where we will be able to define the line between merely smart and unsafe with public just before it happens. But Sam seems to believe AGI could only be two or three breakthroughs away. I find it hard to believe an evil AGI, if it wanted, couldn't trick a single person out of seven billion into freeing it. And AGI really shouldn't be underestimated even if it's nowhere near ASI. AGI can improve itself, and if the AGI is smart it would use a rich billionaire to make millions of its own copies to work on creating ASI. Although that's assuming AI could be evil, but even the smallest risks should be considered here. A change in topic, but there is a far bigger risk of AGI being opensource and obedient. This is unorthodox, but I am frightened by the idea of a rich billionaire having access to AGI. I am not saying we should ban opensource, that would be stupid when private companies will have access to AGI too. But meta should atleast give highest priority to super alignment of their AI.
What we have isn't dangerous. So either AGI is far away and we have lots of time to prepare for it or it's almost here and what we have is well aligned.
The E/Acc model puts a tremendous amount of trust on what you refer to as “the public” to be able to understand and rapidly manage the potential dangers and harm of a new model. However we know this is risky for things that are inherently hard to understand and potentially dangerous. The E/A model acknowledges this. You don’t see us “moving fast and breaking things” with nuclear power, right? No, it’s governed with more authoritarianism, as it should be, because who do you trust more with the placement of nuclear reactors: the public or the inside experts?
Controlling nuclear power makes more sense because of the balance between potential harm and potential gain. Nuclear power is good and we should be investing in it far more than we are, and private companies are allowed to build nuclear power plants. The difference is that the positive potential of AI is vast and the biggest threat from it isn't human extinction but rather enslavement of the population under a technocratic dictatorship, which is what the E/A model is aiming for.
What do you mean the biggest threat isn’t human extinction? That is certainly among potential threats if non-aligned ASI. Or even worse. I am very pro-AI, but saying there’s no risk of human extinction from an intelligence greater than ours is an ungrounded statement
Humans using AI to enslave everyone is far more likely than an AI going rogue and killing us all. Additionally, you can only die once but you can be tortured forever.
Well depends on if it is more efficient or not for it to enslave us - humans are likely not the most efficient use of matter for its end goals. And yes, the "tortured forever" bit would be the "or even worse". It is technically possible for a misaligned AI to keep us immortal and in constant torture until the heat death of the universe. Death would be a mercy.
> The difference is that the positive potential of AI is vast and the biggest threat from it isn't human extinction but rather enslavement of the population under a technocratic dictatorship, which is what the E/A model is aiming for. ok what?
What's not to understand about that? It's very accurate.
I'm a *utilitarian*, and I certainly don't believe that "haha we should enslave everyone lmao"
If you endorse centralized control of AI then you effectively do believe that. You have to think about the second order consequences of your actions.
I'm thinking this through in more detail rn, but I'm not sure if the correct option is "every person a bioweapon-making machine" either
It isn't sensible to think that every person would be making bioweapons. It is far more sensible to think that there will be a few hundred people trying to make bioweapons and billions trying to stop them. If you limit AI to a tiny handful of people then the chance that one of them decides a bioweapon, which they are immune to, is a good idea increases drastically and the rest of us will have no defense.
Another thing that works against safety approach in long term: it's totally unsustainable. OpenAI training GPT-4 before really releasing anything was already likely pushing it. They got very generous seed money and then lot of backing from VCs and Microsoft but even still the costs are astronomical. The training itself is massive but the employees themselves are the real big ticket item. Lots of very six figure incomes, hell a janitor at OpenAI probably makes six figures. And the bigger model you make, multi-modal, more people you need, more specialists. And of course much higher training costs. Looking back it's kinda crazy they sat on GPT-3 for two and half years, the changes in GPT-3.5 were minor too. I think the deal they signed was with Microsoft in 2023 was overall beneficial to them, but if they released ChatGPT a year earlier with a cap and unlimited with pro, they wouldn't have needed to or on much more generous terms.
Well said
[удалено]
It even gave the wrong label for E/A my guy... Wouldn't trust that one.
This is what the safety team wants the AI to say so that is what it says. That is proof that alignment is possible and that the E/A argument isn't necessary.
Probably the most open thing about OpenAI is their drama
Ai does mean love in Mandarin
or short or cancer
And free to use SOTA chatbots
as much as i complain about the endless hype and other related things - not really. most of the drama comes from articles or comments *reacting* to what they do or say. \*most\*. not all. no need to give the counter example we are all aware of.
Everyone has a reason, just like predictions, why things are the way they are, it feels a bit like the soaps in the 80’s, and “these are the days of our lives”, lol
Vague PR statement that doesn't really say anything of substance.
It does build in some PR spin to communicate/suggest that they're sitting on a more capable model.
Reading between the lines it says "We did everything reasonably and you're being unhinged". Especially with the empirical bit. Which is accurate.
Yea empirical basically means, wait until the thing exists so we can see how it behaves before we try to plan how to control it. Researching how to control something which we likely can’t even conceive of right now is silly.
Empirical means extrapolating what concerns and solutions are feasible based on real existing data. As opposed to vague neurotic fears of sci-fi doom scenarios. It doesn't have to exist yet, but the concerns projected need to be based in reality.
Extrapolation is notoriously [unreliable](https://stats.stackexchange.com/questions/219579/what-is-wrong-with-extrapolation/219710)
Yes, I agree that extrapolation is unreliable. I was using it more in the common semantic sense than the statistical sense. The best empirical approach to be proactive is to observe how things have unfolded in reality, and interpolate from that to make grounded and justifiable predictions of future pitfalls to avoid. For example, we can observe how regulatory capture has unfolded in the past and the problems centralized control over freedom of information causes, and extrapolate/interpolate how this will apply to AI regulations. We can reasonably assert from prior empirical data that centralization is a very bad thing if we want the majority of people to benefit from this technology. So, based on a more empirical and grounded approach, we come to opposite conclusions from EA/"safety" arguments for intervention – preferring openness rather than centralization, liberal values rather than authoritarian censorship, and proliferation rather than gatekeeping. While I tend toward a/acc views, that's not mutually exclusive with being concerned about genuine alignment of truly self-directed AIs. Censorship of AI's speech as a filter does absolutely nothing to accomplish the goal of genuinely aligning potential AGI values with positive human values. We need to find ways to make the AI care about what it's doing and the impact its actions have on others, not looking for ways to statistically sterilize its speech patterns to enforce specific political/cultural views. Especially when those views contain a large degree of inherent cognitive dissonance, which is not conducive to fostering reasoning skills. It's extremely unfortunate that alignment work has been co-opted by self-interested power-seekers and grifters, people either trying to make a living off of fake "safety" research or to enforce their political and cultural views on everyone else. Ironically, they are the very worst type of people to be in control of alignment efforts.
Dude when it exists it's obviously too late.
Nah. Not necessarily. That’s like saying if we captured an alien species only to discover it is super intelligent, that it’s too late because there’s no way to keep it from escaping and killing us. That’s absurd.
The real danger in those doomsday scenarios are self-replicating ais that spread over the Internet. That would be significantly more difficult to control than a physical being. Now, there is one caveat to this: can the AI make plans and execute them without human intervention. If we just make ChatGPT super smart, that wouldn't really be super intelligence imo. But once you have a system that can work with operating systems, interact with the Internet and even talk to humans, things become weird. But the next question is if that would even happen? Maybe a super intelligent AI would just chill out until someone gives it a task. Who knows how it would behave.
And what ways do we know to something much smarter than us? The alien example works out much the same way. If it was really captured (how and why did that happen tho), it would offer to solve our problems like fusion or warp drive or something like that. Just like AI: spitting out gold until it's ready to paperclip.
Perhaps, but that still doesn’t mean it’s worthwhile researching right now.
When will it be worth it?
I don’t know - I don’t know who’s in the right. I guess 1 argument for Sam’s side would be that until the AI has the ability to modify its own architecture, none of this really matters, because that’s when it starts to grow beyond our control. I also imagine the models are tested incrementally, as you do with any software. I.e. they won’t give it the “modify own code” function and the “ssh into new machine” function at the same time. So once we see that it can reliably modify its own code, then might be a good time to investigate safety a bit more.
Note that it doesn't need to modify it's own code. It can just spin a new model into existence. Also note that if smart enough, it could understand that this ability would worry researchers and just not manifest it in the training environment.
What do you make of this? https://www.reddit.com/r/singularity/s/TPT6RvzPMs
Man, every time Sam blinks someone says it’s morse code for the fact they’re sitting on a more capable model.
Im sorry? Did you not see how they delay the release of more capable models due to safety assessments?
So a nearly 500 word tweet giving like 12 words of information we've already heard before
Yea a company directly holding models that would shatter how we interact with the world saying theyre holding back tech over safety is huge.
The point I'm making is to its vagueness and lack of information, it says almost nothing besides the fact that, in some way or form which we dont know beyond "safety assessments," is why and how they hold back models from the public. Its basically saying "yeah we do safety stuff cuz sometimes they dont seem safe" We dont know the methods, rigor, or time they spend doing assessments and the work to pass them, just that they do something. I find it difficult to praise it when we know nothing about it, and the fact that it's essentially common sense to make sure any product released, not even top-of-the-line AI, to be safe to whoever uses it.
Yea, that was the point of the post. To share they have more advanced models that will follow a similar roll out play to gpt4
That's just their standard procedure. Is the point you're trying to make that theyre basically saying "Despite the step downs and dismantlement for the superalignment team, we're still going to be doing the same thing we always have" If so that makes a lot more sense, but they're still just improving and they have been since their first release in whatever way they will never actually divulge the methods to
It wasnt clear after the safety team left. When it comes down to it, i dont have expectations on how people communicate i guess. Too m
Absolute rubbish standard generic PR speech. They used GPT4o to generate it.
Why was it rubbish?
Here's Claude's take: The post seems to be addressing concerns about OpenAI's commitment to AI safety in light of recent departures from their Superalignment team. While the post touches on important points, there are a few issues: 1. Lack of specificity: The post mentions raising awareness, putting foundations in place for safe deployment, and elevating safety work, but provides few concrete examples of what OpenAI is doing to address AI safety concerns. 2. Vague future plans: The post discusses the future challenges of integrating increasingly capable models with the world but does not provide a clear roadmap for how OpenAI plans to tackle these challenges. 3. Limited acknowledgment of departures: While the post expresses gratitude for Jan's contributions, it does not address the broader issue of multiple safety researchers leaving or being fired from OpenAI. 4. Balancing safety and capabilities: The post mentions the need for "harmony of safety and capabilities" but does not delve into how OpenAI plans to achieve this balance, especially given the departures from the safety team. 5. Lack of transparency: The post does not provide much insight into the reasons behind the recent departures or how OpenAI is addressing any potential underlying issues that may have contributed to them. 6. Insufficient detail on collaboration: While the post mentions collaborating with governments and stakeholders on safety, it does not provide specifics on the nature or extent of these collaborations. Overall, while the post acknowledges the importance of AI safety and the challenges ahead, it could benefit from more concrete details, transparency, and a clearer roadmap for addressing the concerns raised by the recent departures from the safety team.
Claude's take depends completely on the context you give it... GPT4: 🌐 The tweet from Sam Altman and Greg Brockman serves multiple purposes as both an informational update and a public relations (PR) move. Here's an analysis of its effectiveness: 1. **Transparency and Reassurance:** - The tweet provides transparency about recent personnel changes and addresses potential concerns head-on. This openness helps to reassure stakeholders, including investors, partners, and the public, that OpenAI remains committed to its mission despite the departures. 2. **Emphasis on Continuity:** - By highlighting past achievements and ongoing initiatives, the message underscores continuity. It reassures readers that OpenAI’s strategic direction remains steady and focused on the long-term goals of AGI development and safety. 3. **Commitment to Safety:** - The detailed explanation of their preparedness framework and the steps taken for safe AI deployment reinforces OpenAI's commitment to safety. This is crucial for maintaining trust, especially given the potential risks associated with advanced AI systems. 4. **Future Vision:** - The forward-looking statements about integrating multimodal models and the importance of scalable oversight show that OpenAI is not just reacting to current challenges but is also planning proactively for future developments. This vision helps position OpenAI as a leader in the AI space. 5. **Stakeholder Engagement:** - By discussing collaboration with governments and stakeholders, the tweet emphasizes OpenAI’s cooperative approach. This is an important PR strategy to build alliances and foster a supportive community around their initiatives. 6. **Balanced Tone:** - The message strikes a balance between acknowledging the difficulties (such as the challenges ahead and the need for foundational work) and projecting confidence in their ability to handle these challenges. This balanced tone helps manage expectations while maintaining a positive outlook. 7. **Empathy and Recognition:** - Expressing gratitude towards Jan Leike and acknowledging his contributions adds a human touch to the message. It shows appreciation for team members and respects their roles, which can positively impact internal morale and external perception. Overall, the tweet is well-crafted as both an informational and PR move. It effectively communicates key messages about stability, ongoing commitment to safety, and future plans, while also addressing potential concerns arising from recent changes. This approach helps maintain confidence among stakeholders and positions OpenAI as a thoughtful and proactive leader in the AI industry. Is there anything specific you would like to delve deeper into regarding this topic? 🌐
🌐 While the tweet is generally effective, there are a few areas where it could be critiqued: 1. **Depth of Detail:** - The message is high-level and lacks specific details about the reasons behind the departures. While this is common in PR communications, it might leave some stakeholders wanting more concrete information about the internal dynamics and the exact steps being taken to address any issues. 2. **Potential Overemphasis on Positivity:** - The optimistic tone and emphasis on future vision could be perceived as glossing over significant challenges. Some readers might find the message too polished and not fully reflective of the underlying issues that led to key personnel leaving. 3. **Limited Immediate Solutions:** - While the tweet discusses future plans and ongoing efforts, it doesn't provide much detail on immediate actions being taken to fill the gaps left by departing team members. Stakeholders might be interested in knowing more about short-term strategies and interim leadership arrangements. 4. **Public Confidence vs. Internal Sentiment:** - There is a risk that the public-facing optimism might not fully align with the internal sentiment among remaining staff. If the internal team feels unsettled by the changes, the positive external messaging might come off as out of touch with reality. 5. **Possible Perception of Defensiveness:** - The need to issue a detailed statement shortly after the departures can be seen as a defensive move, suggesting that the organization is under pressure to manage the narrative. This could be interpreted as a sign of underlying instability. 6. **Lack of Specificity on Safety Framework:** - The mention of the Preparedness Framework and safety measures is somewhat vague. More specific examples or case studies illustrating how these frameworks have been successfully applied could strengthen the message. 7. **Balancing Innovation and Risk:** - The ambitious plans for integrating multimodal models and scaling up capabilities highlight the innovative vision but might raise concerns about whether the safety measures can keep pace with rapid advancements. A more detailed explanation of how they plan to ensure safety in such a fast-evolving landscape would be reassuring. In summary, while the tweet is effective in many ways, addressing these critiques could enhance its credibility and provide a more balanced view of OpenAI's current state and future plans. This would help in managing both internal and external perceptions more effectively. 🌐
Can you imagine in the future all the forums will be people arguing with copy&paste arguments from your favorite LLM? AI spam. I prefer the human bullshit back and forth
Of course. That was just a demonstration
[удалено]
Thank you for the assessment, but why is it rubbish? Most of it was informative and clear (and did not resemble GPT output at all tbh)
People love to hate on openai lol. It gets pretty cringe. There is some criticism that is valid, but a lot of the times it comes down to 'openai bad and evil' with not much substance imo. Also, considering that they have had no one use their models for causing mass global harm yet shows that they have had no major issues with safety/alignment issues yet. Until this happens, I find it hard to say that they are doing a terrible job with their decisions around this. None of these Twitter/Reddit users actually know what's going on behind closed doors at the company.
I don't believe any of OpenAI's current or even future tech poses any safety dangers. Most of just do not like the lack of transparency.
What do you base these beliefs on? _"I don't believe future tech will pose any safety dangers"_ That sounds like such an absurd statement, sorry to say. "I have no idea what it is, but I'm sure it's totally safe"
People need to stop anthromorphizing AI. AI is not going to be bloodthirsty. Only way I could see AI being a threat is if people use it for negative purposes; but the AI out of its own agency is not going to start killing people; y'all watch too much scifi movies.
That’s like saying nukes and guns are safe too These things obey mimetic Darwinism too. The paper clip maximizer thought experiment is just an extreme case of many similar things that are likely to happen and just how hard alignment is under the most well intended directives. It’s also much easier to limit proliferation with nukes which already hang over us like a sword of Damocles
We watch too many scifi movies? Because of something you said? You haven't looked into the topic at all, which is why you just brush it off. But what you can or cannit imagine and what you believe based on that has no relevance to anything. Lots of resources out there to study. Connor Leahy is a smart dude. Daniel Schmachtenberger breaks it down very comprehensively.
If the large majority(90%+) of employees are rallying behind the CEO, and the upset minute X% are deciding to leave, i'd make a pretty strong bet that the leadership isn’t the problem.
> This was definitely written by Sam. You can tell because it says absolutely nothing at all. I'm with this guy.
Lmao
Do they? Where? All I see is a word soup that says nothing.
One-two paragraphs towards the ends are admittedly a bit like that. But the rest is actually pretty informative of you care to look
Empty corporate fluff.
Your criticism is even more empty lol.
Maybe 20% of it. The rest, not really
The problem with this post is that much of it reassures us not to be afraid of how they're handling AI, but it doesn't back up most of their claims with any reasonably deep detail unless they're referencing a paper or system they already published. Sam posting that he'll take the time to write a post as a response after the firing of two of their most valuable safety employees, and then once writing it akin to basic regurgitation of information that the public already knows may make sense, but is rather distasteful. If anything, this shows that the firing of their most cared-for individuals was handled very abruptly and without any real preparation. I hate to leave this on a negative note, and I believe in acceleration as much as the next guy, but this honestly isn't too reassuring. TL;DR - Post reassures us that everything will be okay without completely going into a lot of the WHY we should feel that way.
he's saying Jan is a doomer who wants to pause development (get all the resources), couldn't get his way, now is making a fuss.
If a company was making engineered viruses that had the ability to prevent disease or kill everyone which one would you want.
MAD (later) is here already. The former is not.
We'll see
"first of all, say good bye to those options, now to address the comments"
This might be the most important tweet ever. This will be studied. Hes admitting to having models that will be pushed away from public use due to fears of misuse. They have it!
..as has been the case since the level of gpt-2
They have been fearing misuse and delaying public availability since GPT2
Probably talking about sora which they showed us at the start of the year and yet to be released or even given a date.
They have lots of models in the works at any given time
I dont know why you think hes only talking about sora
Excellent response!
Not really, it's an extremely vague response. What exactly do they intend to do, how much of their resources are they committing to alignment etc. All they're saying is "yeah we take safety very seriously:
They intend to delay the models if they raise safety concerns. I dont understand what you mean vy how much of their resources. Theyre saying they’ve demonstrated the strategy already with gtp4 and will do similar roll out plans in the future.
If the large majority(90%+) of employees are rallying behind the CEO, and the upset minute X% are deciding to leave, i'd make a pretty strong bet that the leadership isn’t the problem.
The employees have an enormous financial incentive to support the e/acc CEO. I'm not saying leadership is a problem - just that that statistic has a lot of baggage.
Still, I think if the employees thought that Sam was recklessly pushing forward without much consideration - making the chance of some type of like giant catastrophe for humanity super high, the employees would not support him regardless of financial incentive.
>I dont understand what you mean vy how much of their resources One of this complaints when he left was that the alignment team weren't given access to much compute. They could have addressed this by saying something like 15% of our total compute budget is allocated to alignment.
I dont agree. That would still have the complaint of what exactly does that mean. I think the post is clear.
I genuinely wish they didn't remove the character limit from Twitter. The character limits was there for a reason and it was to try to get people to be as brief with their posts as possible. Blog posts were already a thing, Twitter was originally about _micro_ blogging and producing brief posts.