A group of former OpenAI employees, backed by prominent figures including three Nobel Prize winners and AI experts like Geoffrey Hinton, are urging the Attorneys General of California and Delaware to intervene in OpenAI's planned for-profit restructuring. In a letter to California Attorney General Rob Bonta and Delaware Attorney General Kathy Jennings, the coalition is asking them to protect OpenAI's charitable purpose and block the restructuring, expressing concerns that the move could compromise the company's commitment to developing AI for the benefit of humanity.
Their primary worry stems from the potential shift in control over Artificial General Intelligence (AGI) – AI that matches or exceeds human intelligence – from a non-profit with a public duty to a for-profit entity potentially driven by commercial interests. They argue that this shift could lead to prioritizing profits over safety and ethical considerations in AI development and deployment. Page Hedley, a former policy and ethics advisor at OpenAI, voiced concerns about who will ultimately own and control the technology if it surpasses human capabilities. Nisan Stiennon, an AI engineer who worked at OpenAI from 2018 to 2020, stated that OpenAI might develop technology that could be dangerous and it is important that it remains controlled by a nonprofit with a duty to humanity.
OpenAI, which was founded in 2015 with the goal of ensuring AI benefits humanity, has evolved over the years, facing the challenges of funding its expensive research. While it has a for-profit subsidiary, the proposed restructuring aims to convert its core governance structure, leading to scrutiny and legal challenges. Elon Musk, who co-founded OpenAI but later left to form his own AI company, xAI, has sued OpenAI, accusing it and CEO Sam Altman of betraying the founding principles. Some signatories of the letter support Musk's lawsuit.
OpenAI has responded by stating that any changes to its structure would ensure the broader public benefits from AI. The company has also stated that its for-profit arm will become a public benefit corporation (PBC), similar to other AI labs like Anthropic and xAI, while preserving a non-profit arm. PBCs are legally required to consider stakeholder interests beyond shareholders, balancing profit and social purpose.
However, critics argue that the PBC structure may not provide sufficient protection against the potential risks of advanced AI and that stronger guardrails are needed. They are concerned that the new structure weakens the authority of the non-profit, potentially allowing investor interests to take precedence over OpenAI's original mission. As the "Not For Private Gain" group stated in their letter to the attorneys general, the revised plan might not prevent OpenAI from straying from its mission to ensure AI is developed for the benefit of humanity.
The Attorneys General of California and Delaware are now under pressure to investigate OpenAI's proposed governance structure and determine whether it can truly and independently carry out its original charitable mission. California Attorney General Rob Bonta's office had previously launched a probe into OpenAI's business plans, requesting financial records from the company. Delaware Attorney General Kathy Jennings is reportedly planning to hire an investment bank to conduct an independent evaluation of OpenAI's assets, a step towards ensuring transparency and compliance with state regulations.
The outcome of this situation could have significant implications for the future of AI development and the balance between innovation, profit, and ethical considerations. As Nathanael Fast, director of the Neely Center for Ethical Leadership and Decision Making at USC Marshall School of Business, noted, the big question is what will happen to the values that OpenAI has once it becomes a corporation competing with other for-profit corporations.
