A coalition of former OpenAI employees is urging the attorneys general of California and Delaware to intervene and block the company's transition to a for-profit entity. These former employees, supported by Nobel Prize winners, experts, and advocates, have voiced concerns about the potential ramifications of prioritizing profits over the original mission of ensuring AI benefits humanity.
The core concern revolves around OpenAI's ambition to develop AI systems that surpass human capabilities. The former employees are worried about who will control this advanced technology and whether it will remain accountable to its founding commitment to safeguarding against potential harm. They argue that converting to a for-profit structure could undermine these crucial safeguards, potentially leading to the technology being used in ways that contradict the public good.
Page Hedley, a former policy and ethics advisor at OpenAI, articulated the central anxiety, stating, "Ultimately, I'm worried about who owns and controls this technology once it's created." Hedley and other ex-OpenAI staff are appealing to California Attorney General Rob Bonta and Delaware Attorney General Kathy Jennings to utilize their authority to uphold OpenAI's nonprofit mission and prevent the proposed restructuring.
Anish Tondwalkar, a former member of OpenAI's technical team, emphasized the importance of the "stop-and-assist clause" within OpenAI's nonprofit charter. This clause mandates that OpenAI halt its AI development and assist if another organization is nearing the achievement of better-than-human AI. Tondwalkar cautioned, "If OpenAI is allowed to become a for-profit, these safeguards, and OpenAI's duty to the public, can vanish overnight." Nisan Stiennon, an AI engineer who worked at OpenAI, expressed a more direct concern: "OpenAI may one day build technology that could get us all killed. It is to OpenAI's credit that it's controlled by a nonprofit with a duty to humanity. This duty precludes giving up that control."
The former employees highlight that OpenAI, buoyed by the success of ChatGPT, has been increasingly cutting corners on safety testing and rushing out new products to gain a competitive edge. They fear that these decisions will have far-reaching consequences as the technology becomes more powerful.
In response to these concerns, OpenAI has stated that "any changes to our existing structure would be in service of ensuring the broader public can benefit from AI." The company has clarified that its for-profit entity will be a public benefit corporation (PBC), similar to other AI labs like Anthropic and Elon Musk's xAI, with the distinction that OpenAI will maintain a nonprofit arm. OpenAI argues that this structure will ensure the nonprofit continues to thrive alongside the for-profit, enabling it to fulfill its mission. CEO Sam Altman has stated that OpenAI may require "hundreds of billions and may eventually require trillions of dollars" to fulfill its mission, suggesting the need for a more conventional capital structure to attract massive funding rounds.
However, critics remain skeptical. A coalition of California nonprofits, foundations, and labor groups have also raised concerns about OpenAI's plans to restructure as a for-profit company. They argue that this move could jeopardize OpenAI's nonprofit assets, estimated to be worth as much as $300 billion, which are intended for public benefit.
Previously, OpenAI wanted to fully convert to a for-profit company, with the nonprofit as a separate entity. This decision prompted scrutiny from the Delaware and California attorneys general, who sent letters seeking information about the for-profit plans and the transfer of assets. Elon Musk also filed a lawsuit to stop the transition. In light of the backlash, OpenAI has seemingly abandoned its plans to fully transition to a for-profit entity and has chosen to restructure its commercial arm as a PBC while retaining oversight from its nonprofit board.
The Delaware Attorney General has hired an investment bank to conduct an independent evaluation of OpenAI's restructuring plan as it transitions to a for-profit entity. This evaluation aims to ensure the fairness of the asset valuation and secure necessary approvals from state regulators. The outcome of these inquiries and legal challenges remains uncertain, but they highlight the complex legal and ethical considerations surrounding the development and deployment of advanced AI.
