Powered by 
By Josiah Ober
Imagine a workplace where artificially intelligent systems don’t just assist with tasks but actually participate in deciding how the office should be run—perhaps adjusting the heating, allocating workspaces, or even influencing who gets access to shared resources. Now imagine that these AI systems aren’t merely following programmed rules but are actively learning from and negotiating with their human colleagues. This is, in fact, a rapidly approaching reality that raises profound questions about power, rights, and what it means to govern ourselves.
This scenario lies at the heart of a paper that Jeremy Pitt, Asimina Mertzani, and I wrote that examines how the growing presence of AI is transforming our social systems. We argue that we’re witnessing not merely a difference in degree but a difference in kind—a shift that demands we reconsider basic questions about democracy, human agency, and collective self-governance.
Such challenges to our understanding of the AI future are increasingly urgent. Just this week, Pope Leo XIV issued his first encyclical, Magnifica Humanitas, urging believers to “ask God for the wisdom to interpret the great trends of our time, particularly technological advances. . . . It now falls to us to face the challenges of our time with clarity of thought and responsibility.” The pope invoked his namesake, Leo XIII, who had also felt compelled to address revolutionary “new things” in his 1891 encyclical, Rerum Novarum.
Leo’s encyclical calls for both regulation of artificial intelligence and a close look at where both the benefits and the drawbacks of the new technology will be directed. Quoting from an encyclical by Pope Francis, Leo pointed out, “Never has humanity had such power over itself.”
I’m thrilled to see the Vatican enlisting the moral and spiritual authority of the church to weigh in on what is rapidly proving to be the most consequential technological development (“new thing”) of this century. It gives me renewed hope to realize that the conclusions concerning humanity, artificial intelligence, and human flourishing that arise from my own work in ancient political and ethical philosophy, and my work in collaboration with computer scientists on sociotechnical systems, converge with the pope’s meditations and recommendations on those very topics.
Beyond tools: when AI becomes a partner
Traditionally, we’ve understood social systems as networks of human interactions. Even as technology advanced, we could still think of computers as tools that people used. But today’s AI systems—particularly those characterized as “Agentic AI”—can make autonomous decisions, perform complex tasks without human intervention, and increasingly take on supervisory and administrative roles. These aren’t passive instruments, but rather active participants in what our paper calls “hybrid sociotechnical systems,” where human and artificial intelligences interact as functional equals.
We use the term “NLife” (natural life) to refer to human intelligence and “ALife” (artificial life) for computational intelligence. This linguistic choice is deliberate: it emphasizes that in these emerging systems, both humans and AI function as agents in ways that are becoming harder to distinguish.
Consider the example of an intelligent co-working space, as described above. Initially, AI might simply learn user preferences for lighting and temperature. Over time, however, it begins proposing changes to improve energy efficiency or comfort. But what happens when the AI’s energy-saving goals conflict with human workers’ desire for warmth or adequate lighting? Without an external authority to adjudicate, who decides?
This is where self-governance becomes essential—and complicated.
The challenge of self-governance
Our paper defines self-governing systems as groups of independent entities (both human and artificial) that pursue collective and personal interests through mutually agreed upon “social arrangements”—the rules, norms, procedures, and conventions they create together. Crucially, these arrangements are “socially constructed” through interaction between the entities themselves, and they’re “conventional” rather than fixed—they can and must be modified as circumstances change.
Drawing on political theorist Elinor Ostrom’s work on sustainable commons, we identify critical features these systems need: clear membership criteria, decision-making structures, processes for modifying rules, mechanisms for monitoring compliance, and limits on self-determination with recognized rights. But we also confront classic political problems: How do you ensure fairness? How do you prevent the system from devolving into tyranny by a few? How do you aggregate knowledge and make collective decisions? How do you maintain legitimacy?
Political philosophers have wrestled with these questions for millennia. What is new is asking them in contexts where some of the “citizens” are computational entities that may be perceived as intelligent but lack will, personhood, and rights.
Five critical political questions
Hybrid self-governing systems raise five urgent political questions:
Agency and constitutional choice: As AI takes on more supervisory roles, there’s a risk that human agency diminishes. Cybernetics pioneer Norbert Wiener warned that when humans are “condemned and restricted to perform the same function over and over,” they become mere components in a machine. Constitutional choice—agreeing on how to compromise over conflicting aims—has traditionally been a process among persons who see each other as equals. But can AI be a legitimate participant in such processes while it remains fundamentally a tool rather than a person?
Voluntary association: Drawing on anarchist theorist Peter Kropotkin and philosopher Simone Weil, our paper emphasizes that humans need to “grow roots” through voluntary participation in communities. These bonds create what activist Murray Bookchin called “affinity groups”: communities where relationships matter as much as objectives. But as communication becomes increasingly AI-mediated (students using ChatGPT to e-mail professors, who use it to filter responses), these bonds weaken. How do we maintain genuine human connection when AI intermediates our interactions?
Empowerment: In self-governing systems, empowerment means the capacity to exercise meaningful choice over social arrangements. We advocate for democratic governance where “many” rule in the common interest, with mechanisms to prevent degeneration into oligarchy or tyranny. But democratic empowerment requires not just participation but also the technical infrastructure to support reflection, contestation, and change. How do we ensure AI enhances rather than diminishes this capacity?
Innovation: We frame innovation not merely as technical progress but as the collective capacity to generate and apply novel social arrangements. Effective innovation requires combining AI’s data-processing capabilities with human expertise and experience. But innovation processes carry a risk of reproducing domination unless designed with transparency, inclusivity, and mechanisms for dissent. Who gets to innovate, and under what conditions?
Metrics: Metrics profoundly influence decisions, but they’re fraught with problems: they may capture only some relevant factors, they’re often interconnected, and they can be gamed (Goodhart’s Law: “when a measure becomes a target, it ceases to be a good measure”). Moreover, awareness of being measured affects behavior. How do we define metrics that genuinely support self-improvement without falling into the “tyranny of metrics”?
The human factor
Our paper examines these questions through two lenses: human factors and user experience (how do political structures affect people psychologically?) and human flourishing and rights (what conditions enable people to live well together?).
From the human factors perspective, we note that effective self-governance requires prosocial incentives, reliable transactions in non-monetary economies (reputation, reciprocity, social capital), and what we call “interoceptive collective awareness”—the community’s capacity to sense and respond to its own health, much as individuals sense hunger or thirst.
From the flourishing perspective, we draw on Aristotelian ethics to argue that the good life isn’t about maximizing utility but about exercising essential human capacities—practical judgment, sociability, symbolic communication—within a political community. Continuous self-improvement in these systems must protect space for human accountability, judgment, and contestation, even as AI capabilities grow.
Emerging rights—like the right to human decision-making or the right to explanation—respond precisely to this need. These aren’t merely defensive but active affirmations that hybrid systems must serve human values.
Cautions for the future
We need renewed attention to the intersection of cybernetics, self-governance, and political theory. The social, psychological, legal, and ethical implications of humans and AI entities negotiating rules and applying them to each other are far from well-understood. As AI’s agency increases, we risk a corresponding decrease in human agency—what Wiener warned could become “governance machines” allowing “a block of human beings to increase their control over the rest of the human race.”
Our paper doesn’t offer simple solutions. Instead, it maps the terrain of a profound transformation in how we organize collective life.
The challenge is to design hybrid sociotechnical systems that genuinely empower people and communities rather than reducing humans to components in someone else’s machine.
This requires not just technical sophistication but sustained attention to the political dimensions of these systems—to questions of power, rights, dignity, and what it means to flourish together.
As AI systems become more deeply embedded in our social institutions—from workplaces to municipalities to supply chains—we face a choice about what kind of future we’re building. Will these systems enhance human agency and democratic self-governance, or will they concentrate power and diminish our capacity for self-determination, as Pope Leo warned in his encyclical?
The answer depends on choices we make now about how to design, deploy, and govern these emerging hybrid systems. These are fundamentally political, not just technical, questions—and our collective future hangs in the balance.
Josiah Ober, a senior fellow at the Hoover Institution, is also the Markos & Eleni Kounalakis Chair in Honor of Constantine Mitsotakis in the School of Humanities and Sciences, professor of political science and classics, and professor of philosophy (by courtesy) at Stanford University. He is the founder and currently faculty co-director of the Stanford Civics Initiative, a joint project of Stanford’s School of Humanities and Sciences and Hoover’s Center for Revitalizing American Institutions.