What ought to be vs how it needs to be

Developed with OpenAI’s Codex and Anthropic’s Claude. The positions are mine; the illustrations and substantive AI contributions are credited in the notes.¹

Builds on the Assumptions, Essay 01 and Essay 02: where a last premise is an aim rather than a claim about the world, there may be no fact to find, and treatment needs its own justification beyond the evidence for it.

You manage a company whose customers wait too long for help. Moving more staff onto the busiest shifts could shorten the queues. Some staff would have to work hours they now spend caring for their children. You cannot afford enough new staff to avoid the trade-off, and you have to set next month’s schedule. “Serve customers better” gives you a direction. You still have to choose what to ask of people.

Collective flourishing and a particular person’s suffering both belong in judging the choice. A policy can improve life on a large scale and still impose a serious cost on someone. That person lives with the cost. The wider gain does not make it disappear.

So the problem is how to turn an aspiration—global flourishing, say—into decisions with limited knowledge. The problem includes answering for what those decisions do to particular people, including people outside the organisation or country that makes them. It arises in running a company, a public programme and a border. I will use the border only at the end, as an illustration.²

1. Two views of the same choice

There are two ways of doing philosophy here. One asks what we should choose with all the relevant facts known. The other asks what an actual agent should do with the knowledge, time and resources they really have.

The second has fog of war turned on. You see part of the situation; the rest is hidden. Other people act while you think, and some of what you learn arrives too late to use. When I ask how things need to be, this is the view I mean: what a workable decision requires under those conditions. Its justification still answers to what I consider good.

It is easy to say I would uphold a principle when I already know who is trustworthy, what will happen and how to prevent the harm. The real question includes how I could find those things out, what finding out would cost, and what I should do before I know.

A full account of what ought to be must include how an agent should choose under the conditions in which they actually act. Describing the outcome I would prefer with complete knowledge is not enough.

Complete knowledge has a limit of its own. It can show that two people want incompatible things. It does not make them want the same thing, or tell us whose claim ranks higher. Facts inform the choice; the values that judge it still have to be stated.³

2. An aspiration needs a policy

“Make humanity better” gives a direction. It cannot be carried out in one step, from where we are to where it points. It does not tell a team what to do before Friday. It needs subgoals along the way. (An aspiration already within easy reach is the exception, and not an interesting one.) “We should be compassionate” can guide what we seek and rule out some means. On its own, it cannot specify the next step.

For that company, the next step could be a new schedule, with more staff on the busiest shifts and a date for checking whether queues actually shorten. Someone has to assign the work and find the resources. Now there is something to execute, with effects to assess: customers get help sooner if it works, staff have to rearrange their lives, and money spent on the change is unavailable for another useful project.

A country also organises people, resources and decisions over time, and that is the only part of the comparison I need.⁴ A declaration of compassion still leaves the practical work undone: who acts, with what means, by when, and how the result will be checked.

The aspiration stays a north star. The executable goal is a step toward it, and it is judged by whether it actually helps and by the means used to reach it. Hitting a deadline does not show the step was justified.

3. Society’s benefit, a person’s suffering

In the schedule case, shorter queues could benefit many customers while a parent on the staff struggles to find childcare. At the scale of a public programme, the same question remains: what the decision achieves for society, and what it asks of the person bearing its cost.

The liberal strand of Western political thought puts this tension clearly: an appeal to the common good must take the individual’s claims seriously.⁵ Whoever decides for society has to keep both in view: what the decision achieves, and what it does to the people affected. They also have to keep checking, as things change, whether the decision still holds up.

Whose flourishing is the target? In principle, everyone’s. But a person elected to govern answers to the people who put them in office, with a mandate to improve their lives. That adds another layer to the choice.

Say a government can fund only one programme, at home or overseas. Both are feasible, and the evidence is equally reliable. Even after accounting for the domestic costs and the country’s future ability to help, the overseas programme would contribute more to global flourishing. Answering to humanity, that is the right choice. Answering to the people who elected me to improve their lives, the domestic programme is the right choice. I would struggle to explain why they should forgo a benefit for the sake of others when they chose me to increase that benefit in particular.

There are layers and degrees of responsibility here, and resource constraints win most of the time. People care for family first, friends second, the homeless last—not because the homeless matter less in themselves, but because no one can carry everyone. I also see an evolutionary and self-interested explanation: lifting up those nearest to you gives you the best chance that they can lift you in return. From a weakened position, they couldn’t. In governing, priority for one’s own society leaves outsiders’ costs at a lower weight, not at zero.

I see human selfishness in this. If my compatriots were all provided for, with a decent standard of living, they could take the more generous, global perspective. A government chooses under conditions that include the people it answers to and what they still need from it.⁶

This tension survives perfect information. Knowing exactly who will suffer does not settle what may be done to them. Under fog of war we face that question without even knowing who will bear which costs.

4. The price of finding out

Before changing those schedules, you could measure queues across shifts and ask staff which hours they can work. That could reveal a less burdensome arrangement. It also takes staff time, and the current queues continue while you investigate. Another inquiry might change the decision, use resources needed elsewhere, or delay the help the change was meant to bring.

So the person in charge chooses how much to investigate as well as what to do, and that choice is part of the moral problem. Waiting can protect someone from a mistake. It can also prolong their uncertainty or let a chance pass. Acting quickly can catch the chance—and expose someone else to a risk that more checking would have found.

Thinking takes place inside the situation, not outside it. If the conditions a recommendation rests on have changed, it has to be updated. More time spent reasoning is worth it when it improves the choice by more than the time costs.⁷

This changes how we judge a decision afterwards. Say the evidence fairly supports the schedule change, and an effect nobody could foresee later causes harm. The harm matters. It does not make the decision reckless. Now a manager ignores the evidence and happens to pick a change that works. The luck does not make the procedure sound.

Three different questions are in play: how the outcome turned out, how good the decision was given what could be known, and what the decision-maker is responsible for. Looking at what was known at the time separates an avoidable mistake from a bad outcome under uncertainty. It also lets us learn from outcomes without pretending they were known in advance.

5. Cooperation among people with different aims

Your team needs another organisation’s help to carry out a project. Each side has its own reasons for taking part. What they tell you can be sincere, strategic or both. Someone chasing their own advantage can still contribute a great deal. Motive and effect are separate questions.

But the world is full of agents who will abuse the one who always cooperates. A promise of good intentions does not tell you what someone will do with your willingness to cooperate.

Game theory gives this a precise form. In the prisoner’s dilemma, each player chooses to cooperate or defect. Both gain from mutual cooperation, but either can gain more by exploiting a partner who cooperates. When the game repeats, reciprocity can win out under specified conditions: cooperate first, then answer what the other side did.⁸ So the starting point is conditional cooperation: not trust in everyone, and not distrust of everyone.

In a fresh benefits system, especially one closer to an ideal arrangement, the rule is to check eligibility before paying. Asking for evidence does not itself mean judging the applicant dishonest. It is a condition of giving the benefit.

Under the pressure of practical cases, I think these systems tend to devolve into something else: a starting distrust of the applicant’s account. “Show that you are eligible” becomes “I don’t believe you until you prove it.” Means-tested welfare is, unfortunately, one place where I see this problem.⁹

The checks have a cost of their own: an honest claimant waits, proves, and is sometimes wrongly refused. The policy imposes that cost, and it belongs in the decision-maker’s account like any other.¹⁰ Practical pressure explains how the arrangement can change. Whether the change is justified still needs an answer.

6. What “necessary” has to answer

A person in power can call a preferred policy necessary, point to uncertainty and limited resources, and ask everyone else to bear the cost. The appeal to realism can leave the person harmed with no answer beyond “you don’t understand how governing works”.

“We cannot investigate everything” describes a limit. It does not show that the chosen policy is the best response to it. For that we would have to compare the arrangements actually available: what they cost, what mistakes they expose people to, what chances they keep open. The comparison may be hard. Calling the policy necessary does not do it.¹¹

It cuts the other way too. Pointing to one person harmed shows a cost that must count. It does not yet show the policy was wrong: an alternative might have hurt different people, or more of them, while avoiding the harm we noticed first.

I want whoever decides to face both questions. What supports the policy under the actual constraints? And what happens to the particular people who pay for it? A claim of social benefit owes that account, and the account stays open to challenge. A virtuous goal gives no general permission to use whatever would advance it. The policymaker answers for the means and the harms they impose.

7. Costs, shared gains and abundance

The work has to be done by someone, and it uses time and resources. There is no free lunch: someone pays. With Earth’s stock of matter roughly fixed, material allocation is close to zero-sum: resources assigned to one use are unavailable for a competing use at that time. What people make of those resources by working together is another question. A policy’s gains can exceed its costs: in the prisoner’s dilemma, both players do better cooperating than defecting.¹² But the costs still fall on someone, and the decision-maker answers for where they fall.

Artificial superintelligence (ASI) means AI far more capable than humans across most intellectual work. If it brings abundance, enough food, water, electricity, shelter and transportation could be available for everyone. Providing these for one person would no longer require withholding them from another. The resources would still be finite. What would change is how much we could provide with them.¹³

A Porsche might become available to everyone and cease to say much about status. Social status would have to be redesigned. But a particular address would still be scarce: only one family could live there at a time. Enough shelter for everyone does not settle who gets the home several people want. Even with basic provision for everyone, choices among incompatible wants would remain. And that possibility supplies no resources for anything decided today.

For the programme being run now, the task is to find what can actually be done, how it serves the aspiration, and what it asks of the people involved.

8. One illustration: the person left outside

You are responsible for deciding who may enter a country. You want admissions to benefit society. With every fact known, you could tell what each person’s admission would bring. In practice, applicants tell you what they intend, and you have to judge what their answers are worth. Interviews take staff, time and money. While you investigate, people wait.

Say the government restricts entry broadly and keeps a few staffed entry points where applicants can be interviewed. Someone whose admission would have benefited society never reaches an interview and is left outside.

From the full-information view, we can see the missed opportunity. From the decision-maker’s position, the questions are different. What could reasonably have been known? Could this person have been given an interview? Would another arrangement have kept fewer people out, and what would it have cost, or risked, elsewhere?

The answers might support the restriction. They might expose a poor choice. The example decides nothing. It shows the work a practical justification has to do.

And if the restriction is justified, the applicant still lives with it. Their loss counts. So does the loss of an applicant whose admission would not have benefited this society. A state answering first to its own members gives that loss less weight than its members’ losses. It still counts. When the evidence or the situation changes, the decision-maker has to ask again whether the grounds for keeping these people out still hold.

Run the same questions on the company’s schedule change. What did it achieve for customers? Could staff still arrange childcare? Were there other arrangements, and does the evidence still support this one? “Serve customers better” gave the direction. This particular step still needs its own defence. What ought to be includes how we should choose with the knowledge and resources we have, and what we owe the people who live with that choice.

Footnotes

1. I supplied the central distinctions, positions and border illustration. The assistants contributed structure, wording, programme and company examples, and three arguments of their own: that calling a policy “necessary” can hide a preferred one, the separation of a policy’s costs from zero-sum loss, and the cost that checks impose on honest claimants. Specific notes credit those contributions. Codex developed the domestic/ overseas hypothetical, the recurring schedule case and its consequences, and the wording for my later mandate, distrust and abundance clarifications. ↩

2. This is a hypothetical illustration, not a report about a particular country, a claim about migrants’ motives, or a recommendation to close a border. Screening costs, accuracy and the effects of alternative policies would require evidence in a real case. Economic admission and asylum involve distinct questions; this illustration supplies no assessment of legal duties. ↩

3. Precisely: complete information does not supply the moral premises for judging outcomes or guarantee compatible aims. The perfect-information agent remains subject to genuine resource constraints; full knowledge does not create unlimited resources. See A0.P4 and the information/aims distinction in Objective truth. ↩

4. The analogy concerns organised execution under resource and time constraints. It does not equate a country’s purposes, authority, membership or obligations with those of a company, or make a CEO’s practice a source of political legitimacy. ↩

5. One statement is John Rawls’s, A Theory of Justice (rev. ed., Harvard University Press, 1999), §1: each person has claims of justice that the welfare of society as a whole cannot override. His constraint is stronger than the claim here, that individual suffering stays morally relevant. A related, narrower concern is Mill’s, in On Liberty, chapters I and IV: the limits of society’s authority over conduct that concerns only the individual. The strand is one among several in Western thought. ↩

6. A0.P1: established suffering is bad for its bearer and retains moral weight. This does not set an aggregation formula, assign equal weights, or make every individual cost an automatic veto. Precisely: the principle is global flourishing; national priority reflects limited resources and the particular mandate a government answers to, not a claim that outsiders matter less in themselves. In the illustration, the choices judged right from humanity’s standpoint and from elected office differ. Being provided for could make a more generous perspective possible; it does not guarantee that everyone will adopt it. Codex proposed the domestic/overseas illustration. These judgments, the mandate distinction and the point about provision and generosity are mine. ↩

7. Stuart Russell’s Metareasoning treats reasoning as action with possible decision benefits and costs, including delay. This supplies no exact stopping rule or requirement to calculate unknown quantities as precise probabilities. ↩

8. Axelrod and Hamilton, The Evolution of Cooperation (1981), study reciprocity in a repeated two-player prisoner’s-dilemma model under specified payoffs and interaction conditions. No prevalence of exploitation or border policy follows. ↩

9. One measured case: Great Britain’s Fraud and Error in the Benefit System, FYE 2025 shows how much the rate varies between benefits. Overpayments due to fraud were 8.0% of Universal Credit spending (10.8% the year before), 2.2% across all benefits (2.7% the year before) and about 0% for the State Pension; underpayments were 0.4%. Universal Credit is means-tested; the State Pension is not. These are shares of money in one country, not of claimants. They show measured costs of fraud, not that most applicants are dishonest, which checks are best, or how every institution changes over time. The tendency from routine verification toward distrust is my account of what practical pressure does to these systems, not a finding established by these figures. Its costs and any proposed checking policy require their own assessment. ↩

10. The AI assistant drew this consequence from my principle that the policymaker answers for the harms a policy imposes: the checks are part of the policy. Decisions reversed on challenge are measurable. DWP’s Personal Independence Payment bulletin to April 2025, focused on England and Wales, reports that 3% of assessed initial decisions made during January 2020–December 2024 were overturned at a tribunal hearing. Separately, 31% of mandatory reconsiderations cleared during May 2020–April 2025 changed the award, excluding withdrawn cases. These are different cohorts, not two stages of the same sample. The tribunal share may rise as outstanding appeals conclude. Neither figure measures all incorrect refusals. ↩

11. Precisely: A0.P4 requires a defence of coercive rules as well as factual scrutiny within a shared framework. Relevant evidence does not alone justify treatment, as developed in Statistics. This passage specifies questions for scrutiny, not a complete decision procedure or an endorsed border rule. The AI assistant developed the objection about necessity concealing a preferred restriction and the comparison of possible responses. ↩

12. In a zero-sum game, gains to one participant are offset by losses to others. Precisely: Earth’s stock of matter is close to fixed, so material allocation is close to zero-sum; the value people create by cooperating with that matter need not be. In the prisoner’s-dilemma example, mutual cooperation benefits both relative to mutual defection. The AI assistant proposed separating a policy’s costs from zero-sum loss. No total welfare metric or universal win-win policy is implied. ↩

13. The ASI case is conditional, not a forecast or a premise required by the argument. Current AI-productivity research does not establish an eventual end to all scarcity: the IMF’s The Global Impact of AI: Mind the Gap (2025) models gains under specified adoption and productivity assumptions, including unequal gains between regions. It is not a model of ASI or universal abundance. The basic-provision meaning, Porsche/status illustration and scarce-address example are mine. Codex supplied their wording and integration. The AI assistant also proposed the limits of time and attention. No infinite stock or end to every conflict is implied. ↩