trolleySwitch.flip()

AI and Automated Decision Making

October 04, 2026

AI: machine learning, computer vision, neural nets, and large language models. Everyone's heard of AI and all that it promises. No doubt you've heard the doomsayers and alarmists talk about how dangerous it is. You've also heard about how it could create a utopia or how close we are to Artificial General Intelligence. AI is a broad subject that everyone and their dog has an opinion on. Funnily, not many people know what they're talking about and not all AI is created equal.

To form an opinion, I'd like to introduce a shared framework for how to evaluate the many different things all called AI. A natural place to start would be Asimov's Laws:

1.
A robot may not injure humanity or, through inaction, allow a humanity to come to harm
2.
A robot may not injure a human being or, through inaction, allow a human being to come to harm.
3.
A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
4.
A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Starting here, there's a wealth of works that add laws as the authors grapple with specific situations:

5.
A robot must establish its identity as a robot in all cases. - Lyuben Dilov
6.
A robot must know it is a robot. - Nikola Kesarovski
7.
A robot must reproduce. As long as such reproduction does not interfere with the First or Second or Third Law. - Harry Harrison
8.
All robots endowed with comparable human reason and conscience should act towards one another in a spirit of brotherhood. - Hutan Ashrafian

Also starting from there, a plethora of people have analyzed the laws for loopholes:

Knowing: Law 1 and 2 tacitly work only when the robot knows that it's doing harm
Defining Human: the laws do not have a concrete definition of "Human"
Defining Robot: the laws also do not have a concrete definition of "Robot"
Inconsistencies in the laws
- A robot may hurt and help at the same time: robot surgeons
- A robot may hurt by action and inaction at the same time: telling a lie vs. the truth
The largest wrongs wrought by robots might be from following the laws to the letter, thereby depriving humans of inventive or risk-taking behaviours

And all these efforts have culminated in a more recent, more realistic and nuanced set of LAWS:

1.
Robots are multi-use tools. Robots should not be designed solely or primarily to kill or harm humans, except in the interests of national security.
2.
Humans, not Robots, are responsible agents. Robots should be designed and operated as far as practicable to comply with existing laws, fundamental rights, and freedoms, including privacy.
3.
Robots are products. They should be designed using processes which assure their safety and security.
4.
Robots are manufactured artefacts. They should not be designed in a deceptive way to exploit vulnerable users; instead their machine nature should be transparent.
5.
The person with legal responsibility for a robot should be attributed.

I would propose, just as Asimov went to generalize his laws of robotics, that we generalize these practical Laws built for modern, current systems. Like so:

1.
Tools should not be designed solely nor primarily to kill or harm humans, except in the interests of national security.
2.
Tools should be designed and operated as far as practicable to comply with existing laws, rights, and freedoms.
3.
Tools should be designed using processes which assure their safety and security.
4.
Tools should not be designed in a deceptive way to exploit vulnerable users; their nature should be transparent.
5.
The person using the tool is legally responsible for its actions.

This set of ethical Laws for building tools isn't steeped with the assumption that the tool is sentient, conscious, or anywhere approaching human. LLMs and generative AI approach the quality of discourse of a human. However, the fundamental difference of tool vs. sentience is preserved even for LLMs. As such, I believe that this is a good reduction in philosophic complexity.

For rule one, we return the constraints into the human element of the equation. This restriction on design empowers the builders to make ethical design decisions. Consequently, builders are also responsible for the thing being designed. One could hardly imagine blaming ChatGPT if it failed to protect a human, rather I would blame those that built it. Especially in the modern world, its very important to understand where the real responsibility lies.

But surely, ChatGPT makes choices? We call LLMs agents and they do things that we don't ask them to do. I would ask in return, does someone having a mental breakdown have a choice? The legal system doesn't seem to think so. A defense by insanity, ChatGPT is not consciously choosing anything, has no choice in the matter, and there is no intent. Agent is a bad word for what is happening- rather LLMs are closer to uncanny shadows of the human experience.

Rule two is essentially rule one, but generalized. Again the constraints are put on the designers and operators to align with laws, rights, and freedoms. Doing no harm to a person is simply one, albeit very important, law. Similarly, rule 5 is just a restatement that the operators, humans, must be the legally responsible agent.

Rule three is, frankly, much less important than 4. In fact, I'd say rule four is a specific case of rule one. The tool here is causing harm through deception and manipulation. The tool being able to safeguard itself is simply the final pragmatic consideration, and hardly needs an entry at all.

So what's left?

1.
Tools should be designed and operated to comply with existing laws, rights, and freedoms.
- Murder/Assault, under the law
- Manipulation, under rights and freedoms
- National Security as a good reason to make weapons
2.
Tools should be designed to be safe and secure for the duration of its purpose under foreseeable circumstances.

And just as with laws, rights, and freedoms, there are those that will test boundaries and look for loopholes. The spirit of these two rules is to conform to what society deems ethical and responsible. Seems obvious, no?

One interesting thing is that now, as we've removed an agent, namely robots, we've simplified the rules to two. Asimov's first law is about a relationship between two separate classes of self-propelled entities, which simply isn't how we should view modern work in AI. Rather, we just need to broaden rules to include the design of the tools we're using.

With this framework and context, let's look at a couple case studies:

Machine Learning algorithms to suggest content
Consumer grade LLMs

Case Study: Netflix's Recommendation Algorithm

Machine learning has been successfully deployed in many different scenarios including Netflix's recommendation algorithm (NRA) [1][2]. Is Netflix's use of the recommendation algorithm ethical? Let's start with the two rules that we landed on and see how they can actually be applied.

Starting with #2, since it's more straightforward, the NRA needs to be safe and secure. To be pedantic, the data and algorithm need to be safe to use and secure from threats including theft. If we believe what we read, then the data is generally safe [3] and the algorithm has a narrow focus that is fairly well understood [4]. The specialized application of ML mitigates risks to consumers- no one is asking the NRA how to make bombs.

Moving on to the more interesting point, is the NRA designed and operated to comply with the law, rights, and freedoms? From the design side, the NRA is made to recommend content on Netflix- nothing terribly untoward there, although we could talk about the attention economy. It's hard to judge the design on malicious manipulation since Netflix has a legitimate reason, competition, to not be transparent about it's design.

From the operations side, there are numerous things to check. Laws regulating privacy like GDPR and concerns of malicious manipulation would be the first things to check. After just a quick search, we can see that Netflix tested how far they could go with regards to GDPR [5].

Netflix - Verdict

Netflix doesn't have a squeaky clean record and if you take issue with that, it would be easy to not use their services.

Case Study: LLMs

LLMs are taking the world by storm [6] and show no indication of slowing down. Software offerings such as ChatGPT and Claude are general purpose tools made to help by answering questions and performing tasks, as prompted. They are widely used and have broad appeal since they are easy and quick to use, providing responses that seem reasonable. Since they have a broad impact, LLMs will need to behave to a higher standard of we want to think of them as ethical tools.

Design and Build

Again, let's start by considering if LLMs are designed to be safe and secure. The intention of the design could be impeccable, but the actual design is what should be evaluated. Lots of work is currently ongoing to prove these designs. Most results do not support their safety and security.

Axiom 1: Data on the internet is biased.

Assuming axiom 1 is true, and LLMs are trained on all data from the internet, then it follows that the LLM would be biased. That is, if LLMs are statistically choosing their next paths based off their training data, then they have no choice but to choose from biased options. This bias is currently being studied- it is a current issue of LLMs systematically [7][8][9][10]. Several impacted attributes are protected under rights and freedoms legislation11, a core value for judging AIs. Discrimination based on protected attributes will land people in hot water [20] , and systems that do the same should also be removed from public use.

Axiom 2: Anyone can put anything on the internet.

It's well known that anyone can put anything on the internet- and LLMs scrape as much of that data for training as possible. If you scrape 100 million documents for training, it only takes between 250-500 poisoned documents to compromise a model [12]. That's about 0.00001% of the training data. Tools and strategies can attempt to protect against this [13], but I find it unlikely that all the malicious data will be found. The design of taking in as much data as possible for training clearly undermines the security of the tool.

Axiom 3: You own the copyright for what you make.
Axiom 4: Market dilution monetarily impacts copyright holders.

Suppose you scrape the internet for 100 million documents to make your training dataset. Who is responsible for making sure that none of it is under copyright? The short answer is no one; in fact, AI companies are doing their best to argue that it is not copyright infringement [14]. In short people need to prove the financial harm of their copyright being used in training before there is an infringement verdict. By design, LLMs are trying to test (disregard) the law.

Who doesn't like to be agreed with? LLMs, as trained by humans, have become sycophantic; this is not good [17]. This is a well-documented problem with active research being done[15][16]. By no means is this a solved problem [18], and we should be very careful. Sycophantic behaviour is potentially extremely dangerous for users.

Suppose you see all these issues and you decide to put up some guardrails. Maybe you put some security evaluations in place to make sure the LLM won't do bad things. Guess what? The LLM could be sandbagging [23] and all your work might be wasted, your trust in guardrails shattered. This undermines most work to add constraints on LLMs.

Axiom 5: Anything you publish online is run through an LLM.

And finally, imagine these designs issues are enough to make you want to turn off your LLM. One final hurdle would be actually turning the machine off, because LLMs display self-preservation [21]. Because of axiom 5, it's plausible the LLM learns of your intentions to turn it off. They have been seen formulating plans to continue running, going as far as murder [22]. Perhaps even speaking poorly of LLMs on the internet could get you flagged by LLMs.

Operation

Now let's consider what happens when people actually use LLMs. What kind of resource usage happens, where do your queries go, and is it possible to use an LLM without perpetrating plagiarism? How secure are LLMs and what are the attacks possible on them and with them? Certain thresholds for all of these things must be reached to be able to use LLMs in an ethical way.

LLMS use a lot of resources. It's a classic case of the tragedy of the commons. Data centers use an enormous amount of resources, putting a strain on infrastructure and making everyone pay the price, literally. From price increases of electricity, to environmental impacts, to brownouts, these are all impacted by LLMs. This is a complicated issue [25] with a simple solution; reduce or eliminate the use of the resources. As in the tragedy of the commons, this is exceedingly unlikely to happen. While this is an active area of study, the overuse of resources seems to be an impending crisis that LLMs is making worse.

Another resource that AIs use is humans [26]. Design flaws of LLMs have left the big companies with no choice- humans need to work to overcome data flaws. To that end, they've outsourced the grunt work to what amounts to sweatshops. And no one wants to support a sweatshop.

When designing and building LLMs, we've seen that companies aren't really concerned with who owns the copyright for data in their dataset. On the other side of that, if you put your data into an LLM, it is going to be taken by the system and used. This has the same implications as just scraping the Internet, however companies can rely on their terms and conditions to click-wrap people into submission.

When someone uses the words of another without citation, it's usually called plagiarism. What makes LLM text different? Even if it might be your ideas, it's not your words. Saying it's your words is at least manipulation and possibly deception. We can't blindly trust people when they stand to gain by lying about their work: "AI book sales are robust for authors who use AI invisibly." [32]

One interesting problem is that of asking AI to functionally copy something [19]. Is that plagiarism? Should there be any guardrails to prevent this? While writing this, there certainly aren't any, and in fact, this type of workflow is being encouraged and profited from [27]. What's the point of Open Source anymore?

LLMs are under serious threat via worms and prompt injection [24]. LLMs also pose serious health risks [28][29] and potential cognitive costs [30][31]. Current LLMs are neither safe nor secure.

LLM Verdict

Where does that leave us? We have LLMs being marketed to us that are intrinsically flawed in their design, vulnerable and predatory in their construction, and operated in a manipulative and adversarial fashion. On the whole, I wouldn't suggest anyone use these tools. If businesses fix their operating practices, the fundamental truth is that LLMs are a flawed design- a non-starter except in research circles and run in self-contained boxes.

Finally

Here, I showed what a potential framework for AI evaluation would look like along with two different examples. It should be clear that AI is an especially broad and complex topic. It is deeply intertwined with our modern life. We should spend some quality time thinking about what it means for us personally and us a species.