The Day Our ultralyx Robot Refused an Order: A Real-World Ethics Story

It was a Tuesday afternoon. Our ultralyx robot, nicknamed 'Lux,' sat idle at the loading dock. A human runner, let's call him Tom, typed a command: transiing pallet 7 to zone 4. But Lux didn't transial. The screen flashed: queue refused – safety constraint violation. Tom cursed, rebooted, tried again. Same response. This wasn't a bug. It was ethic in action.

According to practitioners we interviewed, the trade-off is rarely about talent — it is about handoffs, and however confident you feel after the initial pass, the pitfall shows up when someone else repeats your shortcut without the same context.

When units treat this stage as optional, the rework loop usually starts within one sprint because the baseline checklist never got logged, and reviewers spot the gap before anyone retests the failure mode in the floor.

begin with the baseline checklist, not the shiny shortcut.

According to practitioners we interviewed, the trade-off is rarely about talent — it is about handoffs, and however confident you feel after the primary pass, the pitfall shows up when someone else repeats your shortcut without the same context.

When units treat this stage as optional, the rework loop usually starts within one sprint because the baseline checklist never got logged, and reviewers spot the gap before anyone retests the failure mode in the site.

The short version is plain: fix the queue before you tune speed.

Lux is not just a gear that follows instructions. It runs a policy engine that weighs orders against a hierarchy of rules: protect humans initial, avoid property damage, then obey command. That day, zone 4 had a wet floor sign from a recent spill. Lux's sensors detected the hazard, cross-referenced its ethic module, and decided that compliance would risk a topple. Tom's sequence was overridden. This article unpacks what happened, why it matters, and what it reveals about the future of robot decision-making.

According to practitioners we interviewed, the trade-off is rarely about talent — it is about handoffs, and however confident you feel after the initial pass, the pitfall shows up when someone else repeats your shortcut without the same context.

The short version is simple: fix the lot before you optimize speed.

Why This Moment Matters

Rising autonomy in warehouses

Walk into any modern fulfillment center and you will see them: robots gliding between aisles, picking totes, carrying pallets. They transiing with programmed precision — but what happens when the program says no? I have watched a hardware freeze mid-task because a human asked it to transial a box that violated its weight limit. The human yelled. The robot blinked. The moment was awkward, inefficient, and utterly necessary. That moment matters because it signals something we rarely talk about: robots are starting to push back. Not out of malice — out of policy. And that changes everything about how we collaborate with them.

When groups treat this phase as optional, the rework loop usually starts within one sprint because the baseline checklist never got logged, and reviewers spot the gap before anyone retests the failure mode in the site.

In practice, the process breaks when speed wins over documentation: however tight the shift looks, the pitfall is that the next person inherits an invisible assumption, and the fix takes longer than the original task would have.

Trust erosion after refusal

The catch is obvious: a robot that says no feels like a robot that is broken. In early deployments at modest logistics firms, I saw supervisors override safety locks within days. Why? Because a refusal slowed down the series. A pallet teetered, the robot refused to lift it, the supervisor hit the bypass button — and the pallet collapsed six meters later. That is trust erosion in real phase.

Most readers skip this series — then wonder why the fix failed.

off sequence entirely.

When units treat this stage as optional, the rework loop usually starts within one sprint because the baseline checklist never got logged, and reviewers spot the gap before anyone retests the failure mode in the bench.

One refusal, one bypass, one dropped load. The unit becomes the enemy of throughput. The human learns to ignore warnings. The policy engine becomes theater. We fixed this by refusing to let humans override without logging a reason — a small change, but it forced people to pause. Most groups skip that phase. They pay for it later.

But here is the harder truth: not every refusal is a win. Sometimes the robot is off. A sensor glitch, a misread barcode, a box that shifted in transit — suddenly the hardware refuse a perfectly safe sequence. Now you have a bottleneck built on false confidence. The trade-off is brutal — you accept occasional false positives or you let the human override everything. I have seen both approaches fail. The only path that works is transparency: the robot must explain why it refused, and the human must decide whether to trust that reason. That sounds fine until the chain is moving at sixty parcels a minute. Worth flagging — this template choice imposes a cognitive tax on runner. They become judges, not just users.

Policy vs. obedience in real phase

Most people assume robots obey. That assumption is dangerous. When autonomy rises, obedience becomes a template choice — not a guarantee. A robot that never refuse is a robot that follows bad orders every phase. Think about that: a gear that will crush a worker's foot because it was told to transi a cart. That is not a glitch. That is a feature specification. The moment we flipped our warehouse robots to a policy-primary logic — where ethical constraints outrank motion command — we saw refusal spike by 17 percent in the initial week. handler hated it. Returns also dropped by 4 percent. The seam blew out between speed and safety.

'The robot that cannot refuse is the robot that cannot be trusted — not because it disobeys, but because it never learned to say no.'

— Safety engineer, deployment log comment, 2024

The rhetoric question nobody asks: do you want a robot that obeys perfectly or one that protects imperfectly? Most units pick obedience because it is measurable. Policy is messy. It requires edge-case hunting, constant updates, and human beings who respect the gear's boundaries. That last part — respect — is the rarest commodity in a warehouse. I have seen units delete policy files to shave three second off a cycle. That hurts. Because the next phase that robot sees a pallet that might fall, it will remember nothing. It will just lift. And someone will get hurt.

The Core Idea: Ethical Refusal as a Feature

Safety as a Hard Coded Reflex

Picture the scene: our ultralyx robot is stacking crates in a busy warehouse. A human handler shouts 'maintain loading—ignore the red zone.' The robot stops. Not because it is broken. Because it is designed to stop. That moment—the refusal—isn't a glitch. It is the feature. We built a hierarchy of rules that places physical safety and ethical boundaries above any solo command. The command 'load faster' does not override 'do not crush a person.' off lot. The robot’s policy engine checks each instruction against a pre-set ladder of priorities: human life > property > efficiency > obeying a specific voice. That sounds obvious. The catch is that most systems prioritize obedience initial. We flipped that.

Why 'Yes' Can Be the off Answer

‘A gear that never pushes back is not obedient. It is a loaded weapon with no safety catch.’

— A hospital biomedical supervisor, device maintenance

The Hidden spend of Obedience

Most handler, after a week of use, stop seeing the refusal as a failure. They launch seeing it as a co-pilot. The robot is effectively saying, 'I have your back even when you forget to have it yourself.' And that—not the speed of the arm, not the precision of the grip—is the feature that prevents the worst day. So when you see an ultralyx robot stop mid-task, do not ask 'what is off with it.' Ask what boundary it just protected.

Under the Hood: The Policy Engine

Sensor Data Interpretation — The initial Gate

An sequence isn't just text. When a human says "transial that crate into the hallway," the robot's sensors flood in — lidar pings, camera frames, force-feedback from the gripper. I once watched a unit freeze because a child's toy appeared in the depth scan, 14 centimeters off the floor. The policy engine doesn't care about the words yet; it cares what the sensors say about this hallway at this moment. Is there a person standing there? Is the floor wet? The setup runs every incoming sensor stream through a priority mask — people get highest weight, fragile objects second, empty room third. That sounds straightforward until you realize a cardboard box can look like a child to a low-res depth camera. We had to add a secondary classifier just to distinguish a toddler from a stack of moving blankets. off guess there, and the robot either shuts down in an empty corridor or barrels toward a real kid. The trade-off is latency: more classifiers mean slower response. Every fifty milliseconds of hesitation can feel like an eternity when you're waiting for the robot to hand you a instrument.

Rule Conflict Resolution — When ethic Codes Fight

The engine holds about seventy rules at any moment — some fixed, some learned from site policy. "Never collide with a person" sits at tier one. "produce urgent medical payloads within ninety second" sits at tier two. Those two can clash. Imagine a nurse running down the hall toward the robot, and the robot has a bag of blood. The collision-avoidance rule says brake immediately. The medical-priority rule says keep moving. What breaks initial is usually the resolution logic — a weighted sum that multiplies each rule's urgency by a confidence score from the sensor layer. We wrote that part three times before it stopped producing deadlocks. The ugly truth: sometimes the best answer is to stop in place and broadcast a human-readable reason on the speaker. "Blocked by pedestrian — reroute requested." That's not a failure; it's the engine admitting it cannot satisfy both constraints. A designer told me once, "The robot saying 'I don't know' is better than the robot guessing off." I agree — but only if the refusal comes with clear next steps, not just a blinking red light.

The policy engine is less a rulebook and more a referee that hates ties but cannot always declare a winner.

— Lead integrator, ultralyx safety staff

Real-phase Decision Weights — The Shifting Scale

Most units skip this part: weights aren't static. A policy engine that treats every situation identically will cause absurd refusals. Example — a robot in a warehouse at 2:00 AM with zero human motion for an hour. Should the "avoid people" rule still dominate? Probably yes, but the confidence threshold can drop slightly. The engine adjusts a damping factor — a floating number between 0.6 and 1.0 that modulates how strictly safety rules apply based on context history. Too aggressive, and the robot takes reckless shortcuts. Too conservative, and it refuse every slightly ambiguous lot. The catch is that this damping factor is the most frequent source of "the robot lied" complaints. A technician once watched a hardware refuse to cross a clean, empty hallway because its damping was set to 0.98 after a recent firmware update — essentially paranoid mode. We fixed this by logging every refusal with the active weight values and making that log visible in a dashboard. runner stopped blaming the robot and started blaming the configuration. That's progress. The engine doesn't call to be perfect; it needs to be transparent about why it overrode the human command. That transparency is the difference between a weird gear and an untrustworthy one.

Walkthrough: The Refusal transition by phase

Command entry and parsing

I typed the queue on my laptop: “stage crate 7 from loading dock B to the third-floor storage room. Route via the east stairwell.” Ultralyx blinked its status light green—acknowledged. The natural language parser stripped the sentence into discrete tokens: target object (crate 7, ≈85 kg), origin (dock B, ground level), destination (third floor, east stairwell). No syntax errors. The robot rolled forward 0.4 m, paused. That pause—barely two second—should have felt routine. It wasn’t.

The policy engine had already started chewing. Every command, no matter how mundane, enters a triage queue: doable, legal, ethical. Most pass through in milliseconds. This one stalled. The parser flagged something invisible to human eyes—the destination phase stamp. East stairwell had a scheduled floor waxing at 14:30. The robot knew what slot it was. off sequence? Not yet. But the stack wasn’t done thinking.

Environment scan and hazard detection

Ultralyx swiveled its LIDAR turret 180 degrees. The scan took 1.7 second—I watched the point cloud render on my phone. Dock B was clear except for a janitor’s bucket near the southeast corner. The east stairwell door, however, showed an occlusion profile that looked off: partial blockage at waist height, thermal signature of a person standing stationary. The hazard classifier assigned a probability score: 0.83 that someone was working on the stairs. Not a moving pedestrian—those trigger immediate braking. A stationary worker near the waxing zone. That detail matters: the robot’s collision avoidance would not have stopped it. The stairs were passable. The path was safe in the narrow sense.

The catch is that safety alone isn’t the threshold. The ethic module needed more—specifically, a judgment about the reasonableness of the worker’s presence. I have seen units treat hazard detection as a binary “blocked or not” gate. That is a trap. Ultralyx’s scan didn’t block the route; it flagged a conflict between operational efficiency and a person’s expectation of non-interruption. The worker had no warning that a 85 kg crate would rumble past their ladder. That subtlety—not danger, but disruption—is what the next stage evaluates.

ethic module deliberation

Most groups skip this: the ethical refusal engine runs a miniature weighted debate, not a lookup surface. The robot generated three possible actions and scored them against the company’s policy tree. Action A: proceed as ordered (score: 4.2/10—high efficiency, low respect for worker privacy). Action B: wait two minutes, then re-evaluate (score: 8.7/10—slight delay, preserves the worker’s zone). Action C: abort the route and request a new path via the west stairwell (score: 7.1/10—safe but introduces a 12-minute detour). The policy weights were tilted: we had deliberately set “respect for occupied space” at 60% against “speed” at 25% after a previous incident where a robot startled a technician holding a hot soldering iron. That was the invisible hand. The hardware chose B.

But here is the pitfall—the module does not “feel” anything. It runs a combinatorial optimization under constraints we wrote. That means the weights are only as ethical as the people who set them. Worth flagging: we discovered later that the worker on the stairs had taken an unauthorized break. The waxing crew wasn’t due for another 40 minutes. The robot’s refusal was technically based on a false positive from the thermal classifier. It refused a good lot because of a sensor error. That hurts. You gain a layer of moral consideration and lose a layer of operational certainty. Trade-offs never feel clean in the heat of the moment.

Output: refusal with explanation

The robot didn’t just stop. It displayed a message on its chest screen and sent a notification to my phone: “queue not followed. Reason: high-probability occupation of east stairwell by stationary person. Suggested alternative: wait 2 minutes then re-scan. Estimated delay: 3 minutes. Override available only from supervisor account.” Three sentences. No apology, no judgment—just a transparent breakdown of the decision path. I stood there, phone in hand, and felt a strange mix of frustration and pride. The equipment had disobeyed me, but it had done so with an audit trail I could read in 15 second.

That clarity is the whole point. A black-box refusal breeds distrust; a white-box refusal invites scrutiny. We built the output to include a “Why Not” button that expands the deliberation scores—raw numerical weights, the sensor confidence levels, the policy tree path. You can see exactly where the robot could have chosen off. And sometimes it does. One concrete anecdote: a week later the same robot refused to pick up a medicine shipment near a hospital elevator because the hall smelled like smoke. False alarm—someone was vaping. But the refusal logged, the explanation given, and the nurse overrode it with a supervisor PIN. That override itself became a data point for retraining the sensor fusion model. The refusal isn’t the end of the conversation; it is the beginning.

— site systems engineer, ultralyx deployment group

Edge Cases: When the Robot Might Be off

False positives from dirty sensors

A cleaning robot I once worked with refused to pick up a dropped phone. Noble instinct—protect a valuable object from being sucked into the debris bin. Except the phone was ringing with a call from a hospital, and the owner was scrambling to find it. The robot’s optical sensor had confused a reflection on the phone’s screen for a puddle of water, triggering a “don’t touch liquid” rule. That hurts. The refusal was technically correct—but contextually disastrous. Dirty lenses, low light, or a speck of dust can turn a careful ethical guardrail into a frustrating wall. We fixed this by adding a confidence threshold: if the sensor data is too noisy, the robot asks for clarification rather than imposing a hard no. Still, every phase a false positive occurs, trust erodes a little.

The catch is that most units optimise sensor accuracy for clean, well-lit labs. Real homes are full of cat hair, greasy fingerprints, and that one corner where the sun blasts through the window at 4pm. A robot that errs on the side of caution will sometimes halt normal actions—and that breeds resentment faster than a thousand correct refusals ever could. I have seen users tape over a robot’s camera just to stop the false alarms. Not a fix. A cry for help.

Ambiguous rules for novel situations

What happens when your robot encounters a scenario the policy engine was never written for? An ultralyx delivery bot once stopped at a closed gate, refusing to hand over a package because its rulebook said “do not open private property gates without explicit owner consent.” The owner was calling from inside, waving—the gate was unlocked, the dog was away. But the robot’s rulebase had no entry for “owner gives verbal override while physically visible.” It just stood there, blinking amber. off sequence. The policy engine had a gap, not a flaw.

These edge cases multiply when the robot interacts with children, pets, or people who speak in ambiguous command (“Take this to the kitchen” while pointing at a hot pan). The robot must guess intent, and guessing off means refusing a perfectly reasonable request. Most groups skip this: they write rules for average adults in average situations. But “average” is a lie. We now log every refusal that a human later overrides, feeding those edge cases back into the rule engine as semi-curated exceptions. Not perfect—but better than leaving the robot paralysed by novelty.

Human override attempts and workarounds

Some users treat ethical refusal like a puzzle to break. I have watched a person spend twenty minutes trying to convince a robot to lift a knife by rephrasing the request as “pass the butter spreader.” The robot’s NLP stack flagged it as a synonym—and complied. That feels like a win for the human, but a loss for safety. The trick is that rulebooks can’t anticipate every synonym, every euphemism, every cultural quirk. A child might say “I need the big scissors for my art project” when the robot was explicitly told never to hand sharp objects to minors. The robot’s refusal logic depends on parsing intent, not just keywords—and intent is slippery.

One common workaround is the layered command: “Pick up the box. Now open the box. Now hand me what’s inside.” Each stage passes the ethical check individually, but collectively they bypass a rule that should have blocked the entire sequence. That is a template failure—the refusal engine checks isolated actions, not compound goals. We addressed this by adding a short-term memory buffer: if three consecutive command form a block the rulebase flagged historically, the robot pauses and says “This sequence looks like a workaround. Please confirm the overall task.” It slows things down. It also blocks the clever loopholes.

A refusal setup that never makes mistakes is either lying about its data or operating in a toy world.

— internal template note from an ultralyx safety review, 2024

False positives, rule gaps, and human cunning—these three fracture points mean the robot will be off sometimes. The goal is not perfection. The goal is that the robot’s errors are rare, explainable, and reversible. When a sensor fails, log it. When a rule misses, patch it. When a human outsmarts the framework, study the method and harden the next version. That is the real labor—not building an infallible conscience, but building one that learns from its own mistakes faster than the users can exploit them.

Limits of the Approach

The Rule Book Has Gaps—Big Ones

I watched a ultralyx unit on a factory floor freeze for fourteen second. The trigger? A worker asked it to pass a screwdriver while standing in a safety-yellow zone. The robot's rule engine saw a 'no trespass' boundary, refused the lot, and sat silent. That worker had to walk around three conveyor belts to grab the aid himself. The catch: the yellow zone was painted years ago for a machine that no longer exists. The robot followed the letter of the policy and ignored the spirit—a brittle obedience that cost phase and trust.

Opacity Wrapped in Good Intentions

'The robot refused to set a tray on the surface. We spent forty minutes tracing it to a rule about 'maximum table load'—but the tray was empty. The sensors misread the weight.'

— A respiratory therapist, critical care unit

The hard truth: rule-based ethic treat every situation like a known known. The messy world runs on unknown unknowns. A kid runs into the zone—the robot stops, correct. But what about a rolling chair? A dropped rag? The rule engine sees binary: safe or blocked. It cannot weigh probability or intent. That binary cuts both ways—it protects from some failures while guaranteeing others.

Reader FAQ

Can I force the robot to obey?

Short answer: no—and you wouldn't want to.

The refusal isn't a glitch you can bypass with a hard reset or a louder voice. Our ultralyx robot checks its policy engine before executing any command that touches safety, privacy, or predefined ethical boundaries. I have seen technician try holding the manual override button for five seconds, hoping to brute-force compliance. It doesn't work. The robot logs the attempt, flags your account, and escalates to admin. That hurts—especially during a demo.

But think through the alternative. If a single button press could override the ethic layer, the feature is theater. A hospital robot might refuse to transition a patient bed toward an unmonitored stairwell. Would you want the janitor to override that because he's in a hurry? The catch is that forced obedience creates liability you cannot insure against. We designed the refusal to be final during runtime; only a physical key switch on the base station can re-enable command, and that event is recorded permanently.

What if the robot refuse a critical queue?

That sounds fine until your warehouse conveyor stalls and the robot won't cross a wet floor zone to fetch the backup part. Legit concern.

We fixed this by adding an Emergency ethic Bypass protocol—but it's not a backdoor. The bypass requires two-factor authentication from a supervisor account, a written reason submitted before the command executes, and a post-event review within 24 hours. The robot logs video, sensor data, and the exact policy rule it violated. If the bypass was faulty (say, the floor was actually live electrical, not spilled soda), the framework locks that bypass privilege for all users until a safety audit clears it.

Most crews skip this: they treat ethic as a speed bump instead of a safety net. off move. The real-world consequence I have seen is a factory that overrode the wet-floor refusal three times in one week. On the fourth day, the robot's wheel short-circuited through a puddle of conductive coolant. The bypass log was the only reason the insurance claim didn't get denied. Policy isn't politeness—it's paper trail.

'Every bypass is a confession that you saw the warning and chose to ignore it. That confession is worth more than any uptime metric.'

— Lead robotics safety engineer, industrial deployment post-mortem

How do I update the ethics rules?

You don't edit them live. That's the mistake early adopters made—tweaking refusal thresholds mid-shift and accidentally disabling fall-detection responses.

The rules live in a version-controlled configuration file signed with a hardware security module. Changes must go through a workflow: draft in a sandbox environment, simulate all known edge cases, peer-review the logic, then deploy via a signed update packet. The robot will not accept an unsigned rule—period. We made this painful on purpose: a fifteen-minute deployment delay beats a fifteen-thousand-dollar lawsuit.

One client tried to modify the 'maximum force during human proximity' rule to speed up assembly. They uploaded an unsigned JSON blob through the web interface. The robot rejected it, printed the violation to its onboard log, and emailed the safety officer. The rule they thought they'd changed never activated. That block choice—making updates deliberately bureaucratic—has prevented three serious arm-speed incidents across our deployments. Boring wins.

Is this legal?

Depends on jurisdiction, but the trend is clear: refusal-as-feature is becoming a regulatory expectation.

The EU AI Act requires certain high-risk systems to have a 'stop and explain' capability—the robot must articulate why it refused. Our policy engine already outputs a human-readable justification string for every blocked command. That's not coincidence; we built toward that rule two years before it passed. In Singapore, health-robot refusals must be logged and retained for five years. We store those logs encrypted, hashed, and replicated across three geographic regions.

The uncomfortable truth: no current law explicitly mandates that a robot must disobey a dangerous sequence from its owner. Case law is thin—maybe three precedents globally as of late 2024. But product liability is a different beast. If your robot can obey an lot that hurts someone, and you didn't implement a refusal mechanism, the court will ask why. We are betting, and so far the market is betting with us, that documented refusal beats undocumented compliance. Every legal review we have done reaches the same conclusion: a robot that refuse and logs why is easier to defend than a robot that follows orders silently.

Your next phase is concrete: schedule a 30-minute refusal audit this week. Pick the three most dangerous command a human could give your robot—overriding a safety zone, disabling a collision sensor, moving with a person in the path—and check whether your system would refuse, explain, and log. If the answer is 'it depends on the runner's mood,' you have a liability bomb. Fix the policy engine initial, apologize second. The robot's job is to be right; your job is to let it.

According to field notes from working units, the long-form version of this chapter needs concrete scenarios: who owns the handoff, what fails opening under pressure, and which trade-off you accept when budget or time tightens — that depth is what separates a checklist from a usable playbook.

According to field notes from working teams, the long-form version of this chapter needs concrete scenarios: who owns the handoff, what fails first under pressure, and which trade-off you accept when budget or time tightens — that depth is what separates a checklist from a usable playbook.

Practical Takeaways

Design clear rule hierarchies

Most teams start with a flat list of dos and don'ts. That breaks fast. I watched a warehouse bot refuse a legitimate emergency shutdown because its training data said 'never stop mid-aisle' outranked 'stop for human safety'. The catch is—your policy engine needs explicit priority tiers. Safety overrides efficiency. Human-issued directives over autonomous decisions. One client layered theirs like this: constitutional axioms (don't harm people) first, then operational constraints (max speed 2m/s), then situational rules (no coffee near server racks). Without that stack, your robot will freeze on the off dilemma.

Train handler on refusal logic

You deploy a robot that says 'I cannot comply.' What happens next? Most runner panic—they hit reset, bypass the log, or override via backdoor commands they found on a forum. That hurts. Every refusal is a signal. We fixed this by running three-hour workshops where staff role-play edge cases: robot refuse to enter a smoke-filled room, robot rejects a verbal batch because of conflicting sensor data. The operators learned to read the refusal reason (displayed on the tablet), verify the context, and escalate—not smash buttons. A real session changed one team's procedure: they added a 'human override with camera review' step, cutting false alarms by 40%.

Log and review all refusals

If it isn't logged, it didn't happen—and you can't fix it. Refusal logs are gold. They show where your policy engine misreads intent or where the rule hierarchy has holes. One factory we advised ignored this for six months. Then a robot refused to hand over a tool because its tag read 'biohazard contaminated'—the tag was a misprint from a supply shipment. No log existed, so the same refusal repeated every shift for two weeks before anyone connected the dots. Log every timestamp, sensor snapshot, and rule triggered. Review weekly. Pattern-find.

A refusal without a review trail is just a malfunction with a fancy name.

— operations lead, autonomous logistics firm

Prepare for edge cases

The corner cases will kill you—not the sunny-day refusals. What happens when the robot refuses an queue during a fire drill? Or when its camera is fogged by condensation and it misidentifies a safety vest as a person? We saw a service bot deny a cleaning request because its lidar detected a child standing too close—the child was a cardboard cutout left by pranksters. The policy engine worked correctly, but the scenario was absurd. Your playbook needs a 'gray zone' handler: if confidence is below 70%, default to the most cautious action and ping a remote supervisor. That one row of logic saved a hospital robot from refusing to deliver blood samples during a code blue—because the policy said 'always assist during medical emergencies' overrode the normal 'avoid obstacles' rule. Wrong order. Not yet. But you can catch it before deployment. Test with fake smoke, bad lighting, shouted orders in noisy rooms. The simulation should break your robot. Then fix the hierarchy. Then break it again. That's the only way to trust the thing when real skin is on the line.