When Your Robot's Brain Can't Keep Up: Field Notes for Busy Engineers

You've got a robot that works on the bench. Now take it to the warehouse floor — and watch it drift, stall, or crash into a pallet it's seen a hundred times. That gap between lab success and field failure is where most robotics projects die. This isn't a close look into control theory; it's a collection of field notes for engineers who need their robots to survive contact with the real world.

We'll talk about the things conference papers skip: why your LIDAR sees ghosts near loading docks, how thermal expansion messes up calibration, and why the best path planner can't save you from a flaky motor encoder. You'll get patterns that hold up under pressure, anti-patterns that every team falls into at least once, and a frank look at when you should turn off the fancy autonomy and just drive the thing manually.

Field Context: Where Robotics Meets Reality

Perception mismatches in semi-structured environments

I watched a $30,000 picking arm freeze for three seconds because morning frost on a warehouse skylight shifted the color temperature by 1200K. The team had trained their vision model on sunny-afternoon data. That frost? Not in any training set. The arm dropped a crate of glass vials. The catch is that your robot lives in a world that changes when you aren't looking—dust on lenses, a worn reflective strip on the floor, a new safety vest color on a Tuesday. What worked in the lab's controlled lighting bay falls apart under sodium-vapor lamps at 6 AM. Most teams skip this: they validate perception accuracy against static test images, not against the actual lighting gradient across a shift. That hurts.

Latency chains from sensor to actuator

The simulation says your control loop runs at 100 Hz. Reality says the depth camera buffers frames for 37 milliseconds, the Ethernet switch introduces a random jitter of 12 ms, and the motor controller applies a 20 ms low-pass filter you forgot to configure. Wrong order. You're now commanding a position based on data the robot occupied two control cycles ago. I have seen an autonomous floor scrubber overshoot a charging dock by half a meter—not because the path planner failed, but because the latency stack turned a gentle deceleration into a late panic stop. The repair bill was three service calls to swap a bent actuator arm. Worth flagging—most industrial fieldbuses (EtherCAT, CAN FD) guarantee deterministic latency only if every node in the chain plays by the rules. One misconfigured gateway and your 1 kHz loop becomes a drunk walk.

“The robot’s localization looked perfect on the bench. On the factory floor it drifted 8 cm in four minutes. Nobody could explain it until we noticed the steel shelving unit vibrating from a nearby compressor.”

— Field engineer, logistics automation deployment, after swapping three IMUs unnecessarily

Real-world failure modes not in simulation

What usually breaks first is not the algorithm—it's the assumption that the environment stays still long enough for the algorithm to converge. A mobile manipulator I debugged last year kept losing its map in a corridor where a forklift driver routinely parked a pallet two inches outside the taught safety corridor. The lidar saw a new wall. The particle filter collapsed. The robot sat there, blinking a red LED, until a human nudged the pallet. That's not a software bug. That's a mismatch between the tidy static world you model and the messy, half-sloppy world your robot has to survive. The anti-pattern is treating simulation as a proof of correctness instead of a hypothesis generator. Simulation told you the grasp planner works. The field showed you the part surface was oily from a mold release agent the supplier changed without telling anyone. How do you model that in Gazebo? You don't. You build a watchdog that detects when reality has crossed a threshold your sim never captured. Most teams don't. Then they revert.

Foundations Readers Confuse: State Estimation vs. Filtering

Why People Call Every State Estimator a 'Kalman Filter'

I once watched a team spend two weeks debugging a wheeled robot that kept veering left. Every conversation started with “our Kalman filter must be broken.” It wasn’t a Kalman filter. It was a complimentary filter slapped together from a Stack Overflow snippet—and worse, they’d miswired the heading update so yaw drift accumulated instead of correcting. The filter label stuck because it sounds authoritative. Most engineers reach for “Kalman filter” the way non-cooks reach for “garlic”—it improves almost anything, but too much ruins the dish. The real problem: people confuse any sensor fusion with the specific linear-Gaussian assumptions a Kalman filter demands. Your IMU bias won’t read the memo. Neither will your slip-prone wheels.

The Difference Between Filtering and Smoothing—and Why Your Robot Cares

Filtering is real-time. Smoothing is hindsight. That distinction sounds academic until your robot enters a dark corridor and the EKF collapses because it can’t peek backward at the data it just saw. Smooth algorithms—like Rauch-Tung-Striebel—recompute the whole trajectory offline. Great for post-mortems. Useless when the motor controller needs a torque command now. Most teams filter in deployment and smooth only during debugging. The catch: they often use smoothing tuning parameters on their live filter, which gives them brittle performance. I have seen a six-month development cycle stall because the lead engineer insisted on forward-backward smoothing in real time. It doesn’t work. The trade-off is plain—accept noisier estimates during operation or accept delayed, clean estimates after the fact. Choose your pain.

What usually breaks first is the boundary between filtering and prediction. A standard EKF assumes you can linearize around the current state. That’s fine on flat concrete. On loose gravel or a loading dock ramp—where dynamics shift suddenly—the linearization error spikes and your filter diverges before your watchdog can flag it. The fix isn’t a better filter; it’s an observer that explicitly models the nonlinearity. But observers require knowing your plant dynamics well enough to prove convergence. Few teams have the math depth for that. So they band-aid with process noise inflation. That works until it doesn’t.

When You Actually Need an Observer (and When a Filter Is Fine)

Filters estimate state from noisy measurements. Observers reconstruct unmeasured internal states—like motor temperature or joint stiffness—using a dynamic model. The line blurs when people say “observer” for a simple low-pass on encoder velocity. That’s not an observer; that’s a filter with a fancy name. You need an observer when failure mode analysis says we can't directly sense the variable that kills the robot. I fixed a gripper that kept crushing boxes—the force sensor was too slow to prevent over-torque. A sliding-mode observer on the motor current gave us a virtual torque signal within 2 milliseconds. Wrong order on that build: we started with a Kalman filter, spent three weeks tuning it, then switched to a Luenberger observer and solved it in two days. The lesson? Match the tool to the physics, not the hype.

‘We used a particle filter because it sounds sexier than a complementary filter. The robot still crashed into a wall. Turns out sampling 10,000 particles doesn’t fix a bad odometry model.’

— Anonymous lead engineer, field repair log, 2023

Patterns That Usually Work: Hybrid Planners and Watchdogs

Combining global and local planners effectively

Your robot has a map and a destination — that's the easy part. The hard part is getting there without scraping a wall or freezing mid-crosswalk. I have seen teams spend three months tuning a global planner only to watch the robot panic at a delivery dock because the local planner couldn't handle a pallet that was moved two feet left. The pattern that actually works: separate concerns brutally. Let the global planner (A*, Hybrid A*, or lattice-based) produce a coarse path that respects the map but ignores dynamic clutter. Then hand that to a local planner — usually Timed Elastic Band or Model Predictive Control — that replans at 10–50 Hz and can swerve around a stray box or a person stepping out. The catch: the handoff point. If the local planner gets too much leash, it will cut corners into walls. If too little, it stops for every dust spec. We fixed this by clamping the local planner's deviation to 0.3 meters from the global path — a number we found by literally taping tape lines on the floor and counting failures.

Most teams skip this: you need a recovery behavior when the planners disagree. After three seconds of local planner cost spikes, drop into a pure-pursuit mode that just follows the global path at 0.2 m/s. Not pretty. But it beats a stalled robot blocking a warehouse aisle. Worth flagging — that recovery transition must be logged with millisecond timestamps, or you will debug phantom stalls for weeks.

Not every robotics checklist earns its ink.

Not every robotics checklist earns its ink.

Watchdog timers for safety-critical actions

A robot arm that keeps pressing after the limit switch fires is no longer a robot — it's a hydraulic press looking for something to destroy. Watchdog timers are your cheapest insurance, but only if wired into the non-ROS layer. I once consulted on a lab robot that had a software-level watchdog in the planner node — which crashed along with the rest of the stack when a memory leak hit. The arm swung through a partition. That hurts. The pattern that survives field conditions: a hardware watchdog on a dedicated microcontroller, watching a heartbeat pin from the main computer. If the heartbeat stops for 200 ms, the watchdog kills motor power and engages brakes. No exceptions. The trick is setting the timeout — too tight and false positives cripple you during high-CPU map loading; too loose and the arm has time to finish its destructive arc. 200 ms is a starting point; we dialed it to 350 ms after noticing that one team's SLAM node occasionally hiccupped for 290 ms during loop closures.

'The watchdog isn't there to protect the robot. It's there to protect the person who forgot the watchdog existed.'

— field engineer, after a near-miss at an automotive assembly line

Graceful degradation via fallback behaviors

Your lidar dies. Now what? Most teams write one line — exit(1) — and call it a safety case. That's not a plan. The anti-pattern is a binary alive-or-dead check that stops the robot cold. Better pattern: cascade down through sensory fidelity. If lidar fails, drop to a bumper-based follow-wall behavior using only bump sensors and wheel encoders. If the bumper fails, stop and beacon — broadcast a continuous 'stuck' signal on a dedicated safety channel. We deployed this on a fleet of nine warehouse robots and logged 47 lidar dropouts over six months. Thirty-nine of those were recovered by the fallback behavior without human intervention. The remaining eight required a tech to walk over — but no collisions, no injuries. The key: each fallback must be tested in simulation and on the actual floor, because bumper thresholds that work on carpet fail on polished concrete. One rhetorical question: how many fallback layers do you need? Three. Full autonomy, degraded mode, and safe stop. Anything beyond that creates a testing matrix that no team can maintain.

Anti-Patterns and Why Teams Revert: Hard-Coded Thresholds and Single-Threaded Control

Hard-coded speed limits that fail on slopes

Demo day goes flawlessly. Robot glides at 0.8 m/s, stops precisely, everyone claps. Two weeks later it rolls off a loading dock. What changed? A 3-degree ramp the client didn't mention. That hard-coded max_velocity = 0.8 looked clean in the header file—no dynamic scaling, no torque feedback, just a number. On flat ground it's fine. On a downward slope with a loaded basket, gravity adds 0.4 m/s before the motor controller even blinks. The PID loop saturates, the watchdog timer expires, and you get a crash report at 2 AM. I have fixed this exact bug three times in five years. Teams keep hard-coding because it ships today, because the PM doesn't ask about slopes, because the test track is flat. But production terrain laughs at your magic numbers.

The real cost isn't the broken robot—it's the meeting afterward where someone argues for adding more thresholds. Lower the speed limit to 0.5. Add a slope detector. Hard-code a separate limit for downhill. That cascade of brittle conditionals becomes a tangled if-else pyramid that nobody touches except the one engineer who wrote it. Worth flagging—the same pattern kills autonomous forklifts in warehouses when floor wax changes friction coefficients. A single threshold works until it doesn't.

Single-threaded nodes that block sensor data

One loop. One thread. One while(1) that handles planning, then localization, then motor commands. Beautiful in its simplicity. Then the LiDAR callback fires while the planner is crunching a path through a narrow doorway. The sensor data sits in a queue growing stale while the CPU churns. By the time the planner finishes, the robot has moved 30 cm—data is now garbage. The robot corrects, recalculates, and the whole thing starts oscillating. That hurts.

Most teams skip this: a blocking call in the main thread doesn't just slow down one function—it starves every sensor publisher. I watched a four-week deadline turn into a rewrite because nobody caught a sleep(100) in a vendor's SDK callback. The fix wasn't fancy—spin the sensor driver on a separate thread, buffer the last three readings, let the planner grab the freshest one. But under deadline pressure, threading feels like scope creep. So teams revert: merge everything into one node, add a ros::spinOnce() and pray. Wrong order. That prayer fails when the motor controller blocks for 50 ms and the IMU drops a packet.

'Single-threaded control convinced us we could debug anything in five minutes. Then we spent two weeks chasing a race condition that only appeared on Tuesdays.'

— lead engineer, mobile robotics startup, after a demo failure

Ignoring thermal drift in calibration

Calibrate in the morning, ship in the afternoon. The IMU bias looked perfect at 22 °C. At 38 °C the gyro drifts 0.15 °/s—enough that a 30-second spin accumulates 4.5 degrees of error. Your state estimator thinks the robot is turning left when it's actually pointing straight. The controller over-corrects, the odometry diverges, and suddenly the robot is 2 meters off course. Not a software bug—a physics bug that software can't patch without a thermal model.

The anti-pattern here is elegant denial: teams embed calibration params in a YAML file, run a warm-up routine, and assume that's enough. It's not. Thermal drift is nonlinear, sensor-dependent, and sneaky—it doesn't spike, it oozes. Production robots on concrete floors in direct sunlight cook differently than lab units on carpet. I have seen teams burn three sprints trying to tune around a problem that required one thermocouple and a lookup table. The trade-off: adding temperature compensation costs maybe two days of testing, but the deadline says ship. So they revert to the old cal file, cross their fingers, and tell support to "make sure the robot cools down between runs." That's not engineering—that's superstition with a Jira ticket.

Maintenance, Drift, and Long-Term Costs: The Hidden Tax of Robotics

Model drift from component wear

Six months in, the same robot that could pick a peg from a bin with 98% reliability now misses every third attempt. The encoders drifted. The gripper jaws stretched by 0.2mm. Nobody changed the control model because nobody noticed the change happened incrementally. That's the hidden tax—you pay it in missed cycles, re-grasps, and eventually a full afternoon of parameter re-tuning. Most teams skip this: they treat the robot's brain as static software. But the hardware underneath is mechanical, and mechanical things wander. Belts loosen. Bearings develop play. A friction coefficient you tuned for dry aluminum becomes useless once shop-floor humidity hits 70%. The fix is not smarter code—it's scheduled re-identification of those physical parameters. Run a five-second calibration sequence every Friday. Log the results. If the Kalman gain creeps past a threshold you set last quarter, flag it before the seam blows out.

I have seen a fleet of six arms degrade so slowly that the operators blamed each other for bad weld paths. The actual culprit? Worn encoder belts introducing 0.3 degrees of periodic error. That tiny bias took four months to accumulate. By then, the maintenance cost to replace belts plus re-certify each arm's accuracy exceeded the original purchase price of one unit. Worth flagging—the drift itself is not the problem. The problem is the gap between when drift begins and when you detect it.

Honestly — most robotics posts skip this.

Honestly — most robotics posts skip this.

Calibration decay and re-calibration schedules

Force-torque sensors are notorious. You zero them at installation, and six weeks later the reading at rest shows 1.2 N where zero should be. That's calibration decay—systematic, predictable, and routinely ignored until a robot crushes a part because it thought it was applying half the actual force. The catch is that re-calibration is boring. It takes a technician offline, requires a known-weight artifact, and produces a single number that nobody writes down. So teams defer it. Then they patch the control logic to add a fudge factor. Then they patch the patch. A year later, the software stack contains six layers of empirical offsets, none of which match the physical reality.

What actually works? Hard schedules, not smart heuristics. Every 500 operating hours, run a six-point gravimetric check. Automate the logging—if the raw bias exceeds 0.5% of full scale, push an alert. That sounds fine until the production manager says they can't stop the line for calibration. Fair. So run it during changeover. The cost of thirty minutes of idle time is lower than the cost of one scrapped assembly line from a force error.

Logging overhead and debugging debt

Robots generate logs like teenagers generate text messages—constant, repetitive, and mostly noise. A typical ROS 2 bag file for an eight-hour shift can hit 40 GB. Nobody reads it. But teams keep logging everything because "what if we need it later?" That hoarding has a real cost: storage, transfer time, and the cognitive load of searching through terabytes when something actually breaks. The debugging debt grows every shift.

'We reduced our log volume by 80% and found bugs faster. The trick was deleting nine out of ten topics and keeping only the ones that had ever helped us fix a real failure.'

— A clinical nurse, infusion therapy unit

— lead engineer at a palletizing startup, after a painful migration

Most teams skip this: audit your log schema once per quarter. Drop topics nobody has queried in the last three months. Archive the rest to cold storage. That frees up disk for the two or three signals that actually matter—motor current, joint position error, and watchdog latency. The rest is just noise you pay to store.

One concrete action for this week: pick the robot that runs your highest-volume task. Check its last calibration date. If it's older than three months, schedule a re-zero. Then look at its log directory. If the oldest unexamined bag is from last year, delete it. That's three minutes of work that will save you a day of debugging later.

When Not to Use This Approach: Simple Machines Don't Need Complex Autonomy

Finite State Machines vs. Behavior Trees: When Less State Is More

A startup I visited had strapped a full behavior-tree framework — twelve nodes, reactive fallbacks, parallel subtrees — onto a motorized cart that moved boxes between two fixed shelves. The deployment took three months. The behavior tree never ran a full shift without a node timeout. They ripped it out in two days, replaced it with a four-state finite state machine, and shipped. That pattern repeats constantly. Simple machines — pick-and-place arms with one gripper, conveyor followers, single-axis gantries — don't need the compositional power of behavior trees. The overhead of blackboard variables, tick rates, and node lifecycles adds failure modes that a few boolean flags and a timer interrupt handle better. Finite state machines are flat, predictable, and debuggable with a pencil. Behavior trees shine when the robot must choose among many strategies — search, approach, retry, give up — and those strategies change at runtime. If your robot's possible states fit on a napkin, stop building a tree.

When Manual Teleop Beats Autonomous Navigation

I worked on a robot that navigated a warehouse aisle too narrow for two-way traffic. The autonomy stack — costmaps, global planner, recovery behaviors — handled it well until a pallet sat two inches out of place. Then the planner found no path, stalled, called for help, and needed a manual override anyway. The fix? We mapped three critical pinch points and let a human teleop through them. Not elegant. But the robot ran twenty-two hours a day instead of eighteen.

The catch is ego. Engineers hate admitting that a joystick beats their SLAM graph. Yet in constrained, repetitive spaces — a single corridor, a dead-end aisle, a docking station less than 1.5× the robot width — manual teleop or simple waypoint replay with guarded moves costs less, fails less often, and requires no tuning. Autonomy is a tool, not a badge of honor. Ask: would the operator rather intervene twice a shift for ten seconds each, or debug a global planner for two days?

Cost-Benefit of Adding Sensors vs. Improving Algorithms

Most teams reach for a lidar before they clean their data pipeline. Worth flagging — a $500 IMU won't fix a filter fed on corrupted timestamps. The trade-off is stark: sensors add hardware cost, wiring, failure points, and calibration drift. Algorithms cost development time but compound across the entire fleet. A single outlier-rejection improvement in your state estimator can save every robot, every deployment.

“We bolted on a depth camera and spent six weeks fixing the shadows it cast in our IR grid. The old bump sensor worked fine.”

— Field engineer, agricultural robotics retrofit

Not every robotics checklist earns its ink.

Not every robotics checklist earns its ink.

That hurts because it's true. Before adding a sensor, ask: what specific failure mode does it close? Can you close that same gap with better filtering, a more conservative velocity limit, or a mechanical hard stop? If the environment is structured — painted lanes, defined floors, known obstacle positions — a simple bump-switch or a single time-of-flight sensor often outperforms a multi-modal fusion stack that the team doesn't have the expertise to maintain. The hidden cost isn't the sensor. It's the month you lose re-tuning the fusion model when the floor paint changes. Simple machines, simple senses.

Open Questions and FAQ: PID Tuning in the Wild and Simulation Fidelity

How to PID tune when the environment changes daily

You dial in gains on a calm Tuesday morning. Robot tracks perfectly. Wednesday afternoon — wind, direct sun heating the floor, a fresh coat of sealant on the warehouse concrete. Suddenly the same gains produce oscillation that sounds like a washing machine full of bolts. I have watched teams burn two weeks chasing a single PID set that works everywhere. It doesn't exist. The catch is that most PID tuning advice assumes a stationary plant — your robot's plant is a greasy caster wheel on a sloped ramp at 3 PM on a Friday.

Practical fix: build a gain-scheduling lookup table keyed to sensor-derived state. Motor current + IMU temperature + floor reflectance from a downward-facing time-of-flight sensor — three cheap signals that hint at what the surface is doing. Then clamp aggressively. A P-term that works on polished epoxy at 60% duty cycle will destroy your mechanism on rubber matting. We fixed this by logging the last known 'good trajectory' RMS error and reverting gains automatically if error spiked above a threshold for 200ms. That hurts less than a crashed end effector. Worth flagging—our best field fix was a simple watchdog that switched to a conservative 'limp home' gain set if the error trace looked bang-bang for more than half a second. Not elegant. Kept the robot moving.

One unresolved debate: whether to tune for worst-case surface or tune for nominal and let the controller saturate gracefully. I lean toward nominal with saturation limits that feel stingy — a robot that moves 10% slower in bad conditions is still a robot that finishes its shift.

How much simulation fidelity is enough before field testing

Most teams over-simulate. They tune friction coefficients to five decimal places, model flex in the coupling, run Monte Carlo on payload distribution. Then the robot hits the field and the biggest problem is a loose cable snagging on a shelf bracket. Simulation fidelity has a steep diminishing return curve — past a certain point you're optimizing for the wrong reality.

The heuristic that served us: simulate until you can reproduce three failure modes consistently. Lost odometry? Sim it. Stuck on a threshold? Sim it. Joint limit overshoot? Sim it. Once you have those three dialed, ship the robot. You will discover the other fifteen bugs in the first hour of real operation anyway.

'We spent six weeks polishing a simulation that ran at 400x real-time. The actual robot failed in the first three minutes because a QR code was mounted upside down.'

— Lead controls engineer, AMR startup, post-mortem debrief

That said, there is one simulation layer worth over-investing in: sensor noise injection. Teams that simulate perfect lidar scans and ideal odometry always panic when the real robot glitches. Inject realistic dropout patterns, occasional multi-path reflections, encoder jitter. That costs almost nothing to add and saves you the shock of seeing your localization jump three meters because of a sunbeam on the floor.

What to do when your robot keeps losing localization

You see it in the logs: a sudden pose jump, then the particle filter collapses into a cloud of despair. What usually breaks first is the covariance management — the filter believes it's certain when it should not. Most teams skip this: monitor the ratio of estimated covariance to actual residual error. When that ratio drops below 1.0 for more than two consecutive updates, your filter is overconfident. The fix is brutal but reliable — artificially inflate the process noise for 10 cycles to force re-dispersion. Ugly. Works.

Another pattern: losing localization at the same physical spot every time. I have seen this happen at reflective pillars, glass doors, featureless corridors. The engineering reflex is to add more sensors. That's expensive and slow. Instead, mark that zone as a 'trust-reduced area' and drop the update rate from 50 Hz to 10 Hz — let the dead-reckoning dominate until the robot exits the bad zone. Simple logic change. No hardware cost. One team I worked with painted a black stripe on the floor at the trouble spot as a visual cue for the human operators. Not a technical fix, but it stopped people from blaming the software.

Open question: should you use a second, independent localization source as a sanity check, or just accept that localization will fail occasionally and design the mission planner to recover gracefully? I lean toward graceful recovery — a watchdog that reboots the filter stack costs less than a second lidar or an RTK GPS. Try it this week: log your localization confidence alongside your pose, set a flag when confidence drops below a threshold, and force a re-initialization routine that drives the robot to the last known 'certain' waypoint. That one experiment will tell you more about your system than a month of simulation tuning.

Summary and Next Experiments: Three Things to Try This Week

Log your first field failure and trace the root cause

Pick the last crash or weird behavior your robot exhibited. Not the dramatic one where it drove off a loading dock — just the most recent glitch that made you mutter 'not again.' Write it down. Three columns: what the sensors reported, what the planner decided, what actually happened. Most teams skip this step because they're already neck-deep in the next sprint. That's exactly why you should do it. The gap between sensor reading and motor command is rarely empty — it's packed with assumption layers, coordinate transforms, and filter lag that nobody documented. I once traced a six-second oscillation in a differential drive back to a single stale odometry timestamp buried in EKF logs. The fix took ten minutes. The hunt took two days. Don't be that person.

Add an anomaly detector for motor current

A simple threshold on motor current — say, 30% above nominal for more than 200 ms — catches more field failures than any state estimator I've seen. The catch is: hard-code that threshold and you'll get false alarms every time the robot climbs a ramp or bumps a curb. Instead, let the detector learn the baseline during the first minute of operation. Rolling average plus standard deviation. When current spikes outside three sigma, log the context and let the watchdog decide. Worth flagging — this isn't fancy. It's a deque of last 600 samples and two lines of arithmetic. But it catches stalled wheels, jammed actuators, and semi-loose belts before they cascade into a full autonomy reset. What usually breaks first is the motor driver, not the planner. Watch the current, not the plan.

We spent three months optimizing our path planner. The robot died because a gearbox screw backed out by 1.2 mm.

— field engineer, autonomous floor-scrubber deployment

Benchmark your control loop latency end-to-end

Don't trust the theoretical cycle time printed in your RTOS datasheet. Measure from ADC sample to PWM update — including every transform, filter, and safety check between. Use a GPIO pin toggled at measurement start and end, then hook up an oscilloscope or a cheap logic analyzer. You'll almost certainly find more jitter than expected. A colleague's team discovered their '1 kHz' control loop actually ran at 340 Hz on average, with occasional 80 ms gaps when the serial driver flushed its buffer. That hurts. The fix was moving the IMU interrupt to a higher priority and disabling prints during control cycles. Simple. But they hadn't measured because the numbers looked fine on paper. One afternoon with a Saleae changed their entire architecture. Try it this week — before your next field test, not after.