Consider the case of a hospital in Tennessee. Doctors deployed a cutting-edge black box AI to identify patients at risk of pneumonia. The AI was remarkably accurate—except for one glitch. It consistently sent asthmatics home, labeling them "low risk."
The machine was brilliant. It was also dangerously stupid. And no one could see the error until a person almost died. This opacity is crashing headlong into Western jurisprudence. The 14th Amendment guarantees "equal protection under the law." But what happens when a judge uses a black box algorithm to set bail? If the algorithm is biased against a zip code, how can a defense attorney cross-examine it? blackbox
The old black box—the flight recorder—was built to survive a fire. It tells us exactly why we crashed. Consider the case of a hospital in Tennessee