A ranking system does not decide what is true; it decides what is seen, which for most purposes is the same thing.
Its designers usually describe it as on the ground that it applies the same rule to everyone.
A rule applied to unequal inputs does not produce a output, and everyone knows this outside the context of software.
The difficulty is not that ranking is biased but that responsibility for the ranking is genuinely hard to locate.
No engineer chose the order of any particular list; each chose a feature, a or a training set.
The output is a consequence of those choices and is not contained in any of them.
This is a familiar structure in other fields, where it is handled by attaching duties to the organisation rather than the individual.
An airline is responsible for a crash without any employee having decided to crash.
Software firms have resisted this for reasons that are commercial rather than conceptual.
The most common defence is that the system is too complex for its behaviour to be predicted.
Complexity is a reason for testing rather than a reason for , and it is the only industry in which it has worked as the latter.
offers a route that does not require anyone to explain the internals.
An auditor supplies inputs and records outputs, exactly as a laboratory tests a car it has not designed.
Such tests reveal outcomes without any claim about intention, which is what makes them usable in law.
Firms object that supplying test access would expose them to reverse engineering.
The objection is real and it is an argument about how access is granted, not about whether it is.
Every regulated industry has solved the equivalent problem with confidentiality undertakings and testers.
The remaining question is not technical and has never been technical, which is why fifteen years of technical answers have settled nothing.