My mom emailed me to ask about machine interpretation of laws (i.e. software judges and lawyers). I replied:
About fifteen years ago, I got into an argument about this in the context of a video game. The idea was that the game would have guards which would punish you if you did something illegal. The problem was that the game supported building machines, and Turing proved that no computer program can, in the general case, figure out what a machine will do. So if you built a sufficiently complicated machine, either you would get away with murder or be wrongly convicted.
Humans, of course, have the same problem, but we pretend not to.
Self-driving cars are a relatively easy case: only a monster would program in anything but pure consequentialism. Especially since, as a practical matter, self-driving cars are likely to be much safer than human-driven cars.
Before computers can replace lawyers in the the general case, they will have to understand human language (in order to interpret contracts, laws, and precedents). This is the single hardest problem in computer science. We have made almost no progress here. If we can solve this problem, we have achieved general intelligence, and within a year or two nobody will have a job.
There are areas now where computers could assist us. For instance, let’s say we’re considering Solomon’s case: which of two women is the mother of a child. Imagine that one woman is has brown hair and brown eyes, and the other has blonde hair and blue eyes. The child has brown hair and blue eyes. (We don’t have a DNA test for some reason). How do we combine our two pieces of data? There’s a fairly simple formula that does it precisely (given some odds of the various inputs). But right now, in order for a jury to use this formula, we need to call a statistician to the stand. And if the jury finds one of the women twice as credible, they don’t have a way to incorporate that data, even though the formula could easily do it. This would be a good first step, but for political reasons it would be shocking to me if we took it.
I should have added: This thing where the law is designed to be interpreted by humans is famously hard for programmers to grasp. I had forgotten that the thing where computers are bad at interpreting language (yet good at following rules) is just as hard for lawyers. I think the lawyers’ confusion is part of what’s behind the recent encryption debate (roughly 40% members of congress have law degrees).