Interestingly, this paper can be used as model re: the impossibility of ethical consensus:
https://groups.csail.mit.edu/
The process is inevitable, and it is up
to our generation to point out and solve the *very* deep ethical issues with AI making ANY human-level decisions.
With
the tools described in this paper we can try to do this using logic, not opinion or emotion. No one paid attention to the clear
impossibility of securing UNIX back in the day and look at the trouble, insecurity, and expense that has
caused!
-------
One of my personal heroes, John Nash, addressed an aspect of this very issue we're discussing as we consider the
struggle for privacy.
https://arxiv.org/pdf/1211.
Think
of corporations and government agencies as the virus (albeit with more adaptive tools and an
exponentially faster response ability) requiring each individual to mount a sort of
immune response (which is linear, not exponential) to protect private information.
Individuals are forced to attempt to design an optimal, effective response in order to preserve information they deem private, or essential to protection of their assets which includes agency and freedom of expression. This paper gives guidance in proposing a solution.
"....in “Defensive complexity and the Phylogenetic Conservation of Immune Control” by Carl Bergstrom.
The
main idea in the paper is of coevolutionary struggle between a host's
immune system and invading viruses.
This is of the theme of "arms races"
or "evolutionary arms races" where species compete to evolve faster
than the other (this nomenclature comes directly from political arms
races like in war). There are two sides to this-- one is that you can
try to run faster than your opponent, and the other is that you can try
to slow down your opponent. In the case of Bergstrom's paper, the immune
system slows down the virus.
Roughly,
the virus is trying to crack the immune system's code and the immune
system is trying to prevent this. The virus learns the immune system's
code, then the immune system makes the code more complex, then the virus
learns the new code, and so on. What's peculiar about this case is that
a virus can go through an entire evolutionary process while the host
is, in many ways, the same. So why doesn't the virus just dominate? How
can the immune system keep up?
Bergstrom
et. al. find inspiration in a particular arms race that was happening
in the 1950s involving creating and breaking codes (cryptography).
Quoting a letter written by John Nash (who, in addition to game theory,
worked on cryptography-- you can find the quote in the paper), Bergstrom
connects breaking codes as a human activity and a virus-host
interaction. The idea presented by John Nash is that it is possible for a
code to grow or change in linear speed while its decryption grows in
exponential speed.
In
other words, the immune system in a host cannot compete with a virus in
terms of evolutionary speed (the virus is much faster). However, the
immune system can slow down the virus by moving slowly itself (linear
speed) and causing the problem faced by the fast-moving virus (cracking
the code) to grow exponentially."
(From an email sent to members of the Complexity Explorer, made by SFI professor, Prof: Santiago)
