What's in a name?
Alignment -- or control -- are we talking about the same thing?
It is Monday, and time for another installment of KRCP.
Ultimately, Brad, I like your suggestion to focus on alignment rather than control, but what you suggest is more precisely what MHC folks are trying to advocate and implement. My use of the term MHC is not necessarily because I think the term captures the relationship we are trying to grasp – rather, there is not a great alternative (for now). You are correct that we need to ditch the term, but to ditch the concept might be premature. We are dancing around the same idea – perhaps alignment is the better alternative.
Allow me to put words in many people's mouths (hopefully accurately). The broader community of experts is also ready to ditch MHC. At this point, it is clear the term is meaningless (pun intended) at properly grasping the human-machine relationship we want to capture. What is the threshold for meaningful? Which human is appropriate to 'control' a system? Is it possible to conceptualize or operationalize control of autonomous systems without compromising the inherent advantage gained in any context? Likely not. And, as you suggest, if we are not going to absorb the benefits of AI on the battlefield, then why go down this path? Let's figure out what, exactly, we are talking about.
AI alignment aims to match machine performance to human intent or goals in a particular context. This is certainly a significant engineering problem. However, in my reading, the MHC people take this point for granted. They assume that any deployed system will have the capacity to reflect the commander's intent or the broader value system embedded in system control. Alignment is assumed. And maybe it should not be, as others have noted – the issue of alignment is far from settled or simple.
The issue for MHC people is misaligned systems. The weight behind a concept like MHC is the importance of intervention if a system fails, malfunctions, or is otherwise misaligned with human intent or goals. This is the crux of it. How much risk are we willing to assume?
In your last post, you suggested we need to accept a certain degree of risk in deploying systems – and, of course, you are right. But it is far easier to take this point than to implement it. Risk calculation and willingness vary from person to person, potentially depending on their ultimate relationship with the system. This is why breaking down different life cycle stages can crack open the 'human touchpoints’ where risk is ultimately calculated and by whom. For example, the developer(s) may have a threshold for risk that will vary amongst each other (or even by the company), which could inform designers' definitions of objective functions or feedback for the system. Similarly, commanders have different risk assessments and need to calibrate the operation based on their personal assumption of risk and leadership style. Finally, operators will certainly have different degrees of risk assumption for any operation for any particular context.
To be sure, there is risk in employing humans for these operations, too. Human autonomy, as Grog and Lunk can attest, is not new. However, we are more comfortable with human autonomy and assuming the risks inherent to human decision-making because it is easy to empathize with the difficulties of the decision made in an armed conflict. After all, ‘to err is human’ – making mistakes is a basic understanding of our humanity. But it is also easier to live with human autonomy and inevitable mistakes (or malice) because there is a system for accountability, which brings me to my next point.
The MHC people often emphasize responsibility and the 'accountability' gap of employing autonomous weapons. One of the primary arguments for MHC is to find a way to determine a pathway for accountability in the (inevitable) circumstance of mistakes or misidentifications made on the battlefield.
To me, the issue of accountability and MHC (and certainly alignment) are separate issues. Even embracing the concept of MHC does not outline a pathway of accountability. As my first post argued, there are numerous points where MHC (or choose your preferred term) is embedded into any autonomous system. The parameters of a human-machine relationship that MHC and AI alignment are trying to demonstrate is better characterized by the technical relationship between the many humans involved and their subsequent influence on machine performance – ideally toward greater alignment. But the accountability for unintended outcomes is related -- and deserving of a separate post. But nevertheless, these issues are far from settled.
While I am risking the wrath of MHC advocates by putting words, and maybe ideas, into their mouths, they are largely getting at the same type of relationship as the alignment folks. We all want to ensure responsible systems that do what they are supposed to and do it better than humans can. There are many ways to embody the relationship between humans and machines, and your suggestion of alignment is one of the better options for that embodiment.
One thing we can agree on — it is time to move on from MHC. Where we go next is up for debate.