While some AI models can be challenging to interpret, good design and systems can—in some cases—make AI supported decisions more transparent and fair than those made by humans alone. Governments should consider leveraging their open government plans to release relevant, redacted open source code, data and/or reports from AI systems that are operationalized and include performance information. Useful public information on an AI system could include the following items, depending on how it is being applied and the sensitivity of the system:
- The system’s purpose, governance, and intended outputs
- Its performance against outcome metrics (e.g., accuracy, root-mean squared error, cost function, etc.) for different constituencies
- Description of the training data and how it was sourced or engineered
- Documented open source code and methodology documents which would explain how data fields and algorithms were used, and how models were tuned
- Distribution and accuracy of outputs among constituent groups of interest (e.g., geography, demographic, gender, income, etc.)
- Importance of each variable for the model
- The variables that contributed most to a particular decision for a citizen (which is possible, even in complex neural networks)
This type of public reporting on performance and equity is not in place for many government services and decisions today, and AI can be an impetus to improve transparency. As more governments implement AI systems and release such information, we expect that standards and tools to make such reporting easier will emerge from a global ecosystem.
Bias and Equity
Algorithms have the risk of perpetuating our existing biases if designed inappropriately (e.g., using training data based on past discriminatory human decisions or proxying a marginalized group). Human decisions are influenced by conscious and unconscious bias that may not be traceable. For this reason, it is challenging to identify how discrimination may factor into any given human-made decision. While these same biases can manifest in the design and/or data used by AI systems, they can be identified and mitigated to augment human judgment in appropriate applications.
AI systems open the door to embed definitions of fairness and codified policies such as anti-discrimination requirements in them, and can report on or even optimize constituent outcomes within such requirements. Codifying and reporting on these requirements could be part of policy development, procurement, and consultation processes, and help make progress toward equity and efficiency more measurable and transparent.
AI-enabled systems can also make it possible to keep individual information more secure. Greater centralization of data stores do present a risk of misuse at scale, however, they also allow governments to professionalize and standardize cybersecurity and privacy practices. A mechanism such as differential privacy is more effective with larger data stores, and can be considered as a means to safeguard integrity and foster public trust in government systems while enabling more access to data for evidence-based decision making. Such an approach can shift AI systems from privacy risks to solutions.
AI can amplify or mitigate our status quo challenges with transparency, bias, and privacy in government services and decision-making. We have proposed initial ideas on how thoughtful governance and design can mitigate these challenges and improve upon the status quo, however, more thinking and solutions are necessary so that governments can ethically reap the benefits of AI for their constituents.
If you would like to learn more about this emerging technology, we encourage you to read our introduction to AI for policymakers. To participate in, or learn more about our work in this area, please get in touch at firstname.lastname@example.org.