Ethical Application of Artificial Intelligence Framework The Ethical Application of AI Index (EAAI) framework allows for an establishment of a consistent measure upon which one may qualify and quantify components used to create, operate, and improve AI capabilities. It provides the means to monitor and measure the influence AI has on systems throughout the entire lifecycle. Each component should be reviewed and scored based on its applicable indicators and resulting implications. The indicators quantify the quality of the characteristics and the implications quantify the impact. These components will deliver an overall score that should be used to evaluate the ethics of an AI system, which should continually be monitored and checked over time. This paper and its index are intended to be an advisory framework to highlight that humans are ultimately responsible for the ethical application of Artificial Intelligence (AI) solutions. By monitoring and measuring critical elements of AI throughout the lifecycle of development, implementation, and operations, one can assess an AI application's level of credibility, and thus, the level of confidence to place in that instance of this rapidly evolving technology. This confidence can be demonstrated through an index that incorporates five core parameters underpinning the impact of AI on those systems: Bias, Fairness, Transparency, Responsibility, and Interpretability. American Council for Technology-Industry Advisory CouncilACT-IAC_39cf219a-0987-11e0-9947-105e7a64ea2aThe American Council for Technology-Industry Advisory Council (ACT-IAC) is a non-profit educational organization established to accelerate government mission outcomes through collaboration, leadership and education. ACT-IAC provides a unique, objective, and trusted forum where government and industry executives are working together to improve public services and agency operations through the use of technology. ACT-IAC contributes to better communication between government and industry, collaborative and innovative problem solving, and a more professional and qualified workforce. Emerging Technology Community of InterestACT-IAC, through the Emerging Technology Community of Interest, formed an Artificial Intelligence Working Group to give voice to and provide an authoritative resource for government agencies looking to understand and incorporate AI/ML technology and functionality into their organizations. This working group includes government and industry thought leaders incubating government use cases. The ACT-IAC Emerging Technology Community of Interest (ET COI) mission is to provide an energetic, collaborative consortium comprised of leading practitioners in data science, technology, and research, engaged with industry, academia, and public officials and executives focused on emerging and leading technologies which transform public sector capabilities.Artificial Intelligence Working Group This paper was written by a consortium of government and industry. The organizational affiliations of the authors and contributors are included for information purposes only. The views expressed in this document do not necessarily represent the official views of the individuals and organizations that participated in its development. Adelaide O'BrienIDC Government InsightsAlex ReboUnited States GovernmentAnsgar KoeneEYAshley CasovanAi-GlobalChakib ChraibiNTIS-Department of CommerceDavid HernandezRiva SolutionsDon LovettDC GovernmentDr. Jaunita StewartDepartment of the Army, Office of the Deputy Assistant Secretary of the Army for Defense Exports and CooperationEric EskamU.S. General Services Administration, IT Category OfficeFatima AkhtarIBMFrederic de VaulxPrometheus Computing LLCG. Hussein BasariaThe Maven GroupGeorge A. Tilesch, JD, MAGlobal AI Expert, Co-Author: Between Brains - Taking back our Future in the AI AgeHenry JiaExcella ConsultingJeremy WoodMillennium Challenge CorporationJessica DavisMicrosoftJignesh ShahS2Alliance IncJoe PaivaHire VueJune LauNational Institute of Standards and TechnologyKen FarberAIexpertiseMarc WineDepartment of Veterans AffairsMichael BruceLeidosMichelle WhiteU.S. General Services Administration, IT Category OfficeMike RiceCornerStoneNeil (Anil) ChaudhryU.S. General Services Administration, Technology Transformation OfficeNevin TaylorNational Artificial Intelligence InstituteOrlando LopezNational Institutes of HealthSandy BarskyUnited States GovernmentSwathi YoungIntegrity Management Services, Inc.Tim GildayGDITTimothy GeorgeThe Maven GroupTodd D. LyleCouncil for Cyber Risk ControlTodd HagerMacro SolutionsTracy Lynn JonesGrant ThorntonWilliam D. RadcliffNational Institute of Standards and TechnologyThe ethical application of Artificial Intelligence_8e073df4-0aaa-11eb-88ad-54d42c83ea00To allow for an establishment of a consistent measure upon which one may qualify and quantify components used to create, operate, and improve AI capabilities_8e073f48-0aaa-11eb-88ad-54d42c83ea00ImpartialityAI algorithms learn from large quantities of data. The machine learning models that the AI builds can amplify some of the biases inherently present in the data. Accountable owners of AI systems should identify and address bias in AI to prevent negatively impacting desired mission outcomes or individuals in protected classes or statuses.FairnessAI systems should be designed to avoid potential risk of unfair impact within the context of use, whether intentional or unintentional.TransparencyAI systems should be developed so that models, data, and results are auditable and explainable to decision-makers and the general population to the extent and manner appropriate or possible.ResponsibilityThe implementation of an AI solution must be relevant to the purpose of the task. It must ensure that both data and model sources are uncompromised. It must produce repeatable, legal, authentic, auditable, and effective results.InterpretabilityStakeholders should thoroughly understand what AI has been asked to provide. They should be able to ensure that both data and model sources are credible, and will produce repeatable, trustworthy, and effective results. BiasUnderstand how bias influences and affects the inputs to AI, the algorithms that operate it, and the interpretations that provide insights to decision-making_8e07402e-0aaa-11eb-88ad-54d42c83ea001General Services AdministrationThe General Services Administration (GSA) has made Bias training mandatory.GSA Online UniversityAccording to the GSA Online University, "Whether we know it or not, we all possess unconscious biases affecting us inside and outside the workplace." Unconscious bias may result from ingrained stereotypes, omission resulting from lack of awareness as to the variable's relevance (under-fitting the model), or even confirmation bias of data that results from prior association of the data with similar models. Implicit and explicit bias in the adjudication of benefits or in the adjudication of risk is neither novel nor unique. Even before the introduction of Artificial Intelligence ecosystems at scale in the federal, state, and local governments, there were many instances where bias was introduced into systems by humans through written and unwritten policies and procedures related to interactions with government agencies.Bias in an Artificial Intelligence algorithm is a reflection of the organization(s) and person(s) who implement and integrate the AI... Given the increased use and reliance on AI capabilities, it is critical to understand how bias influences and affects the inputs to AI, the algorithms that operate it, and the interpretations that provide insights to decision-making. Since AI algorithms are informed by large quantities of data from which they analyze and provide answers, the machine learning models that the AI builds can amplify some of the biases inherently present in the data. It is imperative that the adoption, implications, and impact of data are monitored and measured. Through this process, one can identify and potentially eliminate explicit bias in AI. This requires collaboration across disciplines to develop and implement technical improvements, and operational practices to address the inherent bias. It's important for all of us to remember that there is no universal and unchanging set of ethics, and that regional and cultural diversity are key to any conversation about AI ethics. Objectives & UsesClearly understand the objectives and usages of AI systems_8e074114-0aaa-11eb-88ad-54d42c83ea001.1Context of Use: The AI use case should be developed with a clear understanding of the objective and usage of the system. When developing an AI solution, one should understand the following:GoalsUnderstand the goals of the use cases_baf1ed3c-0b31-11eb-af7f-be0d0783ea001.1.1Data SourcesUnderstand the data sources used to achieve the objectives_baf1ef12-0b31-11eb-af7f-be0d0783ea001.1.2Owners/GovernorsUnderstand the decision owners/governors_baf1f0a2-0b31-11eb-af7f-be0d0783ea001.1.3TriggersUnderstand the decision triggers (e.g., algorithm or business process)_baf1f232-0b31-11eb-af7f-be0d0783ea001.1.4ClarityUnderstand the clarity about the objectives and usages of the systems_baf1f3ae-0b31-11eb-af7f-be0d0783ea001.1.5DiversityInclude diversity among various aspects of AI systems_8e0741f0-0aaa-11eb-88ad-54d42c83ea001.2Diversity: The AI solution should include diversity among various aspects including the team, protected classes, and stakeholders. Although most government entities have gotten much better at identifying and rectifying explicit bias, there have been many challenges with identifying and rectifying implicit bias resulting in disparate impact to individuals in protected classes or protected status. There are numerous areas where one must include diverse and inclusive perspectives to ensure these biases are addressed, including:TeamsEnsure diversity among the teams involved in data provenance, collection, and labeling_baf1f52a-0b31-11eb-af7f-be0d0783ea001.2.1Protected ClassesEnsure diversity across the nine protected classes (sex, race, age, disability, color, creed, national origin, religion, or genetic information) as per the US federal law_baf1f6ce-0b31-11eb-af7f-be0d0783ea001.2.2Engineering TeamsEnsure diversity of the engineering teams involved in creating the algorithms_baf1f868-0b31-11eb-af7f-be0d0783ea001.2.3StakeholdersEnsure inclusion of Stakeholder committee members, including representation from various departments - Legal, HR, Sales, Marketing etc._baf1f9da-0b31-11eb-af7f-be0d0783ea001.2.4TrainingEnsure training for diversity, equity and inclusion (DEI)_baf1fb74-0b31-11eb-af7f-be0d0783ea001.2.5Protected ClassesIdentify and address bias in the data to prevent negatively impacting individuals in protected classes or status_8e0742cc-0aaa-11eb-88ad-54d42c83ea001.3Data Bias: AI systems should identify and address bias in the data to prevent negatively impacting individuals in protected classes or status. Cleansing the data to remove links in the relationship between outcomes and protected characteristics is essential to editing metadata to produce representations of the data that do not contain information about sensitive classes and status. Conducting an independent validation and verification process is significant to determine if outputs from a model are similar to the original inputs when reverse engineered. The bias inherent in the data must be evaluated from the following areas:Statistical DistributionsEvaluate the statistical distributions of data and methods dealing with skewed data_baf1fcfa-0b31-11eb-af7f-be0d0783ea001.3.1AppropriatenessEvaluate the appropriateness of data sets (confirm the data sets include complete information and no key information is missing)_baf1fe8a-0b31-11eb-af7f-be0d0783ea001.3.2Synthetic DataEvaluate the engineering and impact of synthetic data_baf20042-0b31-11eb-af7f-be0d0783ea001.3.3FittingAvoid over-fitting and under-fitting of the data_baf201c8-0b31-11eb-af7f-be0d0783ea001.3.4DataAddress the biases inherently present in the data_8e074402-0aaa-11eb-88ad-54d42c83ea001.4Data Modeling: Since AI algorithms learn from large quantities of data, the data models must consider solutions to address the biases inherently present in the data. AI solutions and their models need to consider the following:Size & VariabilityConsider the size and variability of the test and training data_baf20344-0b31-11eb-af7f-be0d0783ea001.4.1ReproducibilityConsider the reproducibility of the results_baf204f2-0b31-11eb-af7f-be0d0783ea001.4.2PredictionsConsider the factors used for predictions (e.g. psychological, behavioral, geographical or any other societal inferences)_baf20768-0b31-11eb-af7f-be0d0783ea001.4.3Training DataConsider the training data considerations for protected classes_baf20916-0b31-11eb-af7f-be0d0783ea001.4.4Protected ClassesExplainabilityConsider the explainable methodology for modeling_baf20bc8-0b31-11eb-af7f-be0d0783ea001.4.5FairnessArrive at a trusted solution as free as possible from enabling the potential for unfair strategic advantage or undesirable outcome_8e0744e8-0aaa-11eb-88ad-54d42c83ea002AI system owners should be concerned about fairness because of the potential of discriminatory, unwanted, undesirable, or unacceptable social, economic, health, or legal outcomes. Measuring fairness in AI requires adequate identification of potential risks that might be introduced intentionally or unintentionally. A main objective in a successful and trustworthy implementation of AI is to arrive at a trusted solution as free as possible from enabling the potential for unfair strategic advantage or undesirable outcome. If AI solutions produce unfair outcomes and behaviors, especially if these are harmful, future implementations of AI may face limited adoption and stall its full potential. By establishing a framework that considers key indicators of fairness throughout the various phases of the AI lifecycle, including design, development, implementation, and monitoring, it would be possible to effectively enhance the users' ability to detect, and understand related implications and unwanted biases. Fairness depends on the context of use and the intent of the AI implementation. Fairness in AI points to enabling awareness of underlying data and processes involved to identify inclusive and impartial representation and treatment of all relevant attributes needed to achieve desired objectives. The level of diversity, the inclusion of multiple stakeholders with different perspectives, the understanding of the context of use, the level of transparency, and interpretability of algorithms are key indicators for fairness. Fairness should be continuously monitored throughout the AI process life cycle. It is very important to frequently test for quality and compliance. User feedback, quantitative and qualitative metrics need to be continuously captured and evaluated to detect, understand, and address unwanted bias that may lead to unfair outcomes or behaviors. FitnessEnsure that AI solutions adequately address the problems expressed in the use cases_8e0745d8-0aaa-11eb-88ad-54d42c83ea002.1Understanding of Context of Use (Scope): The AI solution will adequately answer the problem expressed in the use case. Review the Fit for Use according to the following criteria:GoalsIdentify the goals of the use cases _baf20d44-0b31-11eb-af7f-be0d0783ea002.1.1SolutionsDetermine whether the AI solutions seem to adequately answer the problems expressed in the use cases_baf20ef2-0b31-11eb-af7f-be0d0783ea002.1.2Diversity & InclusionAddress the greatest levels of diversity and inclusion_8e0746dc-0aaa-11eb-88ad-54d42c83ea002.2Level of Diversity and Inclusion: The AI solution must demand the greatest levels of diversity and inclusion. The solution should be reviewed for diversity and inclusion according to the following criteria:DifferencesAddress criteria that embraces/ensures group, individual, agent, or entity differences, including, but not limited to: age, ethnicity, gender, educational discipline, cultural perspectives, diverse domains of expertise and perspectives, performance capacity, reach, scope, and dimensionality_baf210aa-0b31-11eb-af7f-be0d0783ea002.2.1FactsAddress criteria for fact-based determination of potential biases in decision making, and overall impact on society, behavior, and performance outcomes_baf21262-0b31-11eb-af7f-be0d0783ea002.2.2Ethical ValuesAddress criteria that embrace universal ethical values_baf2142e-0b31-11eb-af7f-be0d0783ea002.2.3DataAddress the relevant data indicators_8e07484e-0aaa-11eb-88ad-54d42c83ea002.3Data Indicators: The AI System will provide robust Data Lifecycle management, discriminatory behavior, represent objective and subjective indicators, and provide a data framework that ensures transparency. The solution should be reviewed per the following criteria:Life-CyclesAddress fairness at every step of AI solutions_baf2174e-0b31-11eb-af7f-be0d0783ea002.3.1Data Life-Cycle -- A robust approach to fairness is essential at every step of an AI solution: * Data Seeding * Data Discovery and Acquisition * Data Representation and Quality * Data Operations, including Text Preprocessing, Understanding, and Language Feature extractionDiscriminationAnalyze statistical parity or equal group error rates for protected groups_baf2192e-0b31-11eb-af7f-be0d0783ea002.3.2Discriminatory behavior – Group fairness calls for analysis on statistical parity or equal group error rates for protected groups, while individual fairness says analytics should aim only at accurate predictions.RepresentativenessConsider both objective and subjective indicators_baf21af0-0b31-11eb-af7f-be0d0783ea002.3.3Objective and subjective indicators - Includes relevance of historical data, timeliness of data: Subjective data are information from the client's point of view ("symptoms"), including feelings, perceptions, and concerns obtained through interviews. Objective data are observable and measurable data ("signs") obtained through observation, physical examination, and laboratory and diagnostic testing.TransparencyUnderstand all processes and inherent limitations of data availability_baf21cda-0b31-11eb-af7f-be0d0783ea002.3.4Data framework to understand all processes and inherent limitations of data availability:Access & UsabilityFacilitate access and working with data no matter where they are located or what application created them_baf21f64-0b31-11eb-af7f-be0d0783ea002.3.4.1(1) The ability to easily access and work with data no matter where they are located or what application created them.Sources & AccuracyAssure that data are accurate and come from official sources_baf2213a-0b31-11eb-af7f-be0d0783ea002.3.4.2(2) The assurance that data being reported are accurate and are coming from the official source.Methodology & TechnologyAddress the relevant methodology and technology life cycle indicators_8e0749a2-0aaa-11eb-88ad-54d42c83ea002.4Methodology and Technology Life Cycle Indicators: The methodology and technology by which the AI system is employed will provide the ability to explain outcomes, provide transparency with respect to purpose and function and identify potential. The solution should be reviewed from the following perspectives:ExplainabilityMaximize the extent to which outcomes are related to predictions in ways that humans understand the determining factors_baf22464-0b31-11eb-af7f-be0d0783ea002.4.1TransparencyBe open about validation of the purpose, quality of the objective function, structure, and underlying actions_baf2264e-0b31-11eb-af7f-be0d0783ea002.4.2Limitations & AssumptionsIdentify and evaluate in terms of overall risk and impact_baf2282e-0b31-11eb-af7f-be0d0783ea002.4.3Risk & ImpactEvaluate possible outcomes for unfair strategic advantage, including intended as well as unintended outcomes_baf22a2c-0b31-11eb-af7f-be0d0783ea002.4.4Evaluation & ImprovementIteratively evaluate expected and actual outcomes_8e074aba-0aaa-11eb-88ad-54d42c83ea002.5Iterative Evaluation and Improvement across Development Life Cycle: Continuous improvement of the AI system will be by an iterative evaluation mechanism designed to evaluate expected and actual outcomes.DatasetsIntroduce pre-processed training datasets_baf22bf8-0b31-11eb-af7f-be0d0783ea002.5.1A pre-processed training dataset is first introduced into the model.Models & ProcessingBuild models and process the data with them_baf22fc2-0b31-11eb-af7f-be0d0783ea002.5.2After processing and model building with the given data, the model is tested, and the results are matched with the desired result/expected output.Testing & MatchingTest the models and match the results with the desired result/expected outputs_baf231b6-0b31-11eb-af7f-be0d0783ea002.5.3Feedback & TuningReturn feedback to the systems for the algorithms to further learn and fine tune their results_baf233aa-0b31-11eb-af7f-be0d0783ea002.5.4The feedback is then returned to the system for the algorithm to further learn and fine tune its results, repeating until outcomes exhibit minimal variance and have an equal probability of occurrence when similar initial data is input.TransparencyEnsure that AI is explainable to any user, decision maker, or impacted population_8e074bc8-0aaa-11eb-88ad-54d42c83ea003Transparency in the context of AI means that AI is explainable to any user, decision maker, or impacted population. As AI becomes increasingly ubiquitous in all aspects of our lives, it is critical to ensure that these AI systems are developed with data that is fair, interpretable, and representative. In the context of AI Ethics, the definition of transparency takes on an additional, sometimes paradoxical, meaning as it is used to infer the trust and reliability upon the AI use case and the decisions that follow. Transparency in the context of AI is openness about the purpose, structure, and underlying actions of the algorithms used. In an ideal state, AI would be fully transparent to users, decision makers, and those impacted, but this is difficult to achieve in practice given intellectual property concerns. Transparency can be used to identify issues of fairness, bias and trust -- all of which have received increased attention. Used effectively, transparency creates a means to instill credibility and establish confidence; explainable AI improves transparency. OpennessEnsure the results from algorithms can be tracked back from a data, architecture and algorithmic perspectives_8e074cd6-0aaa-11eb-88ad-54d42c83ea003.1Open Data, Architecture and Algorithms: When the results from an algorithm can be tracked back from a data, architecture and algorithmic perspectives, it lends itself to be transparent.DataAppropriately enable open usage of the data_baf23580-0b31-11eb-af7f-be0d0783ea003.1.1An AI system where the data use is open enables transparency.Proprietary & Personal DataAppropriately protect proprietary data and personal data_baf23792-0b31-11eb-af7f-be0d0783ea003.1.1.1The caveat is in those instances where proprietary data or data with PII needs to be used.Diagnostics & ImprovementLeverage transparency as a diagnostic to improve models_baf2399a-0b31-11eb-af7f-be0d0783ea003.1.2Transparency works as a diagnostic to improve your model. Models can run away from you and your improvement is limited. Data undergoes complex transformations within the data pipelines that feed AI systems. This results in "data derivatives" and the risk that the initial meaning of transformed data is lost, which could result in it being misinterpreted.ArchitecturesUse open architectures_baf23b84-0b31-11eb-af7f-be0d0783ea003.1.3AI systems should use open architecture which makes adding, upgrading or swapping components easy and allows users to see inside all or parts of the architecture without any proprietary constraints.ToolsDevelop tools to address explainability_baf23e18-0b31-11eb-af7f-be0d0783ea003.1.4An AI system where the algorithm is not proprietary and the logic behind the algorithm can be shared creates transparency in an AI system. Opaque black-box algorithms, such as those used in deep neural networks, incorporate many implicit and highly variable interactions into their predictions. By contrast, transparent "glass box" algorithms, such as those used for logistic regression, are usually simpler. There is no single approach to understand all algorithms. To aid in the future adoption of deep neural networks (DNN), tools are being developed to address explainability.ObservabilityEnsure access to relevant information and results_8e074e5c-0aaa-11eb-88ad-54d42c83ea003.2Observable: An AI system that is observable has access to relevant information and results in explainability. Explainability is the extent to which the internal mechanics of an AI system can be explained in human terms and provides the ability to answer the why, how, and what to provide insights into where an algorithm succeeds, fails, and errs, allowing for greater understanding of the process. Explainability should not be confused with interpretability, which is about the extent to which a cause and effect can be observed within a system. Rather, it is the extent to which you are able to predict what is going to happen, given a change in input or algorithmic parameters. While Interpretability is about being able to discern the mechanics without necessarily knowing why, Explainability is being able to quite literally explain what is happening in terms of the lifecycle of: Aggregation, Assessments, and Answers. ResponsibilityEnsure accountability through appropriate usage_8e074ff6-0aaa-11eb-88ad-54d42c83ea004The responsible application of AI demands accountability through appropriate usage. In the AI Ethics dialogue, responsibility has less to do with the technology and more to do with its use. Responsibility deals with the intent of application and protects outcomes from any compromised motive. Responsibility will not address technology and never weighs its feature set. Responsibility weighs heavily on the appropriate use of the technology and how to protect stakeholders and targets from outcomes that do not apply to its mission. The implementation of an AI solution must be relevant to the purpose of the task. It must ensure that both data and model sources are uncompromised. It must produce repeatable, legal, authentic, auditable, and effective results. PurposesEnsure that AI systems are used only in accordance to their designed purposes in support of their missions_8e075140-0aaa-11eb-88ad-54d42c83ea004.1Purpose -- The AI System will only be used in accordance to its designed purpose in support of the mission. AI is a family of technologies, each with purpose and targeted function. Solutions may use one or a combination of these technologies to a single application designed for a specific purpose. The goal is to focus the portfolio of apps and tools to support its stakeholder(s). Trying to apply an application portfolio to support another stakeholder's requirements, regardless of how similar, will likely produce unknown outcomes, some of which could prove harmful. PrivacyProtect the privacy of all stakeholders_8e075280-0aaa-11eb-88ad-54d42c83ea004.2Privacy -- The user(s) of the AI system must protect the privacy of all stakeholders. Personally identifiable information (PII) cannot be compromised without detriment to the individual or parties impacted by its use. Protecting individuals' privacy is a matter of Public Law 93-579, known as the Privacy Act of 1974. Even if "security by design" practices are foundational in the engineering of AI tools and utilities, threats outside of the technology must be considered. Again, this section deals with the behavior of the technology's usage; thus, threats perimeter to the technology are exploited through simple mechanisms such as malicious code, social engineering, and improper use. Users of AI must follow sound cyber practices to protect credentials, remote access, removable media, mobile devices, email, social networking, and the like. VeracityEnsure the veracity of data before consuming it for usage_8e0753ac-0aaa-11eb-88ad-54d42c83ea004.3Pedigree -- The application of the AI system will first ensure data veracity before consuming data for usage. It is not the function of the AI to guarantee the pedigree of the data it is consuming for analysis.Input DataEnsure that input data is both untampered with and all-inclusive_baf23fda-0b31-11eb-af7f-be0d0783ea004.3.1However, it is the responsibility of its users to ensure that input data is both untampered with and all-inclusive.RepresentationEnsure that all participating stakeholders and affected groups are represented_baf241d8-0b31-11eb-af7f-be0d0783ea004.3.2All participating stakeholders and affected groups must be represented within the data sets.SourcesVerify the sources of the data_baf24412-0b31-11eb-af7f-be0d0783ea004.3.3Finally, the source of the data must be verified before it can be declared appropriate for use. Purity, completeness and the validation of the source of the data produces repeatable and trustworthy outcomes. Evolution & MaturationAudit the evolution and maturation of data and model structures_8e0754d8-0aaa-11eb-88ad-54d42c83ea004.4Provenance -- The user(s) of AI will audit the evolution and maturation of data and model structures. Pedigree, above, deals with the veracity of the data.AuthenticityProve the authenticity of the data_baf24642-0b31-11eb-af7f-be0d0783ea004.4.1Provenance is the foundation for auditable changes to the data, whether the data is consumable or algorithmic. In short, it is proving its authenticity.Measurement & MonitoringEnable users to monitor and measure the potential impact from malicious alteration_baf2485e-0b31-11eb-af7f-be0d0783ea004.4.2Coupling technology, such as blockchain, may provide users the ability to monitor and measure the potential impact from malicious alteration and automate auditability of planned change. AutomationAutomate auditability of planned changes_baf24ab6-0b31-11eb-af7f-be0d0783ea004.4.3InterpretationEnsure that outputs are clear and consistently understood by a variety of people_8e075622-0aaa-11eb-88ad-54d42c83ea005The results of an AI solution must be operationally relevant and focused upon the goals and objectives of the organization in their ongoing efforts to serve their specific mission. The results of leveraging and applying AI capabilities must ensure that both data and model sources are uncompromised, produce a consistent result, and are reliable and trustworthy. By ensuring outcomes are consistent, the AI application will instill credibility and establish credibility with all who utilize their services. To achieve this, the outputs must be clear and free from ambiguity and assure it is consistently understood by a variety of people. This final step is a culmination of all that goes into the EAAI by ensuring the data that feeds the algorithm, the means upon which it analyzes information, and the resulting assessments that produce the knowledge base provide consistent understanding given current circumstances.Causality/InfluencesUnderstand the relational dependencies and how they are interpreted._8e07574e-0aaa-11eb-88ad-54d42c83ea005.1Correlation/EffectAscertain how the culmination of analysis produces answers._8e0758a2-0aaa-11eb-88ad-54d42c83ea005.2Consequences/ImpactUnderstand how the insights influence and affect the environment._8e075b18-0aaa-11eb-88ad-54d42c83ea005.3Consistency/ReliableAssess how the predictability of the outcomes provide repeatable results given the same circumstances. _8e075c62-0aaa-11eb-88ad-54d42c83ea005.42020-10-082020-10-10https://www.actiac.org/system/files/Ethical%20Application%20of%20AI%20Framework.pdfOwenAmburOwen.Ambur@verizon.net