Next Generation Social Science (NGS2)The Defense Sciences Office at the Defense Advanced Research Projects Agency (DARPA) is soliciting innovative research proposals to build a new capability (methods, models, tools, and a community of researchers) to perform rigorous, reproducible experimental research at scales necessary to understand emergent properties of human social systems. DARPA anticipates that the Next Generation Social Science (NGS2) program may require a fundamental reimagining of the social science research cycle and encourages participation from a wide and diverse combination of disciplines and skill sets -- to include social sciences, but also physics, computer science, biology, game design, mathematics, and others. Specifically excluded is research that primarily results in incremental improvements to the existing state of practice. DARPA-BAA-16-32The NGS2 program is divided into two phases, a 24-month base period (Phase 1) with one 18- month option period (Phase 2). Each phase will consist of two research cycles. For the sake of clarity, this BAA will reference the conceptual elements of each research cycle in terms of the following Technical Areas (TAs), described in detail below: * TA1: Predictive Modeling and Hypothesis Generation; * TA2: Experimental Methods and Platforms; and * TA3: Interpretation and Reproducibility. In Phase 1, performers will develop and demonstrate research tools and methods for rapidly testing and evaluating the accuracy of experimental hypotheses and predictions in multiple populations.Defense Sciences OfficeDSO_90e53b46-f099-11e5-9063-6a63f985f679DARPANGS2 Program PerformersPerformers in the NGS2 program will work to determine fundamental measures and causal mechanisms that explain and predict the emergence of "collective identities." A focus on "what matters most" for emergent social phenomena like collective identity presents an important and complex challenge to focus and validate new NGS2 research communities, tools, and methods. Note that while the NGS2 program will focus on collective identity formation as an exemplar research question, DARPA anticipates that successful NGS2 capabilities will have benefits for tackling other complex problems and topics, including (but not limited to): resilience in social networks and structures, changes in cultural norms or beliefs, emergence of cooperation/competitions, social influences on preferences and cognition, etc... Teams that demonstrate progress or promising technical approaches in support of NGS2 goals and TAs during Phase 1 may be encouraged to continue their research efforts in the next phase. In Phase 2, performers will prove out their capabilities and demonstrate the replicability and rigor of their methodologies and models. Performers will evaluate their models’ external validity and robustness by scoring their predictive accuracy across the experiments of other Phase 2 performers. All proposers are expected to fully address how the proposed research will contribute to NGS2 program goals. The proposal narrative should clearly explain the technical approach and fully describe the research and development required to build and test proposed methods, models and tools. Proposers must also describe how the work advances the state of the art as applied to social science in general and the exemplar problem in particular. Proposals should also identify the highest development, integration, and scaling risks involved and clearly describe how those risks will be mitigated early in the proposed effort. Performer Categories: Performance in the NGS2 program will occur in one of three categories: End-to-End (ETE), Enablers, and Test & Evaluation (T&E). As described below, performers in the first two categories will focus on research and development and the third will perform independent testing and evaluation. Proposers may submit abstracts and proposals in any of these three categories. However, to avoid any real or perceived conflicts of interest between development and T&E, proposers participating on a T&E submission may not participate on an Enabler or ETE team submission (See Section III.D for further information.). Proposers must specify the category to which they are applying on their submission cover sheet(s).End-to-End (ETE) PerformersOne or more complete multidisciplinary teams executing an end-to-end research approach that addresses each of the TAs described below. These teams are expected to engage in multiple cycles of modeling, hypothesis generation, prediction, experimentation, analysis, and reevaluation beginning in Phase 1 and continuing through Phase 2.EnablersSmaller, targeted efforts to develop and test new or early stage high-risk, high-payoff "enabling" technologies in one or more of the TAs. At the end of Phase 1, this second category of performer should enable some new capability (e.g., greater speed, more rigor, higher transparency, more dynamic informed consent, larger numbers of participants) and be poised for integration into a complete end-to-end approach. Proposals in this category must detail the unique enabling technical or methodical approach to be developed in regard to: the next generation research capability made possible by the proposed approach; the feasibility of achieving this goal during Phase 1; and the effort required for maturation, including a risk-reducing technical development plan with clear progress milestones. Proposers should also illustrate how they anticipate the enabling approach integrating with, and thereby impacting, Phase 2 research cycles.Test & Evaluation (T&E) PerformersA single team responsible for assisting the Government with testing and evaluation of the other performers’ programmatic and technical progress towards NGS2 goals. Proposers in this category must have demonstrated experience supporting scientific experimentation, diverse collaboration and evaluation of the accuracy of experimental predictions. They must also have a history of effectively interacting with multiple research communities, including the social sciences, to help advance multidisciplinary, reproducible research in line with NGS2 goals. Proposers should discuss their capabilities in managing experiment registration, innovative forecasting and prediction platforms, engagement with appropriate scientific publications for registered reports of performer protocols as a form of early peer review, and providing consultation, infrastructure support, and collaboration enhancement across scientific teams and geographies. In addition to these activities, the T&E team will provide feedback to the ETE and Enabler performers and DARPA on plans and reports, assist DARPA with program research, and provide overarching documentation of program progress, implementation, and outcomes. As directed by DARPA, the T&E team will also help support transition of successful program products to DoD partners as well as the wider social science research community._90e53b47-f099-11e5-9063-6a63f985f679To build a new capability to perform rigorous, reproducible experimental research at scales necessary to understand emergent properties of human social systems._90e53b48-f099-11e5-9063-6a63f985f679CollaborationThroughout the course of the NGS2 program, it is likely to be necessary for all performers -- regardless of category -- to share relevant information regarding their research and development to support the larger program goals. For example, ETE performers may need insight into the TA1, TA2, or TA3 approaches being tested by other ETE performers as well as access to data and code to reproduce their results. Similarly, ETE performers may need access to tools and methods being developed by Enablers, while Enabler performers may need insight into ETE performer approaches. The T&E team will need access to ETE and Enabler performer plans and methods. DARPA expects all NGS2 performers to work collaboratively with one another to realize the program objectives outlined herein, so proposers should carefully review the goals for the entire program in order to fully understand the context of each program objective, performer category, and TA within the overall program structure. All proposals should describe plans for ensuring transparency of their processes to enable interactions with other NGS2 performers. Proposals that fail to include these plans may be deemed non-conforming and removed from consideration.OpennessIntellectual Property -- As discussed above, data sharing and collaboration are key aspects of this program. Therefore, intellectual property rights asserted by proposers are strongly encouraged to be aligned with open source regimes. See Section VI.B.1 for further information.Human RightsHuman Subjects Research -- It is anticipated that proposals submitted in response to this BAA may involve Human Subjects Research (HSR). All proposers planning to involve HSR should carefully review the HSR requirements in Section VI.B.2 of the BAA. Any required HSR information (including, but not limited to Institutional Review Board (IRB) draft application plans) must be included in the proposal. Furthermore, any task that involves HSR must be clearly identified in the proposed Statement of Work, schedule, and cost details. Proposed schedules must take into account the time required for all IRB approvals. Proposals involving HSR that fail to supply evidence of or a plan for review by an IRB may be deemed non-conforming and, as such, will not be reviewed.ValidationA key programmatic feature of NGS2 is the self-validation of each ETE performer's models, methods, data, and analyses as well as cross-validation by other performer teams. Performers should therefore consider carefully issues of data portability and protocol fidelity when selectingtheir research questions and experimental methods. During each research cycle, ETE performers will pre-register their predictions for their own experimental results, then conduct appropriate experiments with three separate representative populations. The T&E team is expected to assist performers with submitting "registered reports" as appropriate/feasible, as well as identifying and coordinating subject matter experts (SMEs) to provide baseline predictions against which performers' accuracy can be further compared. These SMEs may include, but need not be limited to, researchers or recognized experts from different communities, organizations, or disciplines. After each research cycle, ETE performers will extend their models to incorporate results and provide their data, analyses, and experimental methods to other teams to ensure reproducibility of their results and conclusions. Each ETE performer will attempt to reproduce all other ETE performers' results using the data and tools provided to them. In the case of reproducibility difficulties, the T&E team may serve as a third-party evaluator. ETE performers will then develop new predictions and replicate their own experiment with three new populations. At this time, DARPA does not anticipate asking ETE teams to replicate another performer’s experimental protocol.Models & HypothesesEnhance capabilities to identify, formalize, and empirically test hypotheses derived from models and theories._90e53b5a-f099-11e5-9063-6a63f985f679TA1EnablersAs relevant, Enabler and T&E proposers should detail their proposed tools, approaches, and/or methods to contribute to one or more TA1 goals. They should also describe the anticipated impact and the expected outcome(s) and benefit(s) of their research for the NGS2 program, specifically, and social science more generally.Test & Evaluation (T&E) PerformersPredictive Modeling and Hypothesis Generation: TA1 seeks to enhance capabilities to rapidly identify, formalize, and empirically test hypotheses derived from a wide range of models and theories, each of which may make competing claims about what factors matter most, when, and why, for the emergence of collective identify. A major goal of TA1 is to formalize different variables and parameters from multiple models and theories -- including social and behavioral sciences, but also potentially models or theories from epidemiology, biology, ecology, physics, network science, etc., or combinations thereof -- in order to compare their accuracy in predicting the direction and size of effects of different experimental interventions. This TA also seeks innovative approaches for accounting for biases across research bodies and literatures, and to increase models' responsiveness to changing evidence, to include ensemble modeling. Performers should therefore be able to tackle a number of current challenges, including: how to formalize and compare time-dependent predictions from various stochastic, deterministic, and/or agent-based models; how to push theoretical or descriptive models towards quantitative prediction, while pushing quantitative but often atheoretical models towards greater description; how to connect low level, quantifiable data to higher order theory while managing dimensionality; and how to derive sufficiently precise, falsifiable predictions to help disconfirm otherwise plausible models/theories. ETE proposers should address TA1 by detailing their proposed approaches, tools, and/or methods for (at a minimum):Identity FormationIdentify multiple theories and models, and potentially combinations thereof, that speak to collective identity formation from a diverse set of disciplines and literatures_90e53ca4-f099-11e5-9063-6a63f985f679TA1.1ImplementationsFormalize parametric implementations of those models_90e53dda-f099-11e5-9063-6a63f985f679TA1.2BiasesRecognize and account for various biases in research and publications to appropriately re-weight specific model variables, parameters, and/or dynamics_90e53f10-f099-11e5-9063-6a63f985f679TA1.3PredictionsArticulate and register specific, testable predictions of the direction and size of effects of interventions on different outcome measures_90e54064-f099-11e5-9063-6a63f985f679TA1.4Reproducibility & UpdatingDocument models to facilitate reproducibility, while updating models based on empirical results_90e5419a-f099-11e5-9063-6a63f985f679TA1.5Methods & PlatformsEnable new capabilities for the design and conduct of multifactorial experimental research to test predictions related to the emergence of collective identity._90e542da-f099-11e5-9063-6a63f985f679TA2EnablersAs relevant, Enabler and T&E proposers should detail their proposed tools, approaches, and/or methods to contribute to one or more TA2 goals. They should also describe the anticipated impact and the expected outcome(s) and benefit(s) of their research for the NGS2 program, specifically, and social science more generally.Test & Evaluation (T&E) PerformersExperimental Methods and Platforms: TA2 seeks to enable new capabilities for the design and conduct of multifactorial experimental research of sufficient complexity, time, number of participants, etc., to test the accuracy and robustness of models' predictions related to the emergence of collective identity. For NGS2 , the term "experimental” is used to highlight a focus on predicting and testing hypothesized causal mechanisms by using ethical and empirically sound experimental approaches (including interventions, randomization, control groups, etc.), as opposed to approaches based largely on observation and correlation. ETE proposers should address TA2 by detailing their proposed approaches, tools and/or methods for (at a minimum): * Experimental design * Participant recruitment and retention * Data collectionExperimental Design_90e5442e-f099-11e5-9063-6a63f985f679TA2.1Test & Evaluation (T&E) PerformersMultifactorial ExperimentsDesign multifactorial (many-to-many) experiments that effectively and credibly operationalize dependent measures of collective identity formation_90e5456e-f099-11e5-9063-6a63f985f679TA2.1.1Platforms & NarrativesCreate or leverage experimental platforms and narratives that engage participants, that are easy to understand, have face and ecological-validity to elicit relevant behaviors and knowable ground truth, and provide sufficient sample sizes and diversity_90e546c2-f099-11e5-9063-6a63f985f679TA2.1.2Independent Variable ExperimentsRun experiments that allow multiple independent variables to be manipulated in order to test a wide range of different model predictions -- potentially with "rolling predictions" for particular time-dependent interventions or manipulations -- while facilitating reproducibility of results_90e54938-f099-11e5-9063-6a63f985f679TA2.1.3CommunicationCommunicate experimental plans to various Human Subject Research Protection organizations, to include DoD Secondary Review and international partners as appropriate_90e54a8c-f099-11e5-9063-6a63f985f679TA2.1.4Human Subject Research Protection Organizations(see Section VI.B.2 for more detail)Replication & ExtensionReplicate and extend experiments across multiple research cycles_90e54bd6-f099-11e5-9063-6a63f985f679TA2.1.5Recruitment & Retention_90e54d48-f099-11e5-9063-6a63f985f679TA2.2ParticipantsDetermine and justify the numbers of participants that a proposer will seek to include, sufficient to test model predictions across at least 3 different and representatively diverse populations, with multiple intervention and control groups in each research cycle (i.e., a total of 12 different populations over the entire NGS2 program)_90e54e9c-f099-11e5-9063-6a63f985f679TA2.2.1Proposers should include their rationale for choosing these populations, inclusion/exclusion criteria, etc.Narratives & TasksDevelop experimental narratives and/or tasks of sufficient engagement and interest to attract the number of volunteer participants required for testing_90e54ff0-f099-11e5-9063-6a63f985f679TA2.2.2Recruitment & ConsentAddress approaches to informed consent and participant recruitment_90e5516c-f099-11e5-9063-6a63f985f679TA2.2.3Language, Demographics & CultureNegotiate potential language differences, demographic variation, cultural variables, etc., that may substantively impact experimental methods and data collection_90e552c0-f099-11e5-9063-6a63f985f679TA2.2.4Data Collection_90e5541e-f099-11e5-9063-6a63f985f679TA2.3MeasuresCapture and integrate subjective and objective measures using mixed methods_90e555e0-f099-11e5-9063-6a63f985f679TA2.3.1BiasesQuantify and account for various biases in experimental collection, such as platform- or media-specific bias, sampling bias, biases in existing data corpora, etc._90e5573e-f099-11e5-9063-6a63f985f679TA2.3.2PII & Sensitive DataAddress approaches to protecting PII and other potentially sensitive data._90e558a6-f099-11e5-9063-6a63f985f679TA2.3.3Interpretation & ReproducibilityAdvance capabilities for processing, analyzing, and visualizing data._90e55aa4-f099-11e5-9063-6a63f985f679TA3EnblersAs relevant, Enabler and T&E proposers should detail their proposed tools, approaches, and/or methods to contribute to one or more TA3 goals. They should also describe the anticipated impact and the expected outcome(s) and benefit(s) of their research for the NGS2 program, specifically, and social science more generally.Test & Evaluation (T&E) PerformersInterpretation and Reproducibility: TA3 seeks to advance capabilities for processing, analyzing, and visualizing data collected through next generation experiments, scoring experimental predictions for accuracy, determining the level (or lack) of evidentiary support of models, and enabling reproducibility (i.e., strictly reproducing results given access to the original datasets, code, and analytic procedures) as well as replication (i.e., the ability to replicate an experiment in another condition, with another population, and/or by different researchers). ETE proposers should address TA3 by detailing their proposed approaches, tools and/or methods for (at a minimum):Complexity, Nonlinearity & RecursivenessAnalyze data collected over long(er) periods of time, involving potentially complex, nonlinear, and recursive relationships of variables with large numbers of co-variates and mediators, etc._90e55c2a-f099-11e5-9063-6a63f985f679TA3.1Interpretation, Reproducibility, Replication & ResidualsConduct confirmatory and exploratory analyses in order to enhance interpretation, reproducibility, replication, and efforts to explain residuals related to specific research questions_90e55da6-f099-11e5-9063-6a63f985f679TA3.2Noisy & Missing DataDeal with noisy and/or missing data across potentially large multi-modal data sets_90e55f5e-f099-11e5-9063-6a63f985f679TA3.3UncertaintyQuantify, incorporate and communicate different types of uncertainty in behaviors, measures, and sampling_90e560c6-f099-11e5-9063-6a63f985f679TA3.4EffectsDetermine, validate and visualize effect sizes and variance explained/predicted by different models_90e56224-f099-11e5-9063-6a63f985f679TA3.5 Data ManagementAddress data management, including approaches to documenting and sharing data, meta-data and code(s) for other performers to reproduce results, as well as preserving data for future use by a wider research community_90e563be-f099-11e5-9063-6a63f985f679TA3.6An intended outcome of NGS2 is to enable ethical, rigorous, and reproducible experimental social science research at new scales for greater understanding of emergent properties of human social systems. DARPA anticipates that data collection, processing, curation, sharing, and preservation will be critical in achieving this outcome. Accordingly, all proposals must include a Data Management Plan (DMP). Proposals submitted without a DMP will be deemed non-conforming and will not be reviewed. Proposed DMPs should support DARPA's objectives for capturing, preserving and sharing digital data produced by scientific and engineering activities, including fulfillment of the objectives of the Federal Open Data initiative and DoD guidelines on research data sharing; advancing reproducibility and replicability in sponsored scientific and engineering projects; and, creating and promoting a foundation for new projects and programs that can exploit science and engineering data to accelerate scientific discovery and engineering innovation. As appropriate, the DMP should include, but not be limited to:Data SharingPlan for data sharing -- to include extent and mechanisms during and after the NGS2 program_90e56544-f099-11e5-9063-6a63f985f679TA3.6.1 HostingDescribe the hosting environment(s) for sharing digital research data with user communities_90e566ca-f099-11e5-9063-6a63f985f679TA3.6.2Data & Metadata Standards & Best PracticesIdentify data management standards, including meta-data standards, and/or community best practices that may apply_90e56878-f099-11e5-9063-6a63f985f679TA3.6.3Persistence & PreservationPlan for "data persistence" and preservation beyond the program_90e56a12-f099-11e5-9063-6a63f985f679TA3.6.4Data Kinds & AssetsProvide rough estimates of data kinds and assets; formats; sizes (e.g., KB, MB, GB, TB), etc._90e56bf2-f099-11e5-9063-6a63f985f679TA3.6.5 Kinds of data might include:Data SetsEstimate the experimental, test, and measurement data [to be gathered]_90e56d6e-f099-11e5-9063-6a63f985f679TA3.6.5.1NarrativesEstimate observational logs, journals, collaborations [to be established]_90e56ecc-f099-11e5-9063-6a63f985f679TA3.6.5.2AnalysesEstimate the analyses [that will be required]_90e57016-f099-11e5-9063-6a63f985f679TA3.6.5.3DecisionsEstimate the alternatives, exploration branches, determinations [to be considered]_90e5719c-f099-11e5-9063-6a63f985f679TA3.6.5.4Experiments & SimulationsEstimate the setup, ingest, and output(s) [requirements]_90e572fa-f099-11e5-9063-6a63f985f679TA3.6.5.5Codes, Software, Algorithms & DataEstimate the codes (with build scripts), software (executables with source), algorithms, data consumed or produced by software_90e5744e-f099-11e5-9063-6a63f985f679TA3.6.5.6Models & SimulationsEstimate the models or simulations [to be used]_90e575de-f099-11e5-9063-6a63f985f679TA3.6.5.7 Bibliographies & CitationsEstimate the bibliographies and citations used by your research_90e57746-f099-11e5-9063-6a63f985f679TA3.6.5.8RecordingsEstimate the recordings of various physical phenomena (including images) [to be made]_90e578a4-f099-11e5-9063-6a63f985f679TA3.6.5.9Sensitive Data[Indentify] methods for addressing and protecting sensitive data, to include participant anonymity, privacy or data redaction_90e57a3e-f099-11e5-9063-6a63f985f679TA3.6.6Data Quality[Explain] anticipated current or future data quality issues_90e57b9c-f099-11e5-9063-6a63f985f679TA3.6.7Costs[Address] anticipated costs for digital data management_90e57d04-f099-11e5-9063-6a63f985f679TA3.6.8Validation & Reproducibility[Explain] how the DMP enhances validation and reproducibility of results_90e57ee4-f099-11e5-9063-6a63f985f679TA3.6.9Innovation[Explain] how the DMP may support future innovation_90e58056-f099-11e5-9063-6a63f985f679TA3.6.102016-03-18https://www.fbo.gov/index?s=opportunity&mode=form&id=498e3d5371eb019f37554e3aaf04333e&tab=core&_cview=0OwenAmburOwen.Ambur@verizon.net