Emerging Technologies and the Need for Evaluation


by Jason Tashea
January 20, 2016


Technology is creating new opportunities for the improvement and reform of justice. Smartphone applications, crowdsourced data, and novel visualizations of information are just some of the ways in which technology is increasing access to justice and influencing public safety policy and practice. The growing influence of technology is creating a new urgency for criminal justice reform, but the decentralized development of programs makes it difficult to track promising projects or bring them to scale. Justice stakeholders are faced with the challenge of assessing technical innovations while they often lack the tools and resources to meet the challenge.

Professionals in justice and technology fields would benefit from an effort to manage the intersection of their respective domains. Innovations appear at a rapid pace, but no coordinated initiative exists to monitor new ideas, implement best practices, and evaluate the effects using rigorous research methods. To contribute to the development of worthwhile collaborations between technology and justice, the Research and Evaluation Center at John Jay College of Criminal Justice recently reviewed the nascent field of justice technology. The recommendations in this report are designed to support the growth and dissemination of technology projects that improve the operations of the justice sector, broaden public access to justice-related data, and promote social justice.


New technologies affecting criminal justice are growing in several key categories, including more open data about the occurrence of crime and the actions of law enforcement agencies. Progress, however, remains unevenwith some cities embracing the trend fully while others have yet to begin serious efforts. Some of the most promising projects from across the country (highlighted below) were identified by a review of the professional literature and through interviews with justice system stakeholders. The inclusion of a particular tool is not an endorsement. For a comprehensive list of projects, see the Appendix (Google Doc or PDF).

Read more about highlighted projects in three categories:





The evolution of justice technology is occurring due to wider forces that affect government, technology, and people. Trends like open data, Government 2.0, and civic technology are changing how the criminal justice system is operated and how advocates work to improve the system. While a number of factors are behind this ongoing change, openness is the philosophical backbone of technology’s increased role in public policy and civic life.

Openness has always been a philosophical and architectural component of the Internet (Leiner et al. 2012). One rationale for openness in the digital age was articulated in The Cathedral and the Bazaar by Eric S. Raymond, an essay examining prominent dichotomies in software development: top-down vs. bottom-up, closed vs. open, etc. The “cathedral” is where exclusive teams of developers create products that are later released with the source code— i.e. top-down and closed. In contrast, the “bazaar” is where software is developed online with numerous developers who may have very different approaches— i.e. bottom-up and open. Raymond concludes that the bazaar provides a more effective ecosystem for innovation and development.

logo_api2Application-programming interface (API) is a collection of rules, standards and tools to create communication between software programs or hardware. In some ways, APIs are the building blocks that lead to final products.

Developers rely on APIs to harness the engine of an existing tool and to build new projects. Perhaps the most used API is Google Maps which allows developers and end-users to benefit from the powerful Google Maps project and data without having to create projects from scratch.

Beyond private industry, the federal government creates APIs as well, these include the U.S. Data Service, Department of Justice, and the Bureau of Justice Statistics. In addition, 18F, a digital services agency housed within the federal general services agency, also develops APIs and a series of standards for government APIs.

While initially a philosophy about software development, the influence of openness extends further. The Smart Chicago Collaborative, a nonprofit civic technology organization, summarizes its importance:

We are open. In the technology industry, the primary manifestation of that is the use of open source code. We have dozens of repositories on Github, for every piece of software we’ve made over the last three years. But being open means more than using a particular license for our software. It means having open processes, so that people know what you’re doing, how you’re doing it, and how they can affect it. This is about allowing others ‘in’, wherever that may be in any particular situation.

The philosophy of openness is applied similarly in public policy. Openness for a government is achieved when it provides people access to its process, decision-making, and data. Governments are opening up particularly in the area of data access. A term with various interpretations, the Open Knowledge Foundation’s “Open Definition” project defines open data functionally: “Open data and content can be freely used, modified, and shared by anyone for any purpose” (emphasis theirs).

Open data policies and projects are happening at all levels of government. At the federal level, the U.S. Data Service is leading the way on standardization and release of federal data. State and local governments are also embracing openness. Nearly a third of states and cities have open data policies, according to recent estimates. These policies include what types of information will be open, how it will be made available, and procedures related to data upkeep. Once available, data may be used by companies like Yelp as well as advocacy organizations such as the Sunlight Foundation to improve the public’s understanding of government data.

Initially a Linux project, an open source operating system, Git is primarily used as a filing system for drafts of a software project. In tech parlance, this is a version control system. A version control system is not dissimilar from how Apple’s Time Machine can catalog every version of an evolving document. Git allows an individual to take an entire file, or repository, and make a copy. From there, the user can change the code to make improvements for either personal use or to send back to the owner of the repository for potential incorporation into the original. GitHub is a popular platform that allows for easy sharing of repositories and it doubles as a social networking site and wiki for developers and those affiliated with open source projects. GitHub remains different from other Git repository-hosting services due to its functionality. Before GitHub, a user would download a repository, make changes, and send the list of changes (a “patch”) back to the project’s owner, or the “maintainer.” GitHub users can access source code by “forking” it, which allows the forker to develop code independently and make changes which are sent back as a “pull request.” The increased functionality makes it easier for large groups of developers to collaborate more seamlessly.

Open access to data inspired the Government 2.0 and civic tech movements (Howard 2012). Government 2.0, or Gov 2.0, refers to a government’s use of technology and data to improve its operations and functions (Code for America 2014). An amorphous term, Gov 2.0 initially referred to the incorporation of social media and data technologies in government operations (Ben-Yehuda 2015). As the movement grew, the meaning of the term evolved as well. Gov 2.0 is now thought to include mobile services, open government, privacy and security, and various issues regarding the shared economy.

Watch the full video at Code for America

Civic technology is broader than Government 2.0. Google Trends indicates that civic tech has nearly replaced the term Government 2.0, but civic tech and Gov 2.0 are parts of the same phenomenon. To clarify, Micah Sifry, a co-founder of Personal Democracy Media, sees civic tech as technology that is built for the public good. The IDC Government Insights Report, sponsored by government technology vendor Accela, defined civic tech as merging “technology innovation with civic purpose.” The Knight Foundation made a more detailed attempt at defining civic tech. Knight’s definition includes companies, nonprofits, and projects that use technology to promote a civic outcome both in- and outside government. For Knight this includes government data projects, community organizing around civic issues, social networks and community forums, “collaborative consumption” (a peer-to-peer sharing of goods and services), and crowdfunding to enhance public projects.

Image courtesy of the Knight Foundation (2013)

Ron Bouganim, founder of the Govtech Fund, suggests that civic tech and Gov 2.0 (or “gov tech”) belong on a continuum. Civic tech includes projects that create the operating system of the citizenry, such as the technology that affects elections, community organizing, and legislation. Gov tech, on the other hand, is technology that supports the operating systems of government.

Many organizations (e.g., Code for America, Smart Chicago, and the MIT Civic Media Lab) as well as numerous individuals promote open sourced civic tech development, but not all civic technology projects are free and open sourced. Some organizations, including those in the Knight report, develop projects without open sourcing their code. This does not, however, preclude a project from being a force for the public good or being defined as “civic tech.”

Even with the impressive momentum behind government and civic tech, there remains a dearth of research on how new technologies are affecting justice systems. Individual researchers are exploring topics related to technology and criminal justice, but there are no organized programs to monitor and evaluate the role of technology on social justice. Some of the better-known civic and legal technology initiatives in academic institutions are producing valuable work, but even they do not focus on criminal law and justice systems. These include Stanford Law School’s CodeX Center for Legal Informatics, the Civic Media Lab at MIT, and the A2J Author Program at Chicago-Kent Law School.

CodeX is focused on how technology is evolving the practice of law generally. With a research focus on legal informatics and computational law, the center also develops programs to improve legal practices but less attention is paid to criminal justice. MIT’s Civic Media Lab works on a broad selection of projects, but few address the operations of the criminal justice system. A2J Author is one of the older legal aid tech projects in the country and its work is applied in civil and criminal issues, but it does not make its data public. The Legal Services Corporation promotes the use of A2J Author and it only recently began a modest measurement and evaluation program to understand the platform’s impact.

Collectively, these philosophical, social, and organizational forces are bridging technology, government, and people in new ways. They are creating new contours and greater definition to how we interact with government, and they have already begun to usher in new tools and approaches that will continue to change and modernize the criminal justice system. These efforts, however, require greater support and coordination if they are to be taken to scale. Specifically, researchers and practitioners need to organize the new knowledge base of justice technology.


After surveying more than 30 programs, tools, and projects occupying the intersection of criminal justice and technology, the Research & Evaluation Center at John Jay College supports the creation of an initiative to track emerging projects, share information, and evaluate promising practices. The initiative will serve as a platform for research and as a forum to connect technologists and justice stakeholders. To be truly effective, the initiative will incorporate students and the previously incarcerated, and it will respond to the perceptions and needs of the public as well as key actors in the justice system.

number1RECOMMENDATION: Combine the efforts of practitioners, technologists, and private industry to increase the efficacy and fairness of criminal justice technology.

To ensure the effectiveness of future technological developments, there must be accessible space for open discussion and the development of new projects. A bridging initiative would need to foster open dialogue among justice and tech stakeholders and allow them to learn from each other while promoting new ideas and projects. It would also need to provide a place for collaborative projects via Github and other online tools that foster open development and experimentation. The creation of such an initiative would facilitate the implementation of the next four recommendations.

number2RECOMMENDATION: Identify best practices in justice and technology through research and evaluation.

Technology innovations in the justice sector often lack an effective research component to estimate the social impact of projects as well as their capacity to be expanded and scaled. Currently, there are no efforts underway to organize the lessons learned from the growing number of justice tech projects and to create a taxonomy of best practices for the field. The field needs a crowdsourced, best practices portal that covers numerous civic technology project areas. Tech initiatives in the justice sector should be supported for their contribution to social justice and not merely their commercial value and virality. By cataloging various development efforts in a centralized place, a portal would provide easy access to lessons learned and emerging best practices and it could guide the work of individual jurisdictions that are building new projects or expanding existing ones. New efforts in justice tech should include evaluation from the start. Evaluation-compatible data should be accessible to researchers, journalists, public officials, and members of the public. Documentation of best practices and an active research agenda will create greater credibility and support for particular projects and for the growing use of technology in the justice sector.

number3RECOMMENDATION: Develop a professional cadre of boundary-spanners who can shuttle between technologists and justice stakeholders.

Boundary-spanners are needed to ensure that projects are technically feasible but also responsive to system needs. Boundary-spanners could introduce criminal justice stakeholders and reformers to new tools and applications in the tech field, and as justice experts become more familiar with the capacities of technology, they may conceive new applications that tech experts working alone would not anticipate.

When listening to civic technology leaders, one hears a common theme — that criminal justice projects are uniquely hard to implement. Even when justice system stakeholders have an interest in developing more tech projects, their needs and perceptions are likely to be complicated by the many public controversies in justice, and their work can be hindered by legal barriers.

A bridging initiative should educate justice stakeholders in the uses of technology and it should expose technologists to the needs of justice systems— all to facilitate clearer and more feasible project descriptions. Justice and tech collaborations should not simply serve the interests of criminal justice officials and experts, nor is this an area that can be dominated by engineers and developers.

Cross-sector collaborations should include a focus on drafting requests for proposals (RFP). Due to various translation issues between system stakeholders and technologists, RFPs for technology projects are often vague, unclear, and written using inaccurate vocabulary. The lack of clarity diminishes project efficacy and makes collaborations more difficult than necessary.

number4RECOMMENDATION: Incorporate students and previously incarcerated persons in technology initiatives.

Current projects in justice and technology do not often include criminal justice students. While there are numerous educational programs that incorporate students into government and civic tech, the justice field needs an initiative that focuses solely on criminal justice and that supports students in building projects and conducting research in the field. An initiative focused on criminal justice and technology must also provide opportunities for previously incarcerated people with technological interests and experience. A fellowship for previously incarcerated persons could draw upon their experience to inform projects and to shape the new research undertaken by the initiative.

number5RECOMMENDATION: Educate and involve the public in the intersection of criminal justice and technology to improve user adoption and understanding.

A justice tech bridging initiative should bring together internet ethicists, criminal justice experts, and previously incarcerated persons to promote a better-informed public debate on the challenges of law enforcement, public safety, and justice. Such an effort should engage in the national conversation on critical justice issues, including technology’s role in the institutional discrimination that plagues criminal justice systems. Hackathons should be used as a catalyst for cross-pollination between criminal justice experts and technologists. The most promising projects emerging from hackathons should be supported by the initiative and improved for possible implementation.


The technology projects described in this report reflect the broad range of innovations underway in the justice field, but the list is not exhaustive. Virtual reality, artificial intelligence, and wearable technology are nascent tools that will be even more important in the future. Tech applications hold great promise for system efficacy, fairness, and reform of justice systems, but the rapid expansion of technology requires greater investment in research and evaluation. Researchers and technologists must collaborate to identify best practices. A coordinated initiative is needed to bridge the worlds of technology, justice systems, and social justice reform. With sufficient support, this new field may have significant and sustainable effects on the quality of justice.


Jason Tashea is a legal tech and criminal justice consultant for the Research and Evaluation Center. He creates tech solutions that improve access to justice and criminal justice systems. Jason graduated from Linfield College and he earned the JD from the University of Oregon.

Recommended Citation
Tashea, Jason (2016). Emerging Justice Technologies and the Need for Evaluation. New York, NY: Research & Evaluation Center, John Jay College of Criminal Justice, City University of New York.

Copyright by the Research & Evaluation Center, John Jay College of Criminal Justice, City University of New York (CUNY).

524 59th Street
New York, NY 10019
Published online January 2016

Any views expressed here are those of the author and do not necessarily reflect the policies of John Jay College, the City University of New York, or its funders. This report was written by Jason Tashea with editing assistance from Megan Lucy. Background research was conducted by Jason Tashea, including crowdsourcing efforts through various civic technology organizations and experts around the United States. Jason Tashea developed the “Expunge Maryland” application in a previous role and he has been involved with the Oregon Federal Court’s effort to iterate the Offender Smartphone Monitoring platform.



Anderson, Monica (2015, October 29). Technology Device Ownership: 2015.

Clarke, Ruthbea Yesner (2014). Civic Tech Fuels U.S. State and Local Government Transformation.

[Code for America]. (2015, October 6). Jenny Montoya-Tansey – Deepening Impact in Safety and Justice – CfA Summit (2015).

[Code for America] (2014, September 24). The Emerging Startup Ecosystem.

Coggeshall, John (2015, November 17). Written interview with James McMillan.

Duggan, Maeve and Lee Rainie (2012, November 25). Cell Phone Activities 2012.

Hansen, Mark (2015, July). Is Google more Accurate than the FBI? The Marshall Project.

Howard, Alex (2012, December 11). Making Dollars and Sense of the Open Data Economy.

Leiner, Barry M., Vinton G. Cerf, David D. Clark, Robert E Kahn, Leonard Kleinrock, Daniel C. Lynch, Jon Postel, Larry G. Roberts, and Stephen Wolff. (2012). Brief History of the Internet. Reston, Virginia: Internet Society.

Levin, Sam (2015, October 7). Racial Profiling Via Nextdoor.com. Eastbay Express.

O’Neil, Daniel (2014). The Smart Chicago Model. The Smart Chicago Collective.

O’Reilly, Tim (2010). Government as a Platform. In Daniel Lathrop and Laurel Ruma (Eds.). Open Government.

Ormachea, Pablo A., Gabe Haarsma, Sasha Davenport, David M. Eagleman (2015). NeuLaw Criminal Record Database: A Resource for Large-scale Analysis of Policy and Behavior. The Journal of Science and Law, 1(1).

Patel, Mayur, Jon Sotsky, Sean Gourley, and Daniel Houghton (2013). The Emergence of Civic Tech: Investments in a Growing Field.

Raymond, Eric S. (2001). The Cathedral and the Bazaar. Sebastopol, CA: O’Reilly Media.

Smith, Jordan and Micah Lee (2015, November 11). Not so Securus. The Intercept.

Smith, Megan and Roy L. Austen (2015, May 18). Launching the Police Data Initiative.

Suszan, Brittany (2014, March 12). Spot Crime – The Public’s Crime Map.

Tolia, Nirav (2015, October 15). Racial Profiling: The Opposite of Being Neighborly.

Ward, Stephanie Francis (2015, September 16). Kyle Rimel: Using Technology to Bring Court Services to Remote Areas.