The New Metrics

New methods in bibliometrics and alternative metrics provide us with information about research impact at both increasingly granular and global levels. Here, editor Elaine Lasda and a cast of expert contributors present a variety of case studies that demonstrate the practical utilization of these new scholarly metrics.

Metrics are a hot topic. Executive leadership, boards of directors, management, and customers are all asking for data-based decisions. As a result, many managers, professionals, and change agents are asked to develop metrics, but have no clear idea of how to produce meaningful ones. Wouldn’t it be great to have a simple explanation of how to collect, analyze, report, and use measurements to improve your organization? Metrics: How to Improve Key Business Results provides that explanation and the tools you'll need to make your organization more effective. Not only does the book explain the “why” of metrics, but it walks you through a step-by-step process for creating a report card that provides a clear picture of organizational health and how well you satisfy customer needs. Metrics will help you to measure the right things, the right way—the first time. No wasted effort, no chasing data. The report card provides a simple tool for viewing the health of your organization, from the outside in. You will learn how to measure the key components of the report card and thereby improve real measures of business success, like repeat customers, customer loyalty, and word-of-mouth advertising. This book: Provides a step-by-step guide for building an organizational effectiveness report card Takes you from identifying key services and products and using metrics, to determining business strategy Provides examples of how to identify, collect, analyze, and report metrics that will be immediately useful for improving all aspects of the enterprise, including IT

How the increasing reliance on metrics to evaluate scholarly publications has produced new forms of academic fraud and misconduct. The traditional academic imperative to “publish or perish” is increasingly coupled with the newer necessity of “impact or perish”—the requirement that a publication have “impact,” as measured by a variety of metrics, including citations, views, and downloads. Gaming the Metrics examines how the increasing reliance on metrics to evaluate scholarly publications has produced radically new forms of academic fraud and misconduct. The contributors show that the metrics-based “audit culture” has changed the ecology of research, fostering the gaming and manipulation of quantitative indicators, which lead to the invention of such novel forms of misconduct as citation rings and variously rigged peer reviews. The chapters, written by both scholars and those in the trenches of academic publication, provide a map of academic fraud and misconduct today. They consider such topics as the shortcomings of metrics, the gaming of impact factors, the emergence of so-called predatory journals, the “salami slicing” of scientific findings, the rigging of global university rankings, and the creation of new watchdogs and forensic practices.

How the obsession with quantifying human performance threatens business, medicine, education, government—and the quality of our lives Today, organizations of all kinds are ruled by the belief that the path to success is quantifying human performance, publicizing the results, and dividing up the rewards based on the numbers. But in our zeal to instill the evaluation process with scientific rigor, we've gone from measuring performance to fixating on measuring itself—and this tyranny of metrics now threatens the quality of our organizations and lives. In this brief, accessible, and powerful book, Jerry Muller uncovers the damage metrics are causing and shows how we can begin to fix the problem. Filled with examples from business, medicine, education, government, and other fields, the book explains why paying for measured performance doesn't work, why surgical scorecards may increase deaths, and much more. But Muller also shows that, when used as a complement to judgment based on personal experience, metrics can be beneficial, and he includes an invaluable checklist of when and how to use them. The result is an essential corrective to a harmful trend that increasingly affects us all.

Where do the digital humanities really come from? Are they really news? What are the theoretical and technical influences that participate in this scientific field that arouses interest and questions? This book tries to show and explain the main theories and methods that have allowed their current constitution. The aim of the book is to propose a new way to understand the history of digital humanities in a broader perspective than the classic history with the project of Robert Busa. The short digital humanities perspective neglects lots of actors and disciplines. The book tries to show the importance of other fields than humanities computing like scientometry, infometry, econometry, mathematical linguistics, geography and documentation.

The Definitive Guide to Quantifying, Classifying, and Measuring Enterprise IT Security Operations Security Metrics is the first comprehensive best-practice guide to defining, creating, and utilizing security metrics in the enterprise. Using sample charts, graphics, case studies, and war stories, Yankee Group Security Expert Andrew Jaquith demonstrates exactly how to establish effective metrics based on your organization’s unique requirements. You’ll discover how to quantify hard-to-measure security activities, compile and analyze all relevant data, identify strengths and weaknesses, set cost-effective priorities for improvement, and craft compelling messages for senior management. Security Metrics successfully bridges management’s quantitative viewpoint with the nuts-and-bolts approach typically taken by security professionals. It brings together expert solutions drawn from Jaquith’s extensive consulting work in the software, aerospace, and financial services industries, including new metrics presented nowhere else. You’ll learn how to: • Replace nonstop crisis response with a systematic approach to security improvement • Understand the differences between “good” and “bad” metrics • Measure coverage and control, vulnerability management, password quality, patch latency, benchmark scoring, and business-adjusted risk • Quantify the effectiveness of security acquisition, implementation, and other program activities • Organize, aggregate, and analyze your data to bring out key insights • Use visualization to understand and communicate security issues more clearly • Capture valuable data from firewalls and antivirus logs, third-party auditor reports, and other resources • Implement balanced scorecards that present compact, holistic views of organizational security effectiveness

Annette Holler introduces two new depreciation-adjusted value metrics. Furthermore, a case study illustrates the depreciation-related bias. A regression analysis adds to previous evidence on associations of value creation with stock returns and firm values.

The starkly different ways that American and French online news companies respond to audience analytics and what this means for the future of news When the news moved online, journalists suddenly learned what their audiences actually liked, through algorithmic technologies that scrutinize web traffic and activity. Has this advent of audience metrics changed journalists’ work practices and professional identities? In Metrics at Work, Angèle Christin documents the ways that journalists grapple with audience data in the form of clicks, and analyzes how new forms of clickbait journalism travel across national borders. Drawing on four years of fieldwork in web newsrooms in the United States and France, including more than one hundred interviews with journalists, Christin reveals many similarities among the media groups examined—their editorial goals, technological tools, and even office furniture. Yet she uncovers crucial and paradoxical differences in how American and French journalists understand audience analytics and how these affect the news produced in each country. American journalists routinely disregard traffic numbers and primarily rely on the opinion of their peers to define journalistic quality. Meanwhile, French journalists fixate on internet traffic and view these numbers as a sign of their resonance in the public sphere. Christin offers cultural and historical explanations for these disparities, arguing that distinct journalistic traditions structure how journalists make sense of digital measurements in the two countries. Contrary to the popular belief that analytics and algorithms are globally homogenizing forces, Metrics at Work shows that computational technologies can have surprisingly divergent ramifications for work and organizations worldwide.

‘Represents the culmination of an 18-month-long project that aims to be the definitive review of this important topic. Accompanied by a scholarly literature review, some new analysis, and a wealth of evidence and insight... the report is a tour de force; a once-in-a-generation opportunity to take stock.’ – Dr Steven Hill, Head of Policy, HEFCE, LSE Impact of Social Sciences Blog ‘A must-read if you are interested in having a deeper understanding of research culture, management issues and the range of information we have on this field. It should be disseminated and discussed within institutions, disciplines and other sites of research collaboration.’ – Dr Meera Sabaratnam, Lecturer in International Relations at the School of Oriental and African Studies, University of London, LSE Impact of Social Sciences Blog Metrics evoke a mixed reaction from the research community. A commitment to using data and evidence to inform decisions makes many of us sympathetic, even enthusiastic, about the prospect of granular, real-time analysis of our own activities. Yet we only have to look around us at the blunt use of metrics to be reminded of the pitfalls. Metrics hold real power: they are constitutive of values, identities and livelihoods. How to exercise that power to positive ends is the focus of this book. Using extensive evidence-gathering, analysis and consultation, the authors take a thorough look at potential uses and limitations of research metrics and indicators. They explore the use of metrics across different disciplines, assess their potential contribution to the development of research excellence and impact and consider the changing ways in which universities are using quantitative indicators in their management systems. Finally, they consider the negative or unintended effects of metrics on various aspects of research culture. Including an updated introduction from James Wilsdon, the book proposes a framework for responsible metrics and makes a series of targeted recommendations to show how responsible metrics can be applied in research management, by funders, and in the next cycle of the Research Excellence Framework. The metric tide is certainly rising. Unlike King Canute, we have the agency and opportunity – and in this book, a serious body of evidence – to influence how it washes through higher education and research.