Programming Persistent Memory

Beginning and experienced programmers will use this comprehensive guide to persistent memory programming. You will understand how persistent memory brings together several new software/hardware requirements, and offers great promise for better performance and faster application startup times—a huge leap forward in byte-addressable capacity compared with current DRAM offerings. This revolutionary new technology gives applications significant performance and capacity improvements over existing technologies. It requires a new way of thinking and developing, which makes this highly disruptive to the IT/computing industry. The full spectrum of industry sectors that will benefit from this technology include, but are not limited to, in-memory and traditional databases, AI, analytics, HPC, virtualization, and big data. Programming Persistent Memory describes the technology and why it is exciting the industry. It covers the operating system and hardware requirements as well as how to create development environments using emulated or real persistent memory hardware. The book explains fundamental concepts; provides an introduction to persistent memory programming APIs for C, C++, JavaScript, and other languages; discusses RMDA with persistent memory; reviews security features; and presents many examples. Source code and examples that you can run on your own systems are included. What You’ll Learn Understand what persistent memory is, what it does, and the value it brings to the industry Become familiar with the operating system and hardware requirements to use persistent memory Know the fundamentals of persistent memory programming: why it is different from current programming methods, and what developers need to keep in mind when programming for persistence Look at persistent memory application development by example using the Persistent Memory Development Kit (PMDK)Design and optimize data structures for persistent memoryStudy how real-world applications are modified to leverage persistent memoryUtilize the tools available for persistent memory programming, application performance profiling, and debugging Who This Book Is For C, C++, Java, and Python developers, but will also be useful to software, cloud, and hardware architects across a broad spectrum of sectors, including cloud service providers, independent software vendors, high performance compute, artificial intelligence, data analytics, big data, etc.

Beginning and experienced programmers will use this comprehensive guide to persistent memory programming. You will understand how persistent memory brings together several new software/hardware requirements, and offers great promise for better performance and faster application startup times—a huge leap forward in byte-addressable capacity compared with current DRAM offerings. This revolutionary new technology gives applications significant performance and capacity improvements over existing technologies. It requires a new way of thinking and developing, which makes this highly disruptive to the IT/computing industry. The full spectrum of industry sectors that will benefit from this technology include, but are not limited to, in-memory and traditional databases, AI, analytics, HPC, virtualization, and big data. Programming Persistent Memory describes the technology and why it is exciting the industry. It covers the operating system and hardware requirements as well as how to create development environments using emulated or real persistent memory hardware. The book explains fundamental concepts; provides an introduction to persistent memory programming APIs for C, C++, JavaScript, and other languages; discusses RMDA with persistent memory; reviews security features; and presents many examples. Source code and examples that you can run on your own systems are included. What You’ll Learn Understand what persistent memory is, what it does, and the value it brings to the industry Become familiar with the operating system and hardware requirements to use persistent memory Know the fundamentals of persistent memory programming: why it is different from current programming methods, and what developers need to keep in mind when programming for persistence Look at persistent memory application development by example using the Persistent Memory Development Kit (PMDK) Design and optimize data structures for persistent memory Study how real-world applications are modified to leverage persistent memory Utilize the tools available for persistent memory programming, application performance profiling, and debugging Who This Book Is For C, C++, Java, and Python developers, but will also be useful to software, cloud, and hardware architects across a broad spectrum of sectors, including cloud service providers, independent software vendors, high performance compute, artificial intelligence, data analytics, big data, etc.

This book introduces readers to emerging persistent memory (PM) technologies that promise the performance of dynamic random-access memory (DRAM) with the durability of traditional storage media, such as hard disks and solid-state drives (SSDs). Persistent memories (PMs), such as Intel's Optane DC persistent memories, are commercially available today. Unlike traditional storage devices, PMs can be accessed over a byte-addressable load-store interface with access latency that is comparable to DRAM. Unfortunately, existing hardware and software systems are ill-equipped to fully avail the potential of these byte-addressable memory technologies as they have been designed to access traditional storage media over a block-based interface. Several mechanisms have been explored in the research literature over the past decade to design hardware and software systems that provide high-performance access to PMs. Because PMs are durable, they can retain data across failures, such as power failures and program crashes. Upon a failure, recovery mechanisms may inspect PM data, reconstruct state and resume program execution. Correct recovery of data requires that operations to the PM are properly ordered during normal program execution. Memory persistency models define the order in which memory operations are performed at the PM. Much like memory consistency models, memory persistency models may be relaxed to improve application performance. Several proposals have emerged recently to design memory persistency models for hardware and software systems and for high-level programming languages. These proposals differ in several key aspects; they relax PM ordering constraints, introduce varying programmability burden, and introduce differing granularity of failure atomicity for PM operations. This primer provides a detailed overview of the various classes of the memory persistency models, their implementations in hardware, programming languages and software systems proposed in the recent research literature, and the PM ordering techniques employed by modern processors.

When creating complex Java enterprise applications, do you spend a lot of time thumbing through a myriad of books and other resources searching for what you hope will be the API that's right for the project at hand?Java Database Best Practices rescues you from having to wade through books on each of the various APIs before figuring out which method to use! This comprehensive guide introduces each of the dominant APIs (Enterprise JavaBeans, Java Data Objects, the Java Database Connectivity API (JDBC) as well as other, lesser-known options), explores the methodology and design components that use those APIs, and then offers practices most appropriate for different types and makes of databases, as well as different types of applications.Java Database Practices also examines database design, from table and database architecture to normalization, and offers a number of best practices for handling these tasks as well. Learn how to move through the various forms of normalization, understand when to denormalize, and even get detailed instructions on optimizing your SQL queries to make the best use of your database structure. Through it all, this book focuses on practical application of these techniques, giving you information that can immediately be applied to your own enterprise projects.Enterprise applications in today's world are about data-- whether it be information about a product to buy, a user's credit card information, or the color that a customer prefers for their auto purchases. And just as data has grown in importance, the task of accessing that data has grown in complexity. Until now, you have been left on your own to determine which model best suits your application, and how best to use your chosen API. Java Database Practices is the one stop reference book to help you determine what's appropriate for your specific project at hand. Whether it's choosing between an alphabet soup of APIs and technologies--EJB, JDO, JDBC, SQL, RDBMS, OODBMS, and more on the horizon, this book is an indispensable resource you can't do without.

Elements of Programming provides a different understanding of programming than is presented elsewhere. Its major premise is that practical programming, like other areas of science and engineering, must be based on a solid mathematical foundation. The book shows that algorithms implemented in a real programming language, such as C++, can operate in the most general mathematical setting. For example, the fast exponentiation algorithm is defined to work with any associative operation. Using abstract algorithms leads to efficient, reliable, secure, and economical software.

Emerging fast, byte-addressable Persistent Memory (PM) considerably increases the storage performance compared to traditional disks and makes it possible to build complex data structures that can survive system failures. However, programming for PM is challenging, not least because it combines well-known programming challenges like locking, memory management, and pointer safety with novel PM-specific bug types. It also requires logging updates to PM to facilitate recovery after a crash. A misstep in any area can corrupt data, leak resources, or prevent successful recovery after a crash. Additionally, its high price limits its capacity to scale as same as traditional block devices. This dissertation first presents Corundum, a Rust-based library with an idiomatic PM programming interface that leverages Rust's type system to avoid the most common PM programming bugs statically. Corundum lets programmers develop persistent data structures using familiar Rust constructs and have confidence that they will be free of those bugs. We have implemented Corundum and found its performance to be as good as or better than Intel's widely-used PMDK library, HP's Atlas, Mnemosyne, and go-pmem. Then, the dissertation presents Carbide, a robust multilingual programming framework that allows developing safe data structures in Corundum and using them in C++. Carbide strictly checks the data structure implementation for PM safety and allows unrestrictedly using them in C++ with an accurate persistent type checking process. Our experimental results show that Carbide not only does not incur a significant slowdown, but it is also faster than purely using Corundum in some of our benchmarks due to the flexibility that it provides. Finally, it introduces Ziggurat, a tiered file system that combines PM and slow disks to create a high-performance and extensive storage system. Ziggurat steers incoming writes to PM, DRAM, or disk depending on application access patterns, write size, and the likelihood that the application will stall until the write completes. Our experimental results show that with a small amount of PM and a large SSD, Ziggurat achieves up to 38.9X and 46.5X throughput improvement compared with EXT4 and XFS running on an SSD alone, respectively.

There are several theories of programming. The first usable theory, often called "Hoare's Logic", is still probably the most widely known. In it, a specification is a pair of predicates: a precondition and postcondition (these and all technical terms will be defined in due course). Another popular and closely related theory by Dijkstra uses the weakest precondition predicate transformer, which is a function from programs and postconditions to preconditions. lones's Vienna Development Method has been used to advantage in some industries; in it, a specification is a pair of predicates (as in Hoare's Logic), but the second predicate is a relation. Temporal Logic is yet another formalism that introduces some special operators and quantifiers to describe some aspects of computation. The theory in this book is simpler than any of those just mentioned. In it, a specification is just a boolean expression. Refinement is just ordinary implication. This theory is also more general than those just mentioned, applying to both terminating and nonterminating computation, to both sequential and parallel computation, to both stand-alone and interactive computation. And it includes time bounds, both for algorithm classification and for tightly constrained real-time applications.

Efficiency is a crucial concern across computing systems, from the edge to the cloud. Paradoxically, even as the latencies of bottleneck components such as storage and networks have dropped by up to four orders of magnitude, software path lengths have progressively increased due to overhead from the very frameworks that have revolutionized the pace of information technology. Such overhead can be severe enough to overshadow the benefits from switching to new technologies like persistent memory and low latency interconnects. Resource Proportional Software Design for Emerging Systems introduces resource proportional design (RPD) as a principled approach to software component and system development that counters the overhead of deeply layered code without removing flexibility or ease of development. RPD makes resource consumption proportional to situational utility by adapting to diverse emerging needs and technology systems evolution. Highlights: Analysis of run-time bloat in deep software stacks, an under-explored source of power-performance wastage in IT systems Qualitative and quantitative treatment of key dimensions of resource proportionality Code features: Unify and broaden supported but optional features without losing efficiency Technology and systems evolution: Design software to adapt with changing trade-offs as technology evolves Data processing: Design systems to predict which subsets of data processed by an (analytics or ML) application are likely to be useful System wide trade-offs: Address interacting local and global considerations throughout software stacks and hardware including cross-layer co-design involving code, data and systems dimensions, and non-functional requirements such as security and fault tolerance Written from a systems perspective to explore RPD principles, best practices, models and tools in the context of emerging technologies and applications This book is primarily geared towards practitioners with some advanced topics for researchers. The principles shared in the book are expected to be useful for programmers, engineers and researchers interested in ensuring software and systems are optimized for existing and next generation technologies. The authors are from both industry (Bhattacharya and Voigt) and academic (Gopinath) backgrounds.

A guide to writing computer code covers such topics as variable naming, presentation style, error handling, and security.

Over the past two decades, there has been a huge amount of innovation in both the principles and practice of operating systems Over the same period, the core ideas in a modern operating system - protection, concurrency, virtualization, resource allocation, and reliable storage - have become widely applied throughout computer science. Whether you get a job at Facebook, Google, Microsoft, or any other leading-edge technology company, it is impossible to build resilient, secure, and flexible computer systems without the ability to apply operating systems concepts in a variety of settings. This book examines the both the principles and practice of modern operating systems, taking important, high-level concepts all the way down to the level of working code. Because operating systems concepts are among the most difficult in computer science, this top to bottom approach is the only way to really understand and master this important material.