Recent reports from the Energy Information Administration (EIA) have indicated a 14 percent decline in U.S. energy-related carbon emissions from 2005 to 2017. The main driver behind this emissions reduction is that more electricity has been generated from natural gas than from other fossil fuels, and natural gas is a less carbon-intensive fuel than either coal or petroleum.1
The transition to natural gas (gas) generated electricity has been driven by the abundance of inexpensive gas produced from unconventional resources (unconventionals) – such as shales – using a combination of directional drilling and hydraulic fracturing. The gas production rate from hydraulically fractured wells increased from less than 5 billion cubic feet per day in 2000 to over 50 billion cubic feet per day in 2015 – over two-thirds of total US gas production.2 Gas production from unconventionals provides approximately $100 billion of benefit to consumers each year.3
The commercial development of these technologies is sometimes attributed to the efforts of an oil-industry entrepreneur, George Mitchell, and his company Mitchell Energy. While Mitchell deserves great credit, both for unusual persistence and for the ultimate development of an approach that combined horizontal drilling and hydraulic fracturing, there were other factors that made critical contributions to the commercial success of gas extraction from unconventionals.4 One of the most significant factors was government-funded research and development that helped move the technology toward commercialization, as noted by Dan Steward, former Vice President at Mitchell Energy.5
“They did a hell of a lot of work, and I can’t give them enough credit for that. [The Department of Energy] started it, and other people took the ball and ran with it. You cannot diminish DOE’s involvement.”
– Dan Steward, former Executive Vice President, Mitchell Energy
In the sections below, we provide a brief overview of the process typically associated with the development of energy technologies. This is followed by descriptions of the publicly-funded initiatives that helped move gas extraction from unconventionals to commercialization.
Section 1: The Energy Technology Development Process
The development of advanced energy technologies commonly involves a down-selection or filtering process of technologies from a large portfolio of laboratory/bench-scale projects to a small portfolio of large-scale pilot projects, as portrayed in Figure 1.6 The process involves multiple steps because there is insufficient information generated at the laboratory/bench scale to confidently predict eventual commercial success. The rationale for government funding of technology development is to offset risk that private industry will not absorb all along the development pathway. In recent years, the U.S. Department of Energy (DOE) has adopted the Technology Readiness Level (TRL) concept to track the progress of technology development. The TRL approach is a measurement system that supports evaluation of the maturity of a particular technology based upon the scale, degree of system integration, and test environment in which the technology has been successfully demonstrated. TRL scores associated with different stages of technology development are also shown in Figure 1.
Figure 1. Energy technology development progression6
Section 2: Public Contributions to the Commercialization of Fracturing Technologies
Natural gas has been extracted from unconventional formations as early as the 1820s7 , but high costs made widespread deployment uneconomic. Advancements were needed in three principal technological areas in order to improve the economics of unconventional gas extraction:
- Drilling (directional drilling and drill bits)
- Hydraulic fracturing, and
- Imaging to facilitate subsurface characterization
Integration of these advancements was then required to complete the process.
Government support of R&D starting in the 1970s, drove down costs of unconventional extraction and led to the economic benefits we see associated with natural gas today. Significant milestones in the technology development pathway are summarized in Figure 2 and described in additional detail in the sections below. The TRL approach had not been implemented at DOE at that time, so the TRL scores shown in Figure 2 and in the sections below represent approximations of levels for each part of the development process.
The principal programs/components of federal involvement in the development of unconventional gas extraction technologies included:
- The Eastern Gas Shales Project;
- Collaboration with the Gas Research Institute (GRI);
- Three-dimensional microseismic imaging;
- Drill bit technology development;
- The Section 29 production tax credit for unconventional gas.7
Information regarding each of these components is provided below.
Figure 2. Summary of technological advancements that, when integrated, led to economic extraction of shale gas8
Eastern Gas Shales Project
Energy crises in the early and mid-1970s led to the creation of the Energy Research and Development Administration (ERDA), which later became the Department of Energy.9 One of ERDA’s goals was to promote access to unconventional gas resources10 in order to overcome declining gas production. In 1975, ERDA partnered with industry to drill the first Appalachian Basin directional wells to tap shale gas and the first horizontal shale well to employ seven individual hydraulically fractured intervals.11 In addition, two federal engineers patented an early directional shale drilling technique that allowed operators to span larger radial expanses of shale deposits.12 Building from these TRL 2 to 4 successes, ERDA launched the Eastern Gas Shales Program (EGSP), which focused on three key issues:9
- Developing a resource inventory of the Devonian shales in the Appalachian, Michigan, and Illinois Basins – uncertainty regarding the magnitude of the shale gas resource prevented industry from knowing where to drill;
- Determining the recoverable reserves of Devonian shale gas – exploration into this high risk, limited economic resource was hindered because cores, logs and maps of the shale gas depositional environment and tectonic activity were unavailable;
- Determining the most effective technologies for extracting this gas at least cost – technology for shale gas recovery was crude, and the benefits of costly horizontal drilling in gas shales had never been investigated.10
Funding for EGSP totalled about $274 million in 2017 dollars over the course of the program, with the highest spending in the early stages of the project, but was significantly reduced after 1982. 13,14
“The DOE’s Eastern Gas Shales Project determined there was a hell of a lot of gas in shales. It was the biggest accumulation of knowledge and data to date. It set the stage for people to have the basic background and caused people to start asking questions… Mitchell was interested in Barnett [shale] and his geophysicist said, ‘It looks similar to the Devonian [shale], and the government’s already done all this work on the Devonian.”
– Dan Steward, former Executive Vice President, Mitchell Energy
The EGSP operated as a cost- and risk-sharing collaboration with industry from the very beginning, which encouraged private investment in technology development.15 The effort involved more than 40 partners from industry (e.g., Columbia Gas Corp., Petroleum Technology Corp.), universities (e.g., West Virginia University, University of North Carolina), and research institutions (e.g., Sandia National Laboratory, Lawrence Livermore National Laboratory). Initial unconventional shale gas resource inventories were largely completed through contracts with state geological surveys (e.g., Illinois, Pennsylvania) and universities. Descriptions of the resource base were contracted to universities, research institutions, and industry.12 Cost-sharing arrangements with oil and gas operators were executed for drilling and coring of approximately 35 demonstration wells, which demonstrated the impact of horizontal drilling on shale gas recovery. In all, the EGSP consisted of nearly 30 key projects that spanned developmental stages ranging from TRL 2 (early shale drilling and fracturing experiments) to TRL 6 (the first successful multi-fracture horizontal well, drilled in West Virginia in 1986). The project produced over 110 major technical reports along with maps, evaluations, and cross sections relevant for unconventional shale gas exploration and production.16
Gas Research Institute
At about the same time that the EGSP was initiated, the Federal Energy Regulatory Commission (FERC) approved a charge on interstate gas sales to fund gas technology R&D at the Gas Research Institute (GRI) – an industry collaborative research organization supported by publicly-administered funds.17 GRI’s early budget was approximately $40 million per year, growing to $200 million per year in the 1990s, but less than 15% of this total budget was focused on unconventional production at its peak.15
The DOE and GRI coordinated extensively and complemented each other. DOE concentrated on basic research to develop new exploration and production techniques, while the GRI program focused on commercialization and deployment of technologies for industry. DOE representatives participated on guidance committees related to GRI research, and the two organizations conducted semi-annual meetings to discuss high level planning and research coordination. Both groups hired external experts and involved industry partners in R&D and commercialization efforts.15 The public-private partnership helped align industry efforts with greater strategic DOE goals and allowed industry knowledge to update and inform government priorities. The DOE partnership with GRI also assisted technology diffusion because the presence of federal support required all industry partners involved in research to give up intellectual property claims to their findings and full publication of findings. In the early 1990s, DOE and GRI worked with several companies to conduct publicly subsidized, cost-shared demonstration projects in which several wells containing multiple hydraulically fractured zones were completed, including Mitchell Energy’s famous first horizontal well in the Barnett Shale in 1991. These TRL 7 efforts were the first demonstrations of the multiple-fracture horizontal drilling method in gas shales that would become the norm for all shale gas recovery.16
Three-dimensional microseismic imaging
Microseismic imaging allows well operators to detect and measure faint signals at the base of deep wells to generate a map of the fractures in a particular shale or sandstone formation. The ability to generate this type of map leads to lower drilling risks, higher well productivity, and lower overall costs.16 Advances in microseismic mapping were made at several national laboratories based on low TRL level (2 to 3) R&D associated with geothermal, mining, and military applications. Open access to cutting-edge computing facilities at the national laboratories played a significant role in advancing the technology.
In 1985, Sandia National Laboratory, in partnership with GRI, led a team focused on the development and demonstration of techniques for real-time microseismic mapping to be used in hydraulic fracturing and gas production. This effort was conducted using several experimental wells that had been developed with federal funding in Colorado (called the Multiwell Experiment (MWX) and later called the M-Site). This TRL 5 to 6 level work advanced the technology considerably. Real-time microseismic fracture mapping technology was subsequently successfully validated (TRL 7) by the Sandia-led team in the mid1990s, aided by advances in measurement-while-drilling technologies previously developed in partnership with industry. These microseismic mapping techniques were rapidly commercialized (TRL 9) and improved upon thereafter, as noted by Dan Hill, head of the petroleum engineering department at Texas A&M University.18
“[Microseismic monitoring], which has now been applied to tens of thousands of fracture treatments, and which is now itself a multi-million dollar industry, has allowed engineers to greatly improve hydraulic fracturing and well completion practices. The development of microseismic monitoring of hydraulic fracture treatments was clearly enabled by the Department of Energy funded research that proved its viability”
– Dan Hill, Petroleum Engineering Department Head, Texas A&M University
Drill bit technology development
In the 1970s, DOE supported R&D led by Sandia laboratory focused on the development of polycrystalline diamond compact (PDC) drill bits for high density, high-temperature applications needed to drill geothermal wells. PDC drill bits are made by coating drill cutters with graphite powder and sintering them under extreme pressures to convert the graphite to a millimeter-wide layer of synthetic diamond. This allows the drill bit to cut harder rock, drill faster, and last longer.17 Sandia partnered with General Electric to develop laboratory test procedures and computer models of drill bit performance. As part of this effort, the technology advanced from TRL 2 through TRL 5. Results from these laboratory tests and Sandia’s computer modelling of drill bit performance, as well as further industry collaborations, spurred additional applied research and development and eventual commercialization (TRL 9) by private industry. Despite being developed originally for geothermal energy applications, PDC drill bits came to be widely used for unconventional oil and gas well drilling in the 2000s, accounting for as much as 60% of all drilled footage in 2005.17 A recent study estimates that PDC drill bits yielded cost savings of $15.6 billion from 1982 to 2008, with half of this added value attributed to the DOE’s investment of $26.5 million.19
Section 29 production tax credit
The Section 29 production tax credit was included as part of the Windfall Profits Tax Act of 1980. It applied to natural gas production from unconventional natural gas wells (shale, tight sands, and coal-bed methane) drilled in between 1980 and 1992, and extended to natural gas produced from those wells until 2002.18 This tax credit generated tax savings of about $10 billion for operators across all unconventional gas sources and contributed to a more than doubling in production of unconventional gas from 1980 to 2002. Even small operators who lacked substantial tax liabilities were able to benefit from the credits by engaging in tax equity financing transactions in which they effectively ‘sold’ their credits to larger firms. This led to more investor interest and leveraged more private dollars in unconventional gas than existed previously. But beyond those more direct financial implications, the credit prompted industry to drill more wells and collect more data, driving down costs through learning-by-doing. This allowed unconventional gas resources to remain economic even after the Section 29 tax credits expired.20
Section 3: Conclusion
The advancements made in drilling technologies, hydraulic fracturing, and microseismic imaging that were supported by government programs and initiatives clearly played a pivotal role in advancing technologies that were ultimately integrated and commercialized by industry and that have led to the abundance of U.S. natural gas. Three significant lessons that can be gleaned from this technology development process are:
- Government investment in R&D can reduce risks that industry is unwilling to take on
- Public-private partnerships in terms of both investment and research direction throughout the development process is critical
- Energy technology development is often a long-term process that can ultimately result in significant economic benefits
Lesson #1: Government investment reduces risk
In the 1970s when federal programs were initiated, private sector R&D funds were declining, and industry was unwilling to take on the risks of ongoing, unproductive exploration using experimental methods. The federal government did not take over private sector innovation, but instead lowered risks following the energy technology development process described above. Alex Crawley, a former Program Director at ERDA who oversaw a significant portion of the federal research that went into the development of shale gas fracturing technologies, including the ESGP, described the outlook of the industry at the time as follows:21
“As far as shale is concerned, I don’t know that industry would have ever taken a look at it without the federal program.”
– Alex Crawley, former Program Director, ERDA
Federal government basic research was critical to technology advancement, and government participation in technology demonstrations lowered industry exploration risks and demonstrated the market and emissions reduction potential of unconventional gas.
Lesson #2: Public-private partnership enhances probability of success
Public-private partnerships played a critical role in ESGP and GRI. They ensured that R&D was effectively directed and helped speed the diffusion of research findings and associated technologies.20 Terry Engelder, a geologist at Penn State University who conducted government-funded research documenting the behavior of underground fracturing, said this:22
“The Government got it really right. In terms of a symbol of an effective public-private venture, it’s shale gas.”
– Terry Englender, Department of Geology, Penn State University
While the private sector – e.g., Mitchell Energy – generally drove applied R&D, there were multiple instances throughout the development process when federal applied R&D made significant contributions and led to additional innovation. Dan Steward from Mitchell Energy, as noted above, acknowledged the broad role played by DOE in helping Mitchell commercialize the technology, but he also discussed the vital role of government at various stages in the development process as described below.23
“They helped us to evaluate how much gas was there, and evaluate the critical properties as compared to Devoninan shale of Appalachia basin. They helped us with our first horizontal well. They helped us with pressure build-ups. And we worked with them on crack mapping.”
– Dan Steward, former Executive Vice President, Mitchell Energy
The partnership extended to funding as well. Estimates of funding to support the technology development and commercialization efforts are summarized in Figure 3. As noted, early funding for work at low TRL levels was dominated by direct DOE expenditures. As TRLs increased to the TRL 4 – 7 range, the GRI public-private partnership became the dominant source of funding. Data regarding direct industry funding are not available, but gas production provides a good surrogate for spending by industry. As noted, gas production increases rapidly as TRLs increase beyond TRL 7, at which point DOE and GRI funding ends and industry moves the technology into full commercial operation.
Figure 3. Summary of R&D funding and tax incentives that contributed to the development of shale gas extraction technology24
Lesson #3: Energy technology development takes time, but it’s worth it
While the main focus of Figure 3 is the financial commitment associated with development of technologies for gas extraction from unconventional sources, the time required for ultimate commercialization also provides a significant message. Global events and markets hinted that something needed to be done regarding natural gas supplies in the early 1970s. This prompted initiation of R&D in the late 1970s that led to successful commercialization in the 2000s. Making it through such a long process required vision and persistence on the part of both government policy makers and scientists as well as commercial interests such as George Mitchell. Because the process took so long, there were many opportunities to abandon the effort along the way. In the end, their vision and persistence were proven worthwhile. As noted above, shale gas provides approximately $100 billion of benefit to consumers each year. Even if the cost estimates included in Figure 3 are low by a factor of two, the expenditures were clearly an excellent investment in terms of both time and money. The suite of Fracturing technologies have been instrumental in propelling the United States to become the world’s top producer of oil and natural gas.