False Equivalencies and False Choices

Getting this out of the way first, lest anyone accuse this article of being in the denial camp: Anthropogenic global warming is almost certainly real and will very likely have significant long-term societal, economic, and ecological consequences.  Studying the processes that contribute to AGW, predicting the effects with a high degree of certainty, and finding technological solutions to reduce climate change’s impact should be a high priority of the world’s governments at all levels, as should incentivizing reducing carbon output from all industrial and business sectors.

However, some industries are more ready than others to make impactful changes, by dint of embedded scientific expertise and economic feasibility. The energy sector has low- (and zero) carbon options, for example, and the transportation industry is developing feasible technologies for reducing emissions as well.  The building sector, for all of architects’ good intentions, is still a significant contributor of carbon emissions and architects, by dint of their lack of rigorous scientific and technical training, do not have the necessary expertise to contribute meaningful innovation.

In his recent column in Architect magazine, AIA President Carl Elefante writes that the newest design imperative is reducing and eventually eliminating carbon output from buildings.  “A zero net carbon building sector is the architectural design imperative of our time,” he argues.  In his article, he makes a number problematic arguments.

First, Elefante invokes the changes made to make buildings more fire- and earthquake-resistant: “In 1871, the need for fire-safe buildings rose from the ashes of the Great Chicago Fire. In 1906, from the rubble of San Francisco came understanding that earthquake risk is a design imperative.”  Elefante acknowledges that fires and earthquakes are singular catastrophic events that cause immediate death and destruction; specific deadly events shocked the public into demanding safety reforms that were rapidly baked into building codes.  This is still a false equivalency.  Climate change is acknowledged by the code writers and the International Energy Conservation Code, and requires incrementally improved energy efficiency in envelope design, mechanical and lighting systems.  But since neither architects nor anyone else really knows how to make a building fully zero-carbon, let alone do it for a reasonable cost, there’s no true mandate for architects to follow.

Elefante blunders again by claiming that “in the decade leading up to Paris, the U.S. building sector grew by 20 billion square feet, yet overall energy consumption remained flat.”  This would be quite an accomplishment if true, but it is not.  In fact, from 1999 to 2012 (dates for which data is actually available) commercial building area grew by 20 billion square feet and site energy use grew by 21%.  The positive news from this is that building area grew faster than energy use, but energy use was not “flat.”  We haven’t made as much progress as Elefante or anyone else would wish for.

chart

Chart made for the author by John Scofield, Ph.D., based on his research

Net-zero carbon buildings may be feasible in the future, but saying that it’s the design imperative of our time requires making dishonest claims for what can be achieved today.  Making dishonest claims about what you can accomplish is terrible professional ethics, and for the leading association of architects to advocate that architects act dishonestly is frankly baffling.

Architects and the rest of the design community need to understand more thoroughly how energy use in buildings responds to design decisions, and where the trade-offs are.  For example, if you make a building envelope more air-tight and highly insulated, you likely need to spend more energy bringing in and tempering the fresh air people need to breathe.

There are strategies to deal with these types of problems, but the solutions need more time and more dedicated scientists, engineers, and climate-friendly policy before they’re ready to be incorporated into real-world projects.  Unless owners are expressly consenting to be guinea pigs (while knowing that results are not in any way guaranteed), architects should not be advocating that owners risk their money on unproven strategies.

 

Advertisements

Architects Are Not Like Chefs

Early last year, Chicago Chapter of CSI presented a chapter program during which educators and students from several local architectural schools discussed their curricula. The bulk of the presentation was myriad slides showing pretty pictures of student work. Some of the educators, realizing their audience was CSI members, paid a little lip service to the idea that construction technology (but not specifications) was at least part of what architecture students should learn. None of them spoke more than a sentence or two about the business of architecture, and the word ‘science’ was not uttered once.  

The reason this is especially worth recounting, however, was an astounding statement made by one of the faculty members: he likened the practice of architecture to that of being a chef or a fashion designer.  It’s likely that what he was thinking when he made this claim had far more nuance than I am describing, and by arguing against what I thought he meant I’ll probably be committing a straw man fallacy, but here’s what I understood his claim to be. Architects, chefs and fashion designers follow analogous creative process and their work results in the creation of something finished and suitable for its intended purpose where before was just raw materials.  All the professions require specialized knowledge, talent, and skill/experience that must be built up over the course of a career to ensure success.

Furthermore, all the professions are subject to constraints.  A fashion designer is constrained by the market that dictates what raw materials are available and what the customers can be convinced to buy.  Chefs are constrained by those same factors as well as by public health regulations pertaining to how their products are handled.

Keep in mind that this professor works with architecture students. They are encouraged to think about some of the ways that they’d be constrained by external conditions when practicing, but aren’t really exposed to realistic work conditions and can, as a result, expand their imaginations and create whatever they want.  This works in architecture school, and I suppose you can say it works for chefs and fashion designers as well.  Chefs, fashion designers, and architecture students all create what they want to create, and their output is consumable or disposable products, especially when compared to the lifespan of an actual building.

If you argue that there’s an analogy between chefs and fashion designers on one hand, and architects on the other, I suggest you’re arguing that capital-A Architecture is architects’ product.  That Architecture is what appears in architectural magazines, and just so happens to have been designed for the purpose of publication in magazines.  Those magazines are ogled and then disposed, so there’s your analogy, chef.

Architects that believe they’re like chefs or fashion designers are doing a disservice to the profession, because architects actually provide a professional service, not capital-A Architecture, not a product – let alone a disposable one.  Rather than working to enhance their portfolio on behalf of their own vanity, they are hired to responsibly manage significant sums of their clients’ money, orders of magnitude more than even the nicest restaurant supper or the best Academy Awards couture gown.  They must understand all their clients’ requirements and desires, (including, but not limited to facility purpose, schedule, and budget) then design the facilities that comport with their clients’ needs, and comply with significant technical and legal requirements along the way.  They must communicate their understanding through design documentation back to the owner and then to contractors, who are relying on the architects to have properly designed the project, coordinated all consultants, and furnished proper specifications.

States have granted architects licenses to perform this service because it is serious work, requiring significant and specialized knowledge, and resulting in significant financial risks to our clients and health & safety risks to the public.  Taking that for granted or actively undercutting it by likening ourselves to other creative but not ‘learned’ professions demonstrates that we architects don’t take our role seriously. 

Architects have a duty to protect their clients’ interests first and foremost; chefs and fashion designers have no duty to anyone.

 

Challenges Faced by Specification Consultants

About a month ago I made a career change. I left my job working for a downtown Chicago architectural firm as an in-house specifier and began working for ArchiTech Consulting Inc., a specifications consulting company in the suburbs.  I’m getting used to the environment, small size of the company and the location, which are all big changes for me, but by far the biggest difference is our approach to the work itself.  You might not think that would be the case, seeing as how it’s going from being one type of spec writer to another, but it’s a very different mindset being a consultant.

One of the biggest differences working with other specifiers is the level of expertise, and the respect that expertise has earned us.  ArchiTech has written specifications for thousands of projects for hundreds of different clients, and those specs include hundreds of thousands of spec sections, covering product choices likely in the millions.  That’s a massive amount of institutional memory, experience and knowledge, and between performing computer searches of our archives and probing different specifiers’ memories, I can find answers to almost every eventuality I’ve encountered so far – even a curved, waterjet-cut COR-TEN steel ground sign mounted on architecturally finished cast-in-place concrete.

The depth of the archive, memory and knowledge of the specifiers is our primary strength, along with the ability (which I’m slowly gaining) in efficiently turning out projects on time.   The most common questions around the office are “when was the last time we wrote a spec for [oddball system] and what do we know about it?” and “do you have a technical sales contact from [oddball system manufacturer]?”  There’s almost always a useful answer to either question.  We use that extensive knowledge for the benefit of our clients’ projects, performing technical review while writing specs, coordinating documents and helping the design team avoid pitfalls and mistakes.

Additionally, when we’re called on by product reps, we get higher level technical information, not the basic CEU courses or sales information aimed at architectural or interior designers.

There is a worrying aspect of this role, however. The most obvious difference is that now my clients are (for the most part) architects, where previously I was working for architects’ clients – building owners.  Readers of my earlier post  on Let’s Fix Construction are aware of the weaknesses I pointed out in the architectural profession and how those weaknesses increase building owners’ risks.  I’m now a giant step further removed from the owner – the stakeholder who is impacted to the greatest extent by the decisions I make.  I’m forced to rely on architects to have accurately communicated the criteria by which they made design decisions for their projects, a dicey prospect.  At the same time I’m bound to follow their distinctive (and sometimes capricious) preferences.

I’ve been finding that the specifiers at my new firm, when guidance to the contrary (or any guidance at all for that matter) is lacking, tend to specify systems in such a way as to provide the highest performing facility. While this seems like a prudent approach on its face, the costs of these types of decisions add up, as a result projects may come back costing more than necessary and occasionally need to go through value engineering cost-reduction exercises (though they probably would have needed VE anyway).  When working without sufficient guidance I’d say we manage to write specs that are not wrong. In an ideal situation, we’d certainly prefer to be right. the difference is a much tighter set of deliverable that best meet the owner’s needs and reduces risk.

The ethics of our role sometimes seems muddled. I acknowledge we are working for our clients – architects – and are expected to deliver the product they pay us for, their preferences and peccadillos included.  If we specification consultants believe those peccadillos make for a lesser-performing or more expensive facility, or a more difficult construction process, or create undue risk for the owner, contractor or design professional, what is the best way for us to carry out our work? What is our duty, and to whom?  Is it enough for us to document our opinions without knowing if the architect considered our expertise and passed along our advice to their clients?  Should we be specifying the highest performing products without knowing whether or not the budget can support them?  These are some of the challenging questions consultants face when we work.

I suggest that as we start every project, whether in the role of architect or independent specifier, it should be incumbent upon us to at the very least get an understanding of where the project should fall on the value <———> performance continuum, and make sure the whole team is on the same page. Architects should understand this in any case – what are the project goals and criteria that they are designing toward, and what is the budget?  Spec consultants have the knowledge to specify projects anywhere along the value/performance continuum, and we’re certainly better off if we can do it correctly from the start.   When this information is deeply understood and freely shared, everyone can be on the same page. The spec consultant can be an integral part of the team to make sure the project meets the owner’s criteria, the architect’s and engineering consultants’ designs will better meet their client’s needs, and risk of conflict and loss can be reduced.

Architects and Fiduciary Duties

At its January 26, 2016 chapter meeting, the Chicago Chapter of CSI was treated to a presentation by Ujjval Vyas, Ph.D., JD, entitled “From 19th Century Gentlemen to 21st Century professionals:  Fiduciary Duties and the New Reality of A/E Practice.”  What Vyas outlined during this talk amounts to a complete sea-change in the way architectural services are delivered.

The “New Reality” Vyas presented isn’t reality yet but he demonstrates that it is coming about as the big insurance companies realize that his work has been thoroughly researched and has detailed and consistent logic and precedent.  Once insurers are fully on board, architects will require an abruptly different mindset when it comes to their work.

Licensed Learned Professionals

There is a category of professional that includes doctors, lawyers and accountants that are all similar to one another in significant ways.  They operate with state-granted monopoly (through licensure), their work involves significant financial or personal risk to their clients, and they enjoy a significant knowledge asymmetry between themselves and their clients in their areas of expertise.[1]

Attributes for Some Licensed Learned Professionals[2]

fiduciary-chart

As indicated in this chart displayed by Vyas during his presentation, architects share numerous attributes with the other licensed learned professions, but architects somehow have not operated under the assumption that they have a fiduciary duty to their clients and instead use a contractual model usually based on AIA Contract Documents and/or professional liability tort notions based on around “standard of care.”  I acknowledge my own lack of education around these complex concepts and have tried to make sense of them as best I can in order to communicate them here, but I’m certainly inaccurate in some measure of the specific detail.

So what makes design professionals different from other licensed learned professionals in terms of fiduciary duty?  In order to address this question, it’s first important to understand what is the fiduciary duty.

What is the Fiduciary Duty?

An excellent definition of fiduciary duty has been provided by Robert H. Sitkoff:

The law tends to impose fiduciary obligation in circumstances that present what economists call a principal–agent or agency problem.  An agency problem arises whenever one person, the principal, engages another person, the agent, to undertake imperfectly observable discretionary actions that affect the welfare of the principal. Agency problems therefore arise not only in relationships governed by the common law of agency, but also in trust law, corporate law, and a host of other contexts.

Agency problems are pervasive because no one has the skills necessary to do everything for himself and because every undertaking has an opportunity cost. By delegating a task to an agent, the principal benefits from specialist service and is freed to undertake some other activity. But these benefits come at the cost of being made vulnerable to abuse if the agent is given discretion the exercise of which cannot easily be observed or verified. In such circumstances, the agent may be tempted to favor the agent’s interests when they diverge from those of the principal. The losses and other inefficiencies resulting from this misalignment of interests are called agency costs.[3]

In general, a fiduciary duty exists in business relationships where there are asymmetries of power, knowledge or prestige and the client lacks capacity for oversight.  It is an affirmative duty from the professional (agent) to the client (principal) and includes the requirement that the agent provide non-negligent services, fulfill the terms of any contractual elements of their engagements and fulfill duties of care and loyalty to the principal.

The care and loyalty of a fiduciary relationship is its most obvious differentiating property and is more readily understood in the medical and law arenas.  When you visit your doctor, the practitioner (who is a learned professional licensed by the state to practice medicine) has a fiduciary duty to protect your interests and to use best professional judgment on your behalf.  You lack the knowledge to diagnose and treat yourself; that’s the asymmetry of knowledge.  The doctor has a duty to attempt to determine the cause of the symptoms and suggest a remedy, and to explain as well as possible what might happen, and obtain your informed consent for any treatments undertaken.  This must be done without outside interests influencing the practitioner; only the doctor’s professional knowledge are brought to bear.  In reality, there may be outside pressures impacting a doctor’s practice (reimbursement rates from insurance companies, for example, or the perceived risk of increased malpractice claim liability), but legally physicians must put their patients’ needs above those other considerations.

As it stands right now, architects do not act as though they legally have a fiduciary duty to their clients.  Vyas argues this is due to the history of the profession being one where instead of having clients, architects until would have patrons.  In the modern world, the relationship between architects and clients is established in lengthy agreements outlining scope of services, various responsibilities, limitations of liability, and resolution of disputes.  These documents include no understanding of the affirmative nature of fiduciary duty and the loyalty and knowledge it requires.[4]

A building owner has a need (a new facility, for example), but no knowledge how to make it into reality.  Under the fiduciary duty model, owners are entitled to expect that by hiring an architect, they’re getting someone who will understand their needs and translate them into technical information and at the same time will actively protect their interests by avoiding conflicts of interest and obtaining informed consent for any solutions proposed.

As the model of practice changes to one of fiduciary duty, the profession becomes one based completely on providing services that are grounded in competency, objective building science, and the art of the profession will no longer involve arbitrary design moves but rather become the result of providing superb knowledge and judgment in the interest of owners.[5]  Architects who can quickly adjust to and embrace the new regime could be very successful, while those who don’t could see their markets diminished.

One of the big changes that architects will need to incorporate is increased reliance on competently-produced complete and correct specifications.  As specifiers have developed deeper understanding of building materials and systems, they are better positioned to verify architects’ designs achieve the performance criteria required by clients and help architects fulfill their fiduciary duty.  Product manufacturers will also have an elevated duty to provide information that specifiers can rely on.

This is obviously a complex topic and I fully admit that it has taken me a lot of time and numerous discussions to wrap my mind around.  It will have wide-ranging impact on the design professions, whether or not the design professionals are ready for the changes this will cause to their practices.  It will be fascinating to see what comes of it and how the profession reacts.  I plan to offer further thoughts on this as it develops.

[1] Vyas, Ujjval K. “From 19th Century Gentlemen to 21st Century professionals:  Fiduciary Duties and the New Reality of A/E Practice” Presented to CSI Chicago Chapter January 26, 2016
[2] Adapted from Vyas, Ujjval K., “Matching Owner and Architect Expectations: Green Advocacy and the Necessity for Informed Consent” in Green Building and the Construction Lawyer: A Practical Guide to Transactional and Litigation Issues, ABA Forum on Construction Law, 2014, p. 126.
[3] Sitkoff, Robert H. “An Economic Theory of Fiduciary Law.” Philosophical Foundations of Fiduciary Law Edited by Andrew Gold and Paul Miller, 2014, pp 198-199
[4] Beutler, Melissa A. and Vyas, Ujjval K., “When Green Turns Mean!” Paper given at ABA Forum on Construction Law 2016 Midwinter Meeting
[5] ibid.

Let’s Fix Construction

cquwwkuukaeptkh

I was recently asked to participate in a new initiative called Let’s Fix Construction that is being established by some of my colleagues in the Construction Specifications Institute: Eric Lussier who is president of CSI Vermont Chapter and Cherise Lakeside, the #CSIKraken and past president of CSI Portland Chapter.  The purpose of the initiative is to separate the complaints from the solutions when we discuss why things go wrong in construction projects, and promote the solutions. Other participants so far are Randy Nishimura, Vivian Volz, Keith Robinson and Marvin Kemp, and the project is open to accepting additional contributors.

My hope, which is why I decided to participate myself, is that we can get past the usual CSI talking points, dig deeper, take a hard look at what is really ailing the industry and come up with solutions and then integrate those solutions into the various professions and trades within AEC.  By “usual CSI talking points” I mean the heavy advocacy for certifications, encouragement of improved communications, promotion of the ‘trusted advisor’ appellation, etcetera.  None of these are wrong, but to be frank, they haven’t fixed construction yet.

The project is named “Let’s Fix Construction.”  I accept the premise that construction is broken (or at least not operating optimally).  Undoubtedly this is due to various factors, but I’d argue a good place to look for them is at the real weaknesses in architectural education and in the way that architecture is practiced.

What follows is an edited excerpt from my first post on Let’s Fix Construction, where I outline my view of where the practice of architecture is complicit in the brokenness of construction and is missing the mark on living up to its promise and to its own vision of itself.

*           *           *

The facility design and construction process (at least in the traditional design-bid-build or design-negotiate-build methods) is for the most part driven by the architect.  The architect is the one who is presented the project goals by the Owner and is tasked with generating the design and construction documents and then helping to facilitate its execution.  In this architect-centric view, the responsibility to faithfully and skillfully execute the work lies with the architect.  The architect comes up with the conceptual design and develops that design, adding more and more technical detail, coordinating the work of engineering and other consultants, incorporating information from myriad sources into one package and shepherding that package through procurement and entitlement, until the job can be built by a contractor. The architect maintains responsibility through construction, working to verify that the project is being built so that it conforms to the design.

As the center of all that activity, the architect is the source of (or at least contributing to) many problems that, if solved, would go a long way toward ‘fixing’ construction.  The words “many but not all” should of course be a given in front of each item below.

  • Architects don’t view their work as providing a professional service in which they have a duty to put their clients’ interests above all others and to making sure they communicate honestly with their clients and to obtaining informed consent for all important decisions.  Instead, architects see themselves as designing for themselves, or believe they’re working for the good of society, the environment, adopting “improving life” or other lofty goals that create real conflicts of interest that they don’t even recognize.  

(Before reading on, I strongly recommend reading Ujjval Vyas’s comment to my post on LetsFixConstuction, and also his lengthy comment to Randy Nishimura’s post, Revitalization + Reinvention. The conceptual underpinning of the duty I briefly mention above is his idea, and I will be writing more about that in the future).

  • A few architects even go so far as to think of themselves as more important than their own work.  As an example, during an interview I conducted with a prominent and outspoken Chicago architect, he mentioned that contractors respect him “because I know how to say ‘fuck'”.
  • Architects have inadequate practical knowledge of construction. They don’t understand how their designs and details get translated into physical components in a building and what it really takes for human beings to assemble what they’ve designed.
  • Architects, and humans in general, to be fair, have a overly optimistic view of their own knowledge and competence.  This is known as the Dunning-Kruger Effect.  As a result, they forcefully promote making decisions based on faulty information that they have high levels of confidence in.  Most “green” advocacy falls into this category.
  • Architects are slow to recognize and adapt to changes in construction technology, and end up lazily copying solutions from project to project long after they’re obsolete.
  • Architects are hesitant to participate in the code-writing process, even though the content of the codes and the way they’re developed directly impact their work.
  • Architects have very poor knowledge of how much construction actually costs, and use loose rules of thumb to try to determine whether or not their designs are within clients’ budgets.  They rarely know how the details they create affect the project cost, and the resulting necessary VE costs them time, money and prestige.  

The idea that we’re going to fix construction means that these and other problems should be identified, given serious thought individually and collectively.  I look forward to working to affect the changes that the industry needs, and I hope the Let’s Fix Construction project can be on the vanguard of inventing and implementing those solutions.  If you think you have an interesting perspective on this problem and would like to participate in this project, it is still seeking additional contributors.  Contact Eric at letsfixconstruction@gmail.com for more information.

Are Materials Mindful?

Global climate is changing and humans’ burning fossil fuels is a significant contributor to the change.  We burn fossil fuels which adds carbon dioxide to the atmosphere; the additional CO2 increases our atmosphere’s ability to trap heat from the sun.  Both the construction and operation of buildings are huge consumers of fossil fuels and contributors of atmospheric CO2.  The construction industry has recognized its responsibility to the climate by developing methods to construct buildings in a manner that is less environmentally impactful and also to make our buildings consume less fossil fuels.  

There is some encouraging progress; the US Department of Energy recently published its 2012 Commercial Buildings Energy Consumption Survey: Energy Usage Summary which indicates that the rate of increase in energy use is slower than the rate of increase in total building square footage.  It’s a step in the right direction, but what we really need is to decrease how much energy we use and how much CO2 we release.  Architects are in a tough spot in light of this fact, and are under significant pressure individually and as a profession to reduce their projects’ climate impact.  Numerous organizations and initiatives, such as USGBC (LEED), Green Globes, Living Building Challenge, and AIA 2030 have been developed to help architects and owners reduce their projects’ carbon footprints.

Unfortunately for architects, they generally have only minor influence over energy and carbon in their projects.  Some of the biggest contributing variables are not in the architect’s control at all: Does the project need to exist at all? How big is it? Where is it located? How meticulously are energy-consuming systems maintained? How many hours a day will it operate?  Architects have some influence over other considerations pertaining to energy (design of the building envelope, for example), but most of the hard work of calculating and designing energy efficiency performance is done by engineering consultants.  That left architects, who feel the need to participate in changing the world, a bit out in left field.  In an effort to bring the entire building design and construction community into the “green” conversation, the definition of “what is green” has expanded well beyond carbon and energy and into site considerations, material sourcing, water efficiency, and indoor environmental quality.  These additional domains gave architects more areas to influence the decisions made on a project, and architects have actively participated in expanding their services to fill those arenas.

The problem is that for carbon and energy, results are comparatively easy to measure.  Basically, you count up how much fossil fuel and non-renewable electricity was used during a given time period operating the building.  Same thing goes for water.  But for indoor environmental quality, you quickly get into subjective value judgments over things like natural light and views, and into fear-mongering and questionable science over chemicals in building materials.

On April 6, AIA Chicago and USGBC Illinois co-presented a program called “Mindful Materials: Education & Advocacy.”  The program was marketed to ask participants to “join AIA Chicago and USGBC-Illinois … to explore how to distinguish greener and healthier materials faster to serve your projects better. The Chicago design community has come together to endorse a voluntary product labeling initiative, dubbed Mindful Materials, to facilitate the transition to transparency, sustainable and healthy product selections for your projects.”  

The mindful materials initiative is simply just that – a labeling program and spreadsheet that materials librarians can use to track which products have health product declarations or Declare labels or are Cradle to Cradle certified.  To talk up the value of the labeling program, the speakers acknowledged that architects and designers are not chemists and are not trained to understand the medical literature, but somehow still equipped to understand what’s good and what’s bad sufficiently well to tell their clients that they can make their buildings more healthy.

The science is particularly equivocal on the link between adverse health effects and most building products.  Even chemicals that carry significant health hazards may not be risky at all when properly managed to minimize exposure, used in small concentrations or used in combination with other products.  For architects to not only full-throatedly advocate for certain product attributes but also arbitrarily select products based on a ‘mindful’ label could be doing a serious disservice to their clients.  

Architects and designers can make themselves feel like they’re being “green” through the use of products based on product ingredient ‘transparency’ and claimed (but not substantiated) indoor environmental quality attributes, but it’s a wasteful distraction, like fiddling while Rome burns.  It may not feel as significant, but architects can make a far bigger “green” impact by understanding the energy codes, heat gain and loss, air, water and vapor movement, and how these things affect building envelope.  Then they can make informed design decisions accordingly.

ENERGY STAR for Commercial Buildings is Unreliable

 

mypgm3phMany people are familiar with the US Environmental Protection Agency (EPA)’s ENERGY STAR program as it pertains to consumer products.  The most ubiquitous place to see their logo is on home appliances.  The ENERGY STAR label on an appliance signifies that the appliance is supposedly more efficient than non-certified models.  There are questions about the reliability of the consumer ENERGY STAR program, but that is not a topic that will be explored here.

In addition to energy efficient products, the ENERGY STAR program offers advice to homeowners for ways they can improve the energy efficiency of their homes, from wall and window sealing to insulating to lighting, and more.  Their Home Advisor lets you create a profile of your home and get tailored recommendations for prioritized improvements.

Less well known, but growing in importance is the ENERGY STAR certified commercial buildings program.  On the program website, it’s described as follows:

ENERGY STAR certified buildings and plants meet strict energy performance standards set by EPA. They use less energy, are less expensive to operate, and cause fewer greenhouse gas emissions than their peers. Starting with the first ENERGY STAR certified building in 1999, tens of thousands of buildings and plants across America have already earned EPA’s ENERGY STAR for superior energy performance.

Currently, 21 types of facilities can earn the ENERGY STAR. Commercial buildings start by entering their utility bill data and building information into Portfolio Manager, EPA’s free online tool for measuring and tracking energy use, water use, and greenhouse gas emissions. Industrial plants start by entering key plant operating data into another set of free tools, called Energy Performance Indicators.

Specifically, to be eligible for ENERGY STAR certification, a building must earn an ENERGY STAR score of 75 or higher, indicating that it performs better than at least 75 percent of similar buildings nationwide.

Needless to say, scoring whole buildings this way is not so simple.  EPA scores different uses of buildings separately (schools are separate from malls are separate from hotels, for example), but what do they do about similar buildings in different climate zones?  How about retail stores open 24 hours as opposed to stores that close?  There are many variables in buildings beyond use group that affect energy usage.  Recent attempts to validate the claims the EPA makes, flat-out, that ENERGY STAR buildings use less energy, have shown the picture is not at all clear (at best) and are very possibly simply wrong.

Background

The EPA created the ENERGY STAR system for commercial buildings as a benchmarking tool to track and collect data on how buildings in the US were improving in efficiency.  They provided the Portfolio Manager tool that building owners can use to input data on their buildings.   The information collected includes location, use group, year constructed, technologies used and how much actual energy is either purchased or generated on site.  The EPA used this data to begin scoring buildings on its 1 to 100 scale in the late 1990s.  Since then, many municipalities and green building certification organizations have begun mandating that buildings be scored using EPA’s Portfolio Manager, and receiving ENERGY STAR certification is among the criteria to receive green building labels, which makes the reliability of the EPA’s scoring system critically important.

Current Research

In his paper, ENERGY STAR Building Benchmarking Scores: Good Idea, Bad Science published in the 2014 Summer Proceedings of the  American Council for an Energy-Efficient Economy John Scofield, Ph.D. scrutinized the methodology used by the EPA to assign ENERGY STAR scores, and found that it contains “serious flaws that lead to erroneous results.” The flaws in one model are so severe that Scofield demonstrated that random numbers produced a model just as convincing as the EPA’s model.

The main problem is that there are simply too few actual buildings entered in the comparison database for each building model (use group) and too many independent variables (location, occupancy, size, equipment, etc), and that in some instances a few hundred actual buildings are being asked to represent hundreds of thousands.  The database is not actually populated by information entered in the Portfolio Manager; instead it’s collected in a periodic survey by the Energy Information Administration.  When you look for buildings that closely resemble the target in terms of location and other variables, the number may be only in the teens.  To combat this, the EPA developed statistical models using predicted energy use.  These models are meant to represent a larger cohort of buildings, and new projects are compared against the model, rather than against actual buildings.

The statistical tools used by the EPA and Dr. Scofield are fairly advanced for a lay person – A detailed explanation of the EPA’s methodology and statistics is included in Scofield’s paper; a more lay-accessible description can be found in this video presentation.

Statistical models are imperfect and sometimes the independent variables make more noise than give good results.  In order to test how well the statistical model works, statisticians use the R2 measure, also called ‘goodness of fit’ to determine how well data fit a given statistical model. An R2 of 1 indicates that the regression line perfectly fits the data, while an R2 of 0 indicates that the line does not fit the data at all. A low value could result because you are trying to fit a line into a curve or you are dealing with known variables that do not truly predict the expected value (random or barely correlated to the dependent variable).  So the lower the R2, the higher the uncertainty of the predicted results is.  Another reason why this number can be small is that you do not have enough experience (observed instances) to give you credibility in your model.

Scofield’s paper finds R2 to be as low as 0.33 for some of the models, so the ENERGY STAR scores are extremely uncertain and offer minimal improvement over random guesses.

Conclusion

Making recommendations for improving the statistical tools used to score buildings is beyond my math knowledge.  I had to survey two friends (one a math teacher and the other an actuary) to get to the point where I barely understood the statistics tools used in the scoring method.  As for recommendations, Scofield says, “I would encourage people to continue scoring their buildings with the EPA’s Portfolio Manager simply as a useful way to track the variation of their own usage over time.  Usually these variations are tied to variations in energy use not variations in building operating parameters — and these are not subject to EPA errors.  As a tool for comparing your building to others in the stock it is less useful — and that is where it is subject to the errors and uncertainties in the EPA model predictions.  I am not optimistic that these will get better anytime soon.”

Until the models and tools are vastly improved, “ENERGY STAR Certified” should be considered to be a marketing gimmick and not a true measure of efficiency.