Evergreening IT at Bechtel

Photo: Geir Ramleth of BechtelGeir Ramleth of Bechtel describes how he used cloud computing principles to transform IT and make Bechtel’s computing environment more agile.

Interview conducted by Vinod Baya, Bo Parker, and Gregg Agens

Geir Ramleth is senior vice president and CIO of Bechtel Group, where he leads the global Information Systems and Technology organization. Prior to rejoining Bechtel in January 2002, Ramleth held leadership positions for a variety of companies in the technology and communications industry. Among his responsibilities in those roles, he defined strategy and market position and secured multiple forms of financial resources.

Previously, Ramleth served as chairman and CEO of DigiPlex S.A., based in Zurich, Switzerland, which owned and operated large communications and networking facilities throughout Europe. Prior to that, he served as president and CEO of Genuity, a Bechtel Enterprises company formed to deliver Internet and data services. Ramleth has held management positions with Oracle, PageMart, and PacTel Personal Communications.

In this interview, Ramleth shares his insights on how he is transforming IT at Bechtel to move away from managing, maintaining, and building legacy solutions to an environment that leverages virtualization, standardization, and automation to be flexible and deliver the right services anytime, anyplace, and to any device that has a browser and an Internet connection.

PwC: You have been leading a major transformation of the IT function at Bechtel.
How did you get over the hurdles common to such transformations?

GR: Many of the hurdles that people think are there are really perception hurdles. They’re not necessarily true hurdles. I believe that the technical side is only 20 percent of the problem. Change management is 80 percent of the challenge, and getting your own IT people to step outside their comfort boundaries is a key part of this. When you get them outside their comfort zones, you can get them to do things in a different way.

At one of our planning meetings, I laid out about a dozen and a half benchmarks from Internet services and consumer-based providers. I said, “If they [cloud service providers, Amazon.com, Google, salesforce.com] can achieve totally different operating models, think what we could do.” I went through all these different benchmarks and said, “You know, if people can deliver it for free, why does it cost so much when we do it?” After that first day, we concluded that we can’t directly use any of these 15 or 20 companies’ services. We would have to go out and learn how they [deliver at these price targets, at this speed, or with these delivery dynamics]. We decided that we needed to build it ourselves.

PwC: Such a transformation is quite ambitious. What made you think you would succeed?

GR: If your approach is to modify your current state, you will probably end up incrementally better. But if you go to the desired end state and move back only where it’s absolutely necessary, you will probably end up with a much more profound transformation. So we said, “Assume we have no heritage. Assume we’re starting from scratch. Assume that we actually are a startup that doesn’t have over a hundred years of experience and suboptimized IT legacy.” They started doing that, backing off only when needed, and that’s how we got to where we are now rather than just an incrementally modified state from the past.

We then carried out the plan by designing the desired end state and building it. We made no modifications to existing infrastructure—meaning we built new data centers, we built new networks, we built everything new. We used a rigorous application certification process, and nothing was allowed to come from the old state to the new one unless it passed that certification. Without it, you would just end up diluting a new set of services.

PwC: What was moved from the past state to the new state?

GR: No hardware was moved at all. The only thing that was moved was software. From a fundamental base level, all software was new. But when you look at specific applications, that can’t necessarily be the case. Now we’re moving SAP, for example. SAP has been certified, so we’re going to move it over.

PwC: What were the problems with the old state that you wanted to fix?

GR: We were not in pain with the old state, but we did see that we would need to take a new approach to have the flexibility for what we needed to do going forward. We had already completed a comprehensive rationalization process and brought about other
needed efficiencies. What we needed going forward was a global, flexible collaboration platform.

When you look at our business, our concerns incorporate far more than those of Bechtel itself. Our world is really much broader than that. We operate in more and more locations for shorter durations than we used to. We don’t have many large permanent offices that last 10 or 20 years—some offices might last only for a project’s duration or maybe even just a portion of a project’s life cycle.

And the people who participate with us are not all Bechtel employees. They are contractors; they come from the customer side; they come from the supply chain side; and sometimes they are our competitors.

“We had already completed a comprehensive rationalization process and brought about other needed efficiencies. What we needed going forward was a global, flexible collaboration platform.”

So if you take the geographical diversity and the project staff diversity as the real challenge, what’s your strategy for IT? How do you ensure intellectual property protection, for example? You want to open it up to be available to anybody anytime anywhere, but how do you then protect yourself? How do you move from everything sitting behind a firewall to doing everything outside the firewall?

We asked ourselves, “Now, wait a moment. Nobody sent you to school to learn how to buy a book from Amazon.com. Why should you have to learn how to do business with us? Why shouldn’t we be just as easy to do business with as when you do a Google search?” To get there, we found that you need to accept a new security paradigm. That’s really the biggest challenge on the technical side.

Milestones Phase I: Rationalization Phase II: Transformation
Motivation Address complexity and costs Pursue opportunity for flexibility and competitiveness
(time frame)
(up to 2005)
(2005 to 2007)
(2007 onward)
Model IT centric Company centric Collaboration centric Partner centric (ecosystem)
IT Project, services deployment time 60–90 days 30 days Overnight Policy-driven, real-time,
IT workforce 2,000 1,250 1,100 Balanced to needs
Data center size 20+ centers; 35,000 sq ft 7 centers; 20,000 sq ft 3 centers; 1,000 sq ft TBD
Applications More than 1,600, with several current versions of each 230, with an average of 4 to 5 versions of each 230, with 3.5 versions
of each
~200, ideally with only 1 version of each
Standardization None General standardization Strict standards for optimization (goal of one solution, one version) Strict guidelines with flexibility for rapid change (Evergreen IT)
Server utilization 2–3% 30–40% 60–70% On demand
Resource/services provisioning Manual Somewhat automated Policy-driven provisioning of access Policy-driven access to external IT resources
(move toward end-user self-service)

PwC: Can you provide a high-level overview of your journey?

GR: We see our current cycle, a decade of IT, in four stages. [See Table 1.] The first one was disruption. That was when the business said, “You know, now IT is messing with me again. IT costs too much.” It was when ERP [enterprise resource planning] was installed. The vendor said that the world is going to be so great, but all we had was pain, and it’s slow, and it doesn’t really change the business the way the vendor said it would. So that was the first phase, the disruption phase. It was very IT centric.

The second stage, alignment, was when we heard from the business, “Hey, at least they are working on what they should be doing, and we understand the cost structure.” So you’re aligned with the business and where it is going. During alignment, we went from 27 to 7 data centers, and we made the Internet our friend. But we were still very focused just on what was happening in the company, and the IT group was very focused on what they do from a technology standpoint. This stage was very company centric.

The third stage was transformation, motivated by our need to address the nature of our business, which I talked about earlier—how business opportunities result in frequent changes in geographies where we are present, and how the people we deal with shouldn’t have to learn how to do business with us. How do you transform yourself to just be in the Internet rather than attached to the Internet? This is our current stage and we define it as global-collaboration centric.

The last stage, which is in the future, is business expansion, where we become partner centric (with direct and indirect participants).

PwC: How has IT become more agile through this transition?

GR: In the disruption stage, when we wanted to fire up a new project, it took us about 60 to 90 days to get a project ready from an IT standpoint. In the alignment stage, we got that down to about 30 days, which is really good. Now that we’ve reached the transformation stage, we’re basically saying, “Why shouldn’t it be available overnight? Why shouldn’t it all just be open in the same way that Amazon.com is open?” We’ve come to realize that we needed to build a security environment driven by policies and not one that’s driven by topology.

So instead of saying, “I’m going to get you in here behind this firewall,” you’re saying, “No, I just want to know who you are and what you need, and then by policies I will create the rest for you.”

We think of this as our virtual foundation. You don’t have to have a Bechtel login. You use whatever your e-mail address is, just like when you go to any modern Internet site. As long as we know that and we can authenticate you with that, we can give you what you need.

“The important thing is that we standardized. Now, we very much live by the power of one. Do it one way, in one place, using one mechanism, using one approach, and all operated by one group.”

This is a huge improvement over how bad it was previously. At one time, when using our former authentication approach of assigning internal user IDs and such, one-third of the people on our core network were not Bechtel employees. We don’t have the same right to do background checks for people who come on our network from the outside. The partner or the customer can say, “Add these 200 people. They are now a part of our project.” We have to add them without the same rigor that we apply to our own users. We don’t know who they are other than a name. With our new model, it doesn’t matter. Our trust and security model is set by policy. When you seek more confidential information or try to access more secure IT elements, we put on stricter and stricter authentication, but it’s all done by policy.

PwC: Complexity is a big concern in many IT organizations. How did you deal with that in your transformation?

GR: We said, “Let us make this as simple as possible,” so we went for a single vendor. I say half-jokingly that our old data centers were like a hardware hall of fame. We had one of every vendor’s products. For example, now our computing and storage is from HP, and other solutions follow the same model. The important thing
is that we standardized. Now, we very much live by the power of one. Do it one way, in one place, using one mechanism, using one approach, and all operated by one group.

Initially, we even tried to run one data center in one location. That didn’t work. Our applications are not designed for a high degree of latency. So we designed three data centers: one in the UK, one in Singapore, and one in the US. But we said they would have the same hardware, the same setup, and they would be wired the same way. Then you can say, “The red wire that goes into that Cisco device should be in slot number 4.” We have only one operational group—so we essentially have one virtual data center.

Originally we thought we could manage infrastructure diversity in software. But we decided that we couldn’t do it. If we had tried, we would still be sitting here writing business cases for it, I’m convinced. The best, most direct way forward was to start solving the complexity problem from the infrastructure layer up, and when we have one of those applications that doesn’t port to our new environment, we envelope
that one and we find a new solution for that.

PwC: Did you need to make any changes in how you select and do IT projects?

GR: One thing that was fundamental in this change was to avoid designing solutions for the lowest common denominator. Rather than focusing designs on a functionality that has to work for all, we moved to designs that covered 95 percent of our business needs. We decided to design a base-level architecture for the masses and find a solution for the uniqueness.

Of course, we heard, “What about that project that sits in the Saudi Arabian desert? This will not work for them.” And we basically said, “OK, it will not, but let us plan for the other 95 percent of our business and get special solutions for the unique needs.” That was a huge fundamental change in how we developed our design criteria.

For example, to support unique circumstances, we created what we call a SNAP, which is a server network access point. It’s approximately a half rack, and it’s an absolutely consistent extension of our highly standardized data centers. That SNAP has all the networking gear that the project needs—wire, telephone, all that stuff. It can have 30 to 40 processors and terabytes of storage, and, most important, it has exactly the same kind of hardware that’s in the three data centers. It runs the same software. And the operations staff is one group that sits in two locations (US and India) to provide 24x7 service.

We just ship it to a project location and all it needs is input of power and some form of connectivity. We have a new project in a very remote location where the only connectivity we have is satellite. The operations can’t run on one of our three data centers, so we put a SNAP in there instead.

PwC: One of the problems with legacy solutions is that you have to keep running them even when usage is minimal. What’s your approach to solving that issue?

GR: It’s true. Today we run hundreds of servers in support of projects that are no longer active because we don’t know what to do with them. For example, we have projects that might be within the warranty period. Utilization is extremely low. We run it, we back it up, and we maintain it as if it is a real production system, because we haven’t had a reasonable alternative. Now we’re figuring out how we can put a “time capsule” around it.

“The best, most direct way forward was to start solving the complexity problem from the infrastructure layer up.”

PwC: What do you mean by time capsule?

GR: It’s an interesting concept that something like the Amazon EC2 service may solve. Let’s say you need a server and you configure it, and, after you’re finished running what you need, you shut it down. In the process of shutting it down, you have the option of creating a current image of the environment, which stores the entire state of your application. Then you can come back and say, “That server we ran a few months ago that had this stack, we called it ABC.” And you say, “Fire that up again,” and in a matter of a few minutes, that server is up and running exactly as it was when
you shut it down.

What you’ve done is started untangling the huge, complex array of legacy systems. You can take this down to the lowest level of where the software is and extract it, and plug it in again. It might not go to the same infrastructure next time at all, but it will be
up again exactly the same way as it was before.

PwC: And do you rely on vendors for that, or have you needed to build that yourself?

GR: Here we lucked out a little bit. We started working with Simon Crosby at XenSource to build the virtual server environment more than three years ago. We became one of their first enterprise customers. When Citrix bought XenSource, we took a proactive approach and said, “We have had this relationship with XenSource, and we’re interested in creating a much more strategic partnership with you as a company, because we have a vested interest.”

We told them where we really wanted to go—that we wanted to virtualize the server and the desktop—and we assumed that Citrix bought XenSource because they wanted to get to that state. So we built up a very strong relationship with Citrix that is just great at this point.

I believe we have come to a stage in enterprise IT where all the great young companies that were once legacy killers have become legacy companies themselves. In essence, they’ve all won that previous battle, but now they’re invested in holding enterprise IT to their own approaches. The result is a new legacy that is destroying all the flexibility these companies offered when they were the emergent providers. All that flexibility is gone.

PwC: Is there an approach that doesn’t recycle the old legacy for a new legacy? Something that might be called Evergreen IT, where you can have a future where your IT is legacy free. Do you think what you have done allows you to avoid recycling into a new legacy base?

GR: I think Evergreen IT is a good way of describing our ultimate goal.

PwC: Cloud providers such as Amazon.com appear to be much closer to this vision of Evergreen IT by investing in standardization and automation of IT operations, particularly the provisioning and management of infrastructure. How much scope is there to automate in enterprise IT?

GR: There’s a lot, because we still “tinker” too much. There is still too much knowledge in the heads of operations staff that needs to be formalized and standardized in software. I wish we could run to our internal servers the way you can with Amazon.com—where you can fire up machines in their data center. I don’t know where the servers actually sit, and, at one point, I will not care, but suddenly I have an environment running on my IP [Internet Protocol] space.

I’d like to take our peak loading requirements and offload that to somebody, like some of our time capsule stuff. You start to get into cloud bursting here. In the Evergreen IT sense, in the foreseeable future, you might have a little core IT capacity, and then you tap into external providers for whatever else you need beyond that.

In our business, it could be for a specific project, or it might be for some specific load—like one payroll process that bursts far above normal for a few hours twice a month. In an enterprise IT environment, you buy everything for peak loading. In 2002 we did an analysis of our servers parked around the world, and our average utilization was 2.3 percent. With our second-generation virtualization, we’re starting to get that up into 60 and 70 percent.

PwC: There are many concerns about risks of security in the cloud. How did you mitigate risk in your case?

GR: The point is perceived risk. I believe that the information in our systems is more secure today than it was in the typical legacy environment, because we have tightened up that back end tremendously. Think about it. How many times do you hear about people hacking into the back end of Amazon.com? It’s very secure. You don’t hack into the bottom layer of where Google is operating. It’s very secure.

Basically, you change just the demarcation of where you have flexibility and where you don’t have the flexibility. Behind the demarcation, you tighten up much more than you ever did before. Like I said, we had security policies before. It was just too difficult for anybody to find intruders when they came in. The sheer complexity of the legacy environment defeats all reasonable efforts to control intrusions.

In the old way, you had to make all these exceptions. You had to open up this for this purpose, that for that purpose, and then you would forget to close it again. We now operate in a much more standardized way, and in the process have become more secure than in the past.

PwC: How have you financed your transition?

GR: We decided that we were going to do the transition without asking the company for money. We realized that we needed a financial architecture just as we needed a technical architecture. So we planned our new activities as an integrated part of the existing operations and budgets, and, as a result, we had no overall cost increase. Over time, we strongly believe that this will give us more financial flexibility. It should increase our variable cost and decrease the fixed cost component that IT has tried to tackle for years. I firmly believe that one of my responsibilities is to build an agile operations model that includes financial flexibility, and such operations will have greater success in responding to business needs.

PwC: What were some of the changes that you had to make to your IT organization?

GR: We began with an IT organization that was extremely distributed, both from a technical and operational standpoint. Then we started finding ways
to leverage off each other to deliver services. We called these global leveraged services. So you have a global leveraged service such as data center operations, and one for networking. There are real advantages to this model beyond cost reduction.

We formerly had 33 help desks in the company and none of them had 24-hour service. We couldn’t afford it. But now, with one virtual help desk you can have 24-hour service. You get the back-end systems so that they all can work on a global fabric. They have only one help ticket system, and we have only one phone number, even though it moves with the clock.

PwC: Have you been successful in rationalizing your legacy portfolio of applications?

GR: During our first phase of rationalization, we took our application count down from about 1,600 to 230. We still maintain 800 applications, because for the 230, on average, there are three-and-a-half versions that still need support.

A key decision has been to develop multitenant versions of our core software. One key application is InfoWorks, and we formerly maintained 16 different versions of it. Very early on we created a new version of InfoWorks that is a multitenant environment. This allows us to run InfoWorks in a software-as-a-service model, where we are the service provider for any of the projects. Multitenancy was a requirement due to the nature of our business.

With most of these services, we’re finding that you can buy many of them and they serve our subcontractors very well. But commercial services can’t support us internally when it comes to our complexity. So the procurement system will be a modern multitenant system, and we are working on that today. We’re
not rewriting it, but we are modifying it to the new requirements. A full rewrite would be costly and unnecessary, because this large, well-designed suite of applications has 15 years of development in it.

PwC: But you’re still going to run it on the standardized infrastructure, right?

GR: Yes, and we will run it on one set of code instead of our current situation that requires multiple versions. We will reduce operating costs, and we will be able to deliver faster.

PwC: What would you say at the highest level is your vision for IT?

GR: Our vision is that we want anybody to be able to have access to the right resources at any place at any time with any device. And we want to provide this all in a secure and cost-effective manner—cost-effective meaning that we have the flexibility we need, not necessarily that we want to be lowest cost, but we want that flexibility.

I’m after the tens of thousands of end-user devices. Look at this scenario. You can say that your recycle rate for end-user devices is about three years. The cost over that period is roughly $10,000, of which your hardware upfront is about $1,500 to $2,000. The $3,000 a year is all the tinkering we have to do, often driven by changes from the vendor community.

When you decouple that, when you move away from enterprise-controlled and -supported end-user devices, we are not worried about where you come from, because we might not even give you the data to your machine any longer. That data might all sit in our environment, and, as I said earlier, our data centers today are more secure than they ever were before.

And then we can say, “Well, you know, this person will never be able to download anything.” It’s just a totally different end-user environment strategy and philosophy. You can come to the BYOC, bring your own computer, model where you just say, “Buy anything. I don’t care what you buy. As long as it can connect to the network, we can offer our services to it.”

PwC: In other words, from an Evergreen IT standpoint, we’re talking about complete decoupling so that you’re legacy free. Would that be a requirement?

GR: Oh, absolutely. You can’t get there without it. You must have that to be evergreen. If not, you can’t deliver on any information, any device, anyplace.

PwC: You mentioned four phases. You’ve talked about three: disruption, alignment, and transformation. What’s the fourth phase?

GR: We call it the expansion phase. It’s when IT adds profound value to the customers who buy our products and services. That is when somebody chooses to buy, say, an oil refinery from us over somebody else because we could deliver differentiated, value-add IT services. At that time, it’s mostly information, actually. For example, it might be when a customer starts operating faster, because the system is already in place by the time they take it over from us.

We did this for a project in China, and our key customer said they would never do it any other way after that. They shortened their startup time by several months and they reached higher capacity than they had planned for. In this stage, you have to take a life cycle view of everything, including infrastructure, applications, and information that is generated or used both inside and outside your organization. In addition, what you really need to do is virtualize your workspace, so that, as long as you’re connected to the Internet, you can get your work done. In addition to your basic computer environment, this will include all the other resources you are accustomed to having, including telephony, print services, and file sharing.

“Our vision is that we want anybody to be able to have access to the right resources at any place at any time with any device.”