Consumerization of APIs: Scaling integrations

Consumerization of APIs: Scaling integrations
A new generation of tools based on RESTful APIs will help enterprise IT embrace the opportunities and challenges from social, mobile, analytics, and cloud computing (SMAC) and consumerization of IT (CoIT).
By Carol Hildebrand, Patrick Shankland, and Vinod Baya

The confluence of social networking, mobile computing, analytics, and cloud computing (SMAC) creates a perfect storm of challenges for enterprise IT groups. Generational changes in the way people adopt and use technology further exacerbate this disruption. A parallel subtrend called the consumerization of IT (CoIT) is driving the convergence of these technologies in ways that contradict the traditional IT management methods of command and control, locked-down systems, systematic rollouts, and long cycle-time application development.

Over time, SMAC and CoIT will expand and deepen the digital operating model explored in the previous article, “Exploiting the growing value from information,” on page 06. But these two trends present challenges that will require significant changes in how enterprise IT operates and enables business, which are issues examined in the next article, “Embracing open IT,” on page 54. Fortunately, evolving approaches supported by emerging technologies can help achieve this transition to a permeable enterprise.

These emerging technologies represent a new generation of application programming interfaces (APIs), or code-based programming specifications that allow software components to communicate with each other in a distributed system. A relatively new style of APIs, called representational state transfer (RESTful) APIs, and emerging solutions to manage them, called API management platforms, provide mechanisms so enterprises can engage with SMAC in a strategic and systematic manner to assist the fulfillment of their digital ambitions. Together, they power the consumerization of APIs.

Previous issues of the Technology Forecast have covered the individual impact of social,1 mobile,2 analytics,3 and cloud.4 RESTful APIs and API management platforms are the key technologies that abstract above these individual pieces and support the integration of SMAC, thus making them the key tools for participating in the digital economy.

RESTful APIs enable what PwC calls the permeable enterprise, in which capabilities and assets inside the enterprise are easily combined with assets and capabilities outside the enterprise. Once the sole province of highly experienced software developers with deep knowledge of the enterprise context, APIs are becoming the basis for creating digital value chains that access and act on information from traditional data stores, humans, and an increasing number of physical objects that contain digital content; in short, we’re witnessing the consumerization of APIs.

The rise of RESTful APIs

APIs have been used as a mechanism for linking programs since the early days of software. However, API creation and design have significantly changed. Early methods were proprietary and created interdependent coupling among pieces of code and systems. If one side of the coupling required a code change, the other side was affected. Over time, APIs evolved to reduce the interdependency of tightly coupled interfaces, generally lowering the complexity of integration. (See Figure 1.) Table 1 describes the evolution of integration and interface technologies to create distributed systems.

Figure 1
A conceptual view of the relative trends in complexity of integration and the nature of coupling, in the evolution of software integration technologies over time

Figure 1: A conceptual view of the relative trends in complexity of integration and the nature of coupling, in the evolution of software integration technologies over time

Within the last decade, enterprises began to expose APIs to allow external parties to build new functionality, something software companies did in the past. ProgrammableWeb maintains a catalog of these publicly facing APIs. (See Figure 2.) Its directory has topped 6,000 APIs, and RESTful interfaces far outpace other styles, such as Simple Object Access Protocol (SOAP).5 (See Figure 3.) Today, communication on the web has evolved from the early days of using SOAP standards to using features of RESTful methods, making REST-compliant APIs a major class of web services.

Figure 2
Adoption of externally facing APIs is accelerating

Figure 2: Adoption of externally facing APIs is accelerating

Figure 3
RESTful APIs have been a driver of growth in API adoption

Figure 3: RESTful APIs have been a driver of growth in API adoption
 

Service-oriented architecture (SOA), which gained wide acceptance using web services built on SOAP, has been popular within organizations as a mechanism for sharing information across the enterprise. However, the use of a REST architecture, along with associated technologies such as JavaScript Object Notation (JSON), is accelerating the development and use of APIs. Some of the most popular services such as Twitter, Netflix, and Facebook are now processing API calls on the order of billions per day or month.

“When we step back to see what the big revolution is, we see that APIs are the first serious digital indirect channel.”
—Sam Ramji, Apigee

APIs are becoming the basis for creating digital value chains, including digital indirect channels previously established mainly by software companies and purely web-based companies. “When we step back to see what the big revolution is, we see that APIs are the first serious digital indirect channel,” says Sam Ramji, vice president of strategy at Apigee, an API management vendor. “Although physical indirect channels have supported businesses for a long time, the equivalent in the digital domain has been unclear so far.”

“ In the past, everything was a heavy integration. This meant that if there was a seemingly small need, the cost of integration was a barrier to fulfilling that need.”
—Devon Biondi, Mashery

One big reason behind the successful adoption of RESTful APIs is developers’ ability to build modular capabilities with lightweight interfaces that don’t require heavy integration. “RESTful interfaces create a level of simplicity that didn’t exist previously, and simplicity always speeds things up, making integrations cost-effective,” says John Musser, founder of ProgrammableWeb.

The potential to lower cost expands the opportunities to integrate. “In the past, everything was a heavy integration. This meant that if there was a seemingly small need, the cost of integration was a barrier to fulfilling that need,” says Devon Biondi, vice president of strategy services at Mashery, an API management vendor.

“RESTful interfaces create a level of simplicity that didn’t exist previously, and simplicity always speeds things up, making integrations cost-effective.”
—John Musser, ProgrammableWeb

Despite rising adoption, integration via RESTful APIs is best regarded as a tool in the integration toolkit, and is not the right solution in all use cases. Each of the integration approaches in Table 1 have their advantages and disadvantages. For example, a REST architecture has added benefits such as being highly scalable and undeniably simpler, but depending on the protocols, clients, or servers used, it may have performance inferior to other integration styles, such as CORBA. Therefore, a cost-benefit analysis of specific IT use cases will determine the optimal integration style that must be implemented.

Table 1
The evolution of integration and interface technologies in distributed systems

Table 1: The evolution of integration and interface technologies in distributed systems

More about REST

As noted earlier, REST stands for representational state transfer, meaning that a client communicates with a server—not directly with the source of information on that server. This transfer is done through representations of the state of that resource. The REST architectural style for distributed systems was developed around 2000, and it is patterned after the HTTP6/1.1 Protocol. In RESTful designs, the client does not need to know about the implementation on the server. The server is free to store data as it likes, and the client can store the same information differently. This loose coupling means that as long as the interface is stable, the implementation on the client or the server can independently change. This independence creates flexibility in distributed software systems.

The REST architectural style requires that the following six constraints be met:

  • Client/server loose coupling—A clean separation of duty exists between client and server. The type of data storage does not matter to the client, and the client interface or client state does not matter to the server. With a stable interface, the client and server may be developed and replaced independently of each other.
  • Stateless—The interface that dictates how the client and server interact does not allow client states to be stored on the server. Information about client states is embedded in the messages the clients send to servers.
  • Cacheable—Clients can have the ability (and must let the server know whether they do or not) to temporarily store data received from the server.
  • Layering—Servers do not know whether there are layers of abstraction between themselves and the end client; for example, whether they are passed through multiple security policies, APIs, and so forth.
  • Code on demand—Servers are able to temporarily send custom functions as executable code to clients for them to execute.
  • Uniform interface—Servers and clients can interact, change, and be modified independently as long as the interface that binds them remains the same.

An API implemented using the preceding principles of REST and using standard HTTP communications protocols is a RESTful API, sometimes called a RESTful web service. Like any resource on the web, the RESTful API will need a Uniform Resource Identifier (URI), such as an http:// address.

RESTful API developers are increasingly using REST in conjunction with JSON, a resource-based data transfer mechanism, to further simplify the process of communication between the information seeker (the client), and the system containing the information to be consumed (the server). JSON is derived from the JavaScript scripting language, which is widely used in web browsers to enhance interfaces and build dynamic websites. Like REST, JSON is noted for its simplicity and usability. For instance, data is sent in plain text, which makes it easy to read and understand. Because so many web client programs are written in JavaScript, JSON data arrives ready to be consumed without needing further manipulation. At the same time, JSON lacks display capabilities and has a limited amount of development tool support. Hence, the decision to use JSON in an organization should depend on its intended use.

The early use of RESTful APIs has been concentrated in web companies, in companies active in mobile, in social applications, and in large enterprises leading the adoption of SMAC technologies. However, PwC expects this concentration to change; eventually all large enterprises with considerable data and information assets will use this technology internally and externally. Although SMAC applications motivate RESTful integration today, it is typically useful for any application-to-application integration.

Comparing REST and SOAP

For many enterprises, the path to web services begins with the adoption of a service-oriented architecture (SOA), which uses SOAP as a method for exchanging information. In today’s web service world, both SOAP and REST are used as methods of communication. There are several factors behind the popularity of REST when contrasted with SOAP.

Simplicity. REST uses simple HTTP and therefore standard commands—such as GET, PUT, POST, and DELETE—to coordinate communication between clients and servers. SOAP has no widely accepted methods corresponding to GET, PUT, POST, and DELETE, which leaves developers free to define their own semantics. But the result can be complex, proprietary mechanisms to connect components. Any programmer unfamiliar with these proprietary semantics will need time to learn them, slowing development. (See Figure 4.)

Figure 4
Contrasting SOAP and RESTful mechanisms for web services communication

Figure 4: Contrasting SOAP and RESTful mechanisms for web services communication

Familiarity. Since REST is closely related to web design, web developers do not face a steep learning curve. REST is also language and platform agnostic. On the other hand, SOAP requires a significant skill set in SOA-specific development and delivery tools. This impact can be substantial, as Ramji of Apigee explains: “When you get an order of magnitude shift in the number of developers who have the skill and talent to be able to use something, then you end up with outsize network effects.”

“When you get an order of magnitude shift in the number of developers who have the skill and talent to be able to use something, then you end up with outsize network effects.”
—Sam Ramji, Apigee
 

Efficient with bandwidth. The use of the existing web infrastructure eliminates the need for an additional messaging layer in RESTful APIs. Coupled with the fact that REST uses those short request and response sequences, these APIs consume considerably less bandwidth than SOAP-based APIs.

Scalability. With simpler component implementations and reduced complexity in the connection semantics, RESTful services can scale—as evident from several services that register more than 1 billion API calls each month.Although REST has many benefits, in some situations—such as stateful operations where state needs to be continued, or for applications expecting a guaranteed level of security or reliability, or requiring formal contracts with rigid specification of the interaction—SOAP would be a more appropriate choice over REST, as it has standards to ensure certain requirements are met.

Managing the APIs

As simple as RESTful APIs are to build, they still require management and maintenance. And although it is easy to manage a single API, all enterprises will inevitably have several APIs that internal developers, strategic partners, and public developers use to build applications across diverse platforms. This increase in APIs has led to the need for tools and services that help companies create, publish, manage, operate, and analyze APIs. Such solutions, often called API management platforms, are now offered by several vendors, including 3Scale, Apigee, Layer 7 Technologies, Mashery, and others. Alcatel-Lucent also provides API management solutions but with a focus on the telecommunications vertical, enabling service providers to make their networks addressable by others.

API management platforms bring a wealth of new functionality to API publishing and operations. The following paragraphs describe their key characteristics.

Developer community engagement. Engaging with the developer community is a significant first step to building an API ecosystem; internal and external developers must be aware of exposed services and the APIs to use them. Some API management platforms are free tools for those just getting started, while others are complete developer portals that support easy access to APIs and accompanying documentation, such as wikis, blogs, and discussion forums, to promote developer collab-oration. Some companies also offer outreach and marketing programs designed to make the development community aware of their APIs.

Security. API management platforms offer several facets of security, starting with new developer approvals and developer authorization to access specific APIs. Many companies support the open security standards or OAuth 2.0 for authentication and key management. More sophisticated API management platforms support multi-tier access control methods that specify which developers can access which APIs. Usage monitoring is also part of this multi-tier access control capability. API management platforms should be able to integrate with existing enterprise security systems and protect against external security threats by using encryption, threat detection, and analysis.

Analytics and reporting. Analytics help companies understand and improve the value of their APIs. Value-driven analytics gauge API adoption by measuring traffic, purchases, and registrations. Partner-focused analytics help companies better understand who is using their APIs and via which partner channel they are accessing it, informing an appropriate response. For example, companies might segment audiences by top developers and applications, or they might analyze usage by API method. Operational analytics and reporting tools give visibility into the API platform to improve efficiency. Customers can use such tools to troubleshoot problems, improve or maintain service quality, measure latency, analyze load statistics, monitor transaction data and traffic flow, or identify underlying API problems.

Traffic control. For companies that maintain their API platform in house, traffic control is important to protect back-end systems from overload. API traffic control tools can set platform-wide rate limits, set limits by other rules such as client or IP [Internet Protocol] address, or create tiered systems to allow priority customers to consume more API data. With API traffic control tools, companies can define and enforce levels of partner access to data consumption.

Performance and scalability. API management platforms address performance and scalability issues associated with API platform growth. They also can manage scalability issues and limit lag time and latency when a large number of developers, applications, devices, or end users concurrently use the APIs.

API optimization. Although the use of RESTful APIs is on the rise, sometimes companies must support other methods. Companies can help drive API adoption by offering APIs that use protocols the partner or developer prefers, and API management platforms provide protocol translators to streamline the process. They can optimize API content and format to fit the requirements of specific mobile platforms or devices, and then institute and enforce API version control.

Platform outsourcing. Companies can offload the entire API management task to external service providers via cloud-based services or global API networks. They can also tap these solution providers to take over API management when traffic must remain on the premises.

Table 2 shows the broader group of capabilities and how they are evolving.

Table 2
Capabilities of API management platforms

Table 2: Capabilities of API management platforms

Life cycle of API use

As discussed in the article, “Exploiting the growing value from information,” on page 06, the adoption of RESTful APIs creates significant new opportunities for enterprises to transform internally to a digital operating model and to engage externally with the evolving digital ecosystems. Strategically, any API program will start by analyzing an enterprise’s opportunities to take advantage of data and information assets, turning them into platforms, and facilitating their extension by internal or external developers. Once under way, the program must recognize that the use of APIs is like all other programs in that they will have a life cycle of their own. This API life cycle must be understood and managed, so ongoing changes and refinements can be made.

A typical life cycle of an API program moves from driving developer adoption to using APIs to expand channels, as illustrated in Figure 5. API management platforms provide support along the entire life cycle. One best practice is to treat APIs like products and staff them with the roles and infrastructure typically put in place for any other product. “It is important to keep the API stable, so it should be managed like a product that would be maintained, supported, and evolved with good change management practices,” says Mark Noworolski, CTO of Streetline, a global provider of smart parking solutions.

Figure 5
Life cycle for adopting and benefiting from APIs

Figure 5: Life cycle for adopting and benefiting from APIs

API opportunity in digital transformation

Thus far, this article has discussed APIs as a mechanism to link capabilities provided in distributed software components to RESTful web services in traditional computing environments. However, the use of APIs extends beyond that. “The homogeneous world of PCs, Macs, or web browsers is transitioning to a heterogeneous world where almost anything can have an IP address and be open to collaboration,” says Ramji of Apigee. Coordination across such a diverse and pervasive system requires an easy mechanism to connect and communicate. “That is why APIs are important. They provide such a mechanism in a digital ecosystem,” Ramji continues.

Ramji is referring to what is called the Internet of Things (IoT), a term first used by Kevin Ashton in 1999, referring to the growing tendency to build inanimate objects that have the capability to generate digital information without human intermediation.

IoT is seeding an analog-to-digital transformation of everyday activities by making a greater portion of a person’s environment more digital. Today it is becoming possible to capture information from data points as diverse as a toothbrush, home appliances, a thermostat, a car, or a home. Every object can be equipped with sensors that digitize the information about the object’s state, location, and other relevant data.

APIs then make it possible to access and monitor that data in a standard way. Companies such as Cosm and ThingSpeak provide open source platforms for real-time access, for the sharing and manipulation of information, and for the development of applications that take advantage of IoT. Typical applications automate the everyday environment, such as turning off the HVAC system or lights when a person leaves the home. Applications are also used in enterprise contexts. For example, FedEx SenseAware collects a wide range of information as a shipment moves from its origin to delivery.

While IoT may focus on inanimate objects, the growing use of smartphones and the role they can play in collecting data from sensors creates opportunities to further digitize human actions without active attention from the individual. (See Table 3.) For example, UP from Jawbone uses an iPhone app and a sensor embedded in a wristband to track meals, physical activity, and sleep patterns to later display it to the user to promote healthy habits. Smartphones today have sensors for location, acceleration, orientation, and so on. Many services that collect, link, and analyze this new data are emerging; Table 3 lists a sample of them.

Table 3
A sampling of services that illustrate the use of sensors and smartphones to digitize activities

Table 3: A sampling of services that illustrate the use of sensors and smartphones to digitize activities

Conclusion

The confluence of social, mobile, analytics, and cloud (SMAC) technologies is disrupting the business environment to an extent not seen since the early days of the Internet. SMAC represents a pace and degree of change that is overwhelming to the current IT approach of owning the end-to-end experience. Today’s methods do not scale to address the challenges that SMAC presents: too many variations in use cases and too many endpoints. In addition to understanding and having strategies for these individual SMAC technologies, enterprises should look at abstractions that bring together these technologies where they intersect.

The confluence of social, mobile, analytics, and cloud technologies is disrupting the business environment to an extent not seen since the early days of the Internet.
 

RESTful APIs and API management platforms are just such an abstraction and should be added to the existing integration toolkit. The best way to manage the upcoming challenges stemming from the confluence of SMAC is to use emerging IT tools around RESTful APIs. Combining these tools presents a model where central IT provides a platform for participation with the digital ecosystem. “In the future, the way to increase the barrier to competition will be to lower the barrier to participation,” suggests Biondi of Mashery.

The collective impact of these tech-nologies makes the pursuit of a permeable enterprise an imperative. With the rise of RESTful APIs and API management platforms comes the opportunity for enterprises to extend and deepen their digital footprint by making it possible to easily access and combine data from many more sources than previously possible.


1 “Transforming collaboration with social tools,” PwC Technology Forecast 2011, Issue 3.

2 “Unleashing enterprise mobility,” PwC Technology Forecast 2011, Issue 1.

3 “The third wave of customer analytics,” PwC Technology Forecast 2012, Issue 1.

4 “Driving growth with cloud computing,” PwC Technology Forecast 2010, Issue 4.

5 According to Wikipedia (http://en.wikipedia.org/wiki/SOAP): “SOAP is a protocol specification for exchanging structured information in the implementation of Web Services in computer networks.”

6 HTTP is Hypertext Transport Protocol, the protocol used for communications on the World Wide Web.