Call it a Cache-22. Dynamic Web content is emerging as a key to
successful e-business sites. But that fluid information can lead to
heavy network traffic, slowing application servers to a crawl and,
in turn, bringing an e-business to its knees.
Enter ESI (Edge Side Includes), a new caching standard that
enables developers to craft applications that lower costs and
minimize traffic congestion and customer response time. Products and
services that support ESI are beginning to enter the market, and
some enterprises are trying them out.
The move to display content that can change by the minute stems
from the need to offer customers accurate information from product
catalogs, auctions and online exchanges, as well as the latest stock
quotes, news and weather. This ever-changing information is a far
cry from the static pages and banner advertisements typical of early
Because much of this real-time data is on central servers, the
pipes leading to those servers have become network traffic
bottlenecks. What's more, the servers are not the most cost-
effective place to do all the processing. "Companies would like to
offload processing chores from expensive central servers to
inexpensive commodity hardware, but that hasn't been possible with
dynamic content," said John Magee, senior director for Oracle9ias
product management at Oracle Corp., in Redwood Shores, Calif.
Offloading dynamic content can be tricky because it's hard to
know when some information will be updated or which user will
request it. However, the vendors sponsoring ESI recognized that
while a larger percentage of information was dynamic, there still
were many similarities in the pages shipped to users. Therefore,
they wanted to create an algorithm that would not ship every bit of
information but would cache pieces that were repetitive to reduce
network traffic. They also needed to divvy up Web page generation
chores and then develop a way to deliver rapidly changing content to
users based on different criteria such as the time of day.
Designed to function like HTML, ESI differentiates information
that only a central server can generate from information that can be
stored on other systems, which can be stationed in a company's data
center or along the network edge as part of a private network or CDN
(content delivery network). ESI then provides programming calls so
the distributed items, called content fragments, can be aggregated,
assembled and delivered as complete Web pages to end users.
The standard enables a developer to create dynamic content via
templates that break down Web pages into HTML fragments, basically
subsections of a page. Each template features common elements, such
as logos or navigation bars, and is associated with a URL that an
end user requests. The fragments are markup files containing
discrete text or other objects, and each has a particular profile.
For example, a company may want to store a template for several days
but copy a particular item (say, a stock quote) every 15 minutes. In
addition, fragments, such as a user's bank account number, may be
declared too sensitive to store locally and be kept only in a
In addition to reducing hardware costs, spreading processing
chores among a series of servers can improve the scalability,
reliability and fault tolerance of an IT infrastructure. What's
more, the systems are more secure because an attacker has to shut
down several servers, rather than one, to knock a company's Web
systems offline, according to Peter Christy, founder of NetsEdge
Research Group, a Los Gatos, Calif., market research company.
A Standard Emerges
Development of the specification stemmed from research projects
at Akamai Technologies Inc., in Cambridge, Mass., and Oracle. Both
companies were examining ways to improve delivery of dynamic
information, although they had different starting points.
Akamai wanted to make its EdgeSuite CDN services more attractive,
and Oracle was trying to enhance Oracle 9ias, its Web application
server. "Since our customers work with a wide variety of products,
we didn't see much value in trying to develop a proprietary
technique for pushing dynamic content off of central servers," said
Thorsten Ganz, director of marketing at Akamai.
Unlike many standards efforts, this one emerged without a fight.
"I don't know if the slowing economy played a role, but whatever the
reason, no vendor thought it was worthwhile to develop an
alternative to ESI," said NetsEdge's Christy.
After deciding to work together, Akamai and Oracle solicited help
from other vendors, resulting in a group of 15 crafting the first
version of the specification. The participants represent a variety
of products: Web application servers from BEA Systems Inc., in San
Francisco, and IBM, in Armonk, N.Y.; CDN services from Digital
Island Inc., in San Francisco, and Mirror Image Inc., in Woburn,
Mass.; development tools from Macromedia Inc., in San Francisco, and
SilverStream Software Inc., of Billerica, Mass.; and applications
from Art Technology Group Inc., of Cambridge, and Open Market Inc.,
of Burlington, Mass.
In June, the group completed the first release and then persuaded
the World Wide Web Consortium to take over responsibility. In
September, the W3C passed ESI as a note, its version of a standard.
In addition, the supporters developed ESI for Java, extensions to
the Java programming language that enable JSPs (JavaServer Pages) to
call ESI components. JSPs are server-side software modules that
produce a final user interface by linking dynamic content with
static HTML through tags.
With that foundation laid, the issue then became persuading
vendors to incorporate ESI support into products. "Adding new
features to current products usually takes six to 12 months, so I
expect the number of vendors delivering ESI-compliant products to
increase from the summer through the end of the year," said Neal
Goldman, an analyst at The Yankee Group, a Boston market research
company. Not surprisingly, Akamai and Oracle are leading the pack in
that effort. In December, Akamai added ESI to its EdgeSuite product
line, which pushes dynamic content to the network edge, and in
January, Oracle included support for the interface in
ESI and the Times
The Seattle Times new media department turned to ESI after having
trouble processing Web page requests when noteworthy news events
occurred, such as the World Trade Organization meeting, an
earthquake or the Sept. 11 terrorist attacks. "We weren't sure how
many hits were occurring because many individuals logged off since
response time was so slow or they were denied access to a particular
page," said John Soltys, senior Internet engineer at the publishing
company, which has about a dozen Sun Microsystems Inc. servers
running its applications.
To solve the problem, the company deployed Akamai's EdgeSuite
services in February 2001 and added ESI functions in January 2002.
There were a few hiccups at first. "Initially, we were missing
information in the fragment headers, so it wasn't clear to the
different systems what items had to be pulled from our Sun servers
and what data was stored in the Akamai network," said Soltys,
adding, "We worked with Akamai to resolve that problem."
Akamai's implementation of ESI is different from that of Oracle.
While the Akamai implementation pushes content off the central
server and on to the network, Oracle's pushes the content from a
database server to an application server.
Macromedia's implementation, meanwhile, moves content to client
systems. "Akamai has focused on using ESI to push processing
functions onto its network; we think there are instances when
companies will want to move it back onto the client system," said
Kevin Lynch, chief software architect at Macromedia.
When more ESI-compliant products ship later this year, companies
should gain flexibility in designing applications as vendors
implement ESI in different ways.
Whatever the scheme, the goal remains the same: to get data off a
central server to lower costs and improve performance. That mission
is not a little ironic, some observers say. Application architects
expected many benefits from Web-browser-based applications, not the
least of which would be improved manageability, thanks to moving
processing from client systems to central servers.
Paul Korzeniowski, a freelance writer in Sudbury, Mass.,
specializes in networking issues. He can be contacted at