Comet (programming)
From Wikipedia, the free encyclopedia
Comet is a programming technique that enables web servers to send data to the client without having any need for the client to request it. It allows creation of event-driven web applications which are hosted in the browser.
Contents |
[edit] Motivation
Traditionally, web pages have been delivered to the client only when the client requested it. For every client request, the browser initiates an HTTP connection to the web server, which then returns the data and the connection is closed. The drawback of this approach is that the page displayed is updated only when the user explicitly refreshes the page or moves to a new page. Since transferring entire pages takes a long time, refreshing pages introduces a long latency.
To solve this problem, Ajax can be used which allows the web browser to request only that part of the web page that has changed and update that portion accordingly. Since the overall data transferred is reduced, latency is also reduced, and overall responsiveness of the web site hosting the application increases. Further, by using asynchronous background data transfer, where the user works with partly received data as the rest of the data is being retrieved, the responsiveness of the web application can be further increased.
But this practice also suffers from the problem that the client has to request some data before it will be sent by the server. This problem becomes a major hurdle when designing applications which have to wait for some event to occur at the server side, such as some other user sending some data to the server, before it can proceed, but has no information when the event will occur.
A solution would be to design the application such that it will intermittently poll the server to find out if the event has occurred. But this is not an elegant solution as the application will waste a lot of time querying for the completion of the event, thereby directly impacting the responsiveness of the application. In addition, a lot of network bandwidth will be wasted.
A better solution would be for the server to send a message to the client when the event occurs, without the client having to ask for it. Such a client will not have to check with the server periodically; rather it can continue with other work and work on the data generated by the event when it has been pushed by the server. This is exactly what Comet sets out to achieve.
[edit] Technology
Unlike normal data transfer between web servers and browsers, connections to a server employing Comet require special constructs. The client-side application must keep working while asynchronously initiating a connection to the server. At the server, even though the request is not serviced at once, the connection remains unbroken (and unused) until the event occurs. When the event occurs, the server pushes the data generated by the event to the client over the already established connection.
The connection can be severed after that, or may be kept open if there can be further occurrences of the event. If there are more occurrences, the server can send the event data over the connection, without the client having to request each of them explicitly.
[edit] Scalability and Reliability
There are some potential concerns regarding how well a web server implementing Comet can scale. Because connections are kept alive until events have occurred, it has to cope with many connections if the event occurs infrequently. The problem worsens if different connections wait for different events. Managing this large number of connections poses a considerable load on the system. But servers and other supporting applications are being developed which have better support for such conditions.
The Ajax/Comet Request Router is one mechanism for providing scalable server-side infrastructure supporting low-latency asynchronous and bidirectional Comet applications. It is used as a front-end for arbitrary web- or application-servers, to distribute client requests to a matching server, which may be a specialized program, or a general-purpose server which executes application-specific code for corresponding requests. Java Jetty and Apache 2.2 provide intrinsic support for situations where they use asynchronous network operations to reduce the overhead of managing the connections. Thus the web server logic has to deal only with sending messages back to the user once the event has occurred. Since almost all Operating Systems have an event-driven I/O subsystem, this is not a problem.
Firewalls and HTTP proxies between the browser and web server can pose further network problems. Firewalls are typically configured to drop connections that have been open for too long (as a security heuristic), and so commercial-quality frameworks teardown and recreate the "push" connection periodically. Naive intervening HTTP proxies may buffer pushed messages in chunks up to 32 or 64kb, necessitating a system for adaptive detection of intervening proxy buffer sizes.
Browsers pose a further limitation: section 8.1.4 of the HTTP 1.1 spec states that "A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy", and this recommendation is in fact followed by the most common browsers, including Internet Explorer and Firefox. Holding one connection open for HTTP streaming tends to create performance issues for AJAX applications, such as inability to kick off a new user-initiated data request while a series of images are being loaded. This is often worked around by creating a distinct hostname for push connections (even with a single physical server).
In clustered server environments, messages need to be routed to whichever webserver a given client is currently connected to, and guaranteed delivery may need to be ensured in the event of a webserver crash. This is typically handled by Message Oriented Middleware systems, frequently encapsulated by JMS.
[edit] Implementations
There are several implementations of the Comet Ajax server-push model, both commercial and non-commercial:
- The SmartClient Messaging module (see AJAX IRC Chat example)
- Lightstreamer. Adaptive HTTP streaming with some AJAX APIs.
- Steve and Jay McDonald’s *Fjax
- Pjax (Push technology for Ajax)
- Server-Sent Events standards proposal specified by WHATWG
- COMETd Scalable Open Source Comet Framework
[edit] See also
[edit] References
- Virgil's One™ is a COMET implementation for streaming marketdata solutions for retail-, wholesale-, private banks and brokers fully integrated with Websphere Portal based on Virgil's Live™ product.
- KnowNow - pioneering implementation of Comet. Link goes to a description of LiveBrowser, a feature of the KnowNow LiveServer.
- Beyond AJAX: Accelerating Web Applications with Real-Time Event Notification, by Dr. Rohit Khare (pdf)
- Divmod Athena: Comet implementation in Python
- DojoToolkit - Recent re-implementation
- Changing the Web Paradigm: Streaming AJAX and Comet (pdf)
- Push Technology for RIA: Two ways of introducing iPush Server to a RIA project
[edit] External links
- Virgil's Live™ streams any data from any source to any web client and delivers real time data at low latency, high availability, through firewalls and proxy servers based on COMET.
- Caplin Liberator A proven, scalable, bi-directional highly tuned messaging server that uses comet with HTTP/S tunnelling to provide up to 30,000 concurrent connections with low-latency streaming - used by major financial institutions across the globe.
- Flex Enterprise Services An approach from Adobe/Macromedia for addressing the same problem using Flash.
- Lightstreamer A full Comet solution that comprises a high-performance Push Server and a set of client libraries. The event-based architecture of the Server makes it able to scale to tens of thousands of concurrent connections.
- ICE Technology Corp.'siPush Server - real-time push data to your RIA.
- Asteroid (a small Comet): Comet in Squeak(Smalltalk), by Diego Gomez Deck
- Fjax: an open, lightweight, cross-browser methodology for Ajax-style development
- COMETd: an open source scalable comet framework. A Dojo foundation sponsored project.
