I'd like to hear your thoughts on Curl technologies.

[ O'Reilly editor Andy Oram responds. ]

Curl has been in development for seven years, and the Web is a very different place now from when Curl was conceived. For a long time I thought Curl solved only the old problems, not new ones. Now I think it could also be a solution to new problems, by exposing Web services to easy querying and automation and letting end users communicate directly. My recent contacts at Curl (Joseph Golden, senior director of business development and strategic relationships, and James Joly, product strategist in the Corporate Strategy group) tell me they're part of the new SOAP/P2P/Web Services world.

Exchanging performance for standards

Let me explain the old problems first. Back in 1995, M.I.T. researchers saw the direction the Web was going and realized that the code used to develop Web pages was an unholy mess. HTML was poorly structured. A hodgepodge of many technologies going under the term Dynamic HTML tried to add interactivity and make use of the client's computing power, but they suffered from poor standardization and uneven feature coverage. Communication with the server had to be effected through the rigid and limited CGI standard. Stunning effects could be achieved only through proprietary add-ons, notably Flash and audio/video streamers.

So in a DARPA-funded project, the researchers created something elegant, efficient, and comprehensive that brought together text, images, animation, and interactivity. This technology, which they called Curl, they spun off in 1998 into a start-up with auspicious backing from Berners-Lee himself. The current product is called Surge. It consists of an runtime environment and an IDE to facilitate the use of the Curl language; version 2.0 was released this year.

On this level, Curl was positioned as an alternative to Java, Macromedia Flash, and other popular tools for creating Web pages. The Curl evangelists had some compelling arguments for their product: they said it was easy to program, reduced both server load and bandwidth by putting processing on the client, and offered a security model at least as powerful as Java's. Like Java, Curl offers a sandbox, but builds in delegation. Instead of asking the end user (who knows nothing about the various Internet sites) whether he or she wants to download code from some cooperating site, the administrator makes the decision.

Curl's claim to have overcome the browser and platform incompatibilities that plague so many other Web tools is impressive; if it's true, Curl should be very appealing to sites that want dynamic content.

But all these arguments may fall on deaf ears. People may say, "I can find plenty of programmers to use other popular tools and technologies. You've solved a problem that existed six years ago, but we've found workarounds since then." In the next section, I'll tell you how the Curl evangelists answered this objection.

People are also very attached to standards (although Macromedia has overcome that preference). Curl recognizes this and provides a lot of open source tools for writing and deploying programs. But the engine or virtual machine remains proprietary; unlike Java, nobody is supposed to develop a Curl engine except Curl. (I don't know what the legal subtleties are, though.)

Proprietary products have advantages, to be sure: there's quick response to new feature requests, a single point for responsibility, and (they claim) more consistent behavior across platforms. All that aside, though, I think it will be very hard to persuade programmers and Web designers to adopt this proprietary solution.

In this "old problems" section, I see a deeper issue. Society has moved ahead six years. People are no longer so interested in stunning effects. They want simple, solid information sources. The Web designers who struggled with DHTML and other protocols have gotten used to them and are settling to maintaining legacy sites with thousands of pages. In other words, I think designers are struggling with new problem sets.

Deep entry

Luckily, Curl has another strategy. Instead of getting in at halftime, they're moving with the Web players to a new field.

First, they offer a SOAP interface. So you use them as part of a bigger solution and interoperate with everybody else. Second, they're useful on the back end. You don't have to run Curl programs over the Web. Curl includes a general-purpose language and can manipulate data locally or open a socket to another system. It can also store data on the client like other programming languages. So a Curl program can be used offline and hook up with a server later.

One might argue that Curl faces the problem typical of all companies trying to enter the Web game at halftime by offering a plug-in: They may be able to promise that the user will have access to scads of enticing programs after downloading the plug-in, but many users won't bother to take that first step. And sites will be reluctant to write Curl programs without the assurance of omnipresence. But with version 2.0 targeted at enterprise applications, the user no longer has to be the one to deal with the plug-in. Curl becomes omnipresent within that environment.

The most interesting scenario I discussed with the evangelists is one of facilitating the use of Web Services. Here's one way it might work.

Suppose that several Web sites, which we'll call A1, A2, and A3, offer interesting information like stock quotes or headlines. A user with a browser, which we'll call site B, wants to query and interact with all these sites. To make it easier, a programmer puts a Curl program at site C. This program does lots of sophisticated parsing and formatting and uses lots of nifty communications and user-interface features from the language.

The browser at B starts interaction at site C, downloading the Curl program. Now B can connect directly with A1, A2, and A3 through the program. The server at C is totally out of the picture by now and does not present a bottleneck. There's a quasi-P2P relationship between the A sites and B, but the interaction has been facilitated in invaluable ways by the program from C.

That's one way that Curl might prosper in a Web Services world and meet programmers on the ground they care about in 2002. I wish them well.

Andy

Got a compliment, complaint, or suggestion? Let us know!

Return to: letters.oreilly.com