Anders Hejlsberg, the lead C# architect, talks with Bruce Eckel and Bill Venners about the trouble with distributed systems infrastructures that attempt to make the network transparent, and object-relational mappings that attempt to make the database invisible. The conversation is also joined by Dan Fernandez, Microsoft's Product Manager for C#, and Eric Gunnerson, C# Compiler Program Manager.

Read this Artima.com interview with the creator of C#:

http://www.artima.com/intv/abstract.html

What is your opinion of the issues discussed in this interview?

Comments on this Article

1. Please refrain from turning technical articles into a platform for passive-aggresive non-MS technology bashing. Implying that CORBA/IIOP is an inherently flawed technology because low skilled "system architects" and "designers" didn't know how to use it correctly is just plain foolish. In any case, where does MS's previous contribution to this space (DCOM, COM+ etc) factor into this ? Perhaps MS should critique its own technologies rather than taking the opportunity to try and discredit its competitors.

2. You gotta love a couple of the opening statements of this article "When we first sat down to design the .NET framework we took a step back ..." and "The prevailing wisdom five or ten years ago about how distributed systems would be built in the future was was CORBA, IIOP, object request brokers. The rave at the time..." . A more accurate opening would perhaps have been "When we first sat down to reinvent the wheel for the 20th time we took a step back and decided to use distributed system design techniques that have been around for the last 20 years and give it a new acronym (SOA) so it seems like we dreamt it up..." and "The prevailing wisdom five or ten years ago about how distributed systems would be built in the future was DCOM/COM+, MS Transaction Server and MS Message Queue. The rave at the time..."

3. Can someone please explain what the whole conversation under the "The Stateless Fashion" was about?. Bill Venners asks the (quite reasonable) question "What is the advantage of not having state in the distribution mechanism?". Anders answer is a simplified description of what a web service is (maybe he misunderstood the question). He then goes on to tell us that that we should be thankful that they don't provide us with any session mechanism so we can reinvent it every time for ourselves.

4. With regards to "Object-Relational Mappings", when asked "What are the main issues with O/R mappings?" we are told that caching is critical and most O/R mapping technologies (other than .NET of course in which they "tried hard to make sure the caching policy can be entirely under your control") have inflexible policies. Well, um, err I'm not sure what non-.NET O/R tools Anders has looked at but based on the answer to the question it would seem to have been a somewhat cursory investigation. Oh wait - I get it - you're trying to have a go at Entity Beans without actually saying the J word - gotcha.

All things considered, a drastically oversimplified marketing-oriented "look at what MS has invented now" article. Very disapointing.

> Comments on this Article
>
> 1. Please refrain from turning technical articles into a
> platform for passive-aggresive non-MS technology bashing.
> Implying that CORBA/IIOP is an inherently flawed
> technology because low skilled "system architects" and
> "designers" didn't know how to use it correctly is just
> plain foolish. In any case, where does MS's previous
> contribution to this space (DCOM, COM+ etc) factor into
> this ? Perhaps MS should critique its own technologies
> rather than taking the opportunity to try and discredit
> its competitors.
>
I must say, I sense a lot more MS-bashing in your comment than there ever was non-MS bashing in the interview. Anders was talking about an idea more than a specific technology in the section about remote objects. The example he gave was CORBA, but it was clear to me as I sat there that he was talking about all remote object architectures. He also didn't bash them in all situations. He said that up to a local area network they work well, they just don't scale to the internet. He may have had a secret plan to bash CORBA, but I saw it as an appropriate example given that CORBA was the big dream for inter-language inter-operability. Now web services seems to be that big dream.

> 3. Can someone please explain what the whole conversation
> under the "The Stateless Fashion" was about?. Bill
> Venners asks the (quite reasonable) question "What is
> the advantage of not having state in the distribution
> mechanism?". Anders answer is a simplified description
> of what a web service is (maybe he misunderstood the
> question). He then goes on to tell us that that we should
> be thankful that they don't provide us with any session
> mechanism so we can reinvent it every time for ourselves.
>
Well, I was trying to figure out what Anders meant by stateless in the O'Reilly interview. What I finally figured out was that he meant there is no state in the transport protocol. I.e., HTTP is a stateless protocol. Something like CORBA (or DCOM, or RMI) has state built-in, because it says that as long as you hold onto this reference to the local proxy, the remote object will be kept alive across the network. I kept asking him to clarify because it took me a while to understand the distinction he was trying to make. My current understanding, at least, is that most of these web services interactions will likely have state on the server, so they aren't stateless in that sense. But because the protocol doesn't mandate a certain kind of state model, developers who build these web services have more choices in how that state is implemented. So I believe what Anders was trying to say with the "loosely coupled, stateless fashion" comment in his O'Reilly interview was that by allowing developers to "reinvent a session mechanism each time," as you might put it, developers have the flexibility to do what is required to make the session mechanism scale to internet sizes.

> 4. With regards to "Object-Relational Mappings",
> when asked "What are the main issues with O/R
> mappings?" we are told that caching is critical and
> most O/R mapping technologies (other than .NET of course
> in which they "tried hard to make sure the caching
> policy can be entirely under your control") have
> inflexible policies. Well, um, err I'm not sure what
> non-.NET O/R tools Anders has looked at but based on the
> answer to the question it would seem to have been a
> somewhat cursory investigation. Oh wait - I get it -
> you're trying to have a go at Entity Beans without
> actually saying the J word - gotcha.
>
Is that what he meant? He may have, I don't know.

Everyone I interview has some kind of agenda to push somehow, and to a greater or lesser extent they always do. For some interviewees it is as simple as mentioning a book that happened to have recently written. For others, is pointing out how much better their language is than the competition's. I am also keenly aware that high-visibility people who work for large corporations, such as James Gosling and Anders Hejlsberg, can't always say exactly what they think, because they are seen as corporate spokesperson by the outside world. Therefore, they have to try and not embarrass their employer, because it could potentially hurt its stock price or at least cause a few sales to be lost to the competition.

I attempt to do a couple things to deal with agendas in editing these interviews. I try to cut out most blatant plugs and competition bashing, but I leave in respectful comparisons, especially if they are trying to make a general point with the specific comparison. Also, I make it clear who everyone works for. Every reader needs to take into account on whose team the interviewee plays as they read the interview.

> All things considered, a drastically oversimplified
> marketing-oriented "look at what MS has invented now"
> article. Very disapointing.

I wonder to what extent your reaction comes from the actual content of the article versus the anti-MS glasses through which you seem to be reading the article. Personally, I am also quite concerned and frustrated by MS's business practices, but not to the extent that I refuse to listen to anyone who works at MS. Anders is a smart guy who has done some amazing things in his career. I gained several new insights through the process of interviewing him, and I have been trying to pass those along along to readers through the articles.

My intent in this article was to publish the idea that even though as designers we try to raise the level of abstraction as high as possible, sometimes we can go too high. I found it quite interesting that Anders mentioned the problems with attempting to hide the network. That is the exact point made by Jim Waldo, et. al., in The Note on Distributed Computing and one of the central ideas that influenced the design of Jini. In other words, allowing an object to be implemented locally or remotely as an implemention decision on the part of the local object is an inappropriate abstraction, because the nature of local and remote computing is just too different to fit well in the same abstraction. I think that is a general design insight that goes beyond remote object artchitectures and O/R mappings: we should try to raise the level of abstraction up as high as possible, but no higher.

I wanted to add some references to using CORBA in a stateless fashion specifically to provide very high scalability.

There are two papers on Douglas Schmidt's website (http://www.cs.wustl.edu/~schmidt) that specifically address this:
http://www.cs.wustl.edu/~schmidt/PDF/load_balancing1.pdf
http://www.cs.wustl.edu/~schmidt/PDF/load_balancing2.pdf

Just to make a few quick comments about how CORBA deals with these kind of abstractions:

1) CORBA objects are not synonymous with object instances; and objects are held in stateful memory unless implemented to.

2) A CORBA ServantLocator can map N "objects" to actual instances, where N can be much greater than the number of real instances.

3) Effective use of the IdAssignmentPolicy enables CORBA objects to encode state into the object reference (and therefore out of the protocol).

My experience with CORBA indicates that there are *many* parameters under the control of the developer and that CORBA meets all of the goals that Anders Hejlsberg laid out for stateless communication.

However, I completely understand that the complexity and small exposure of these features dooms CORBA to a niche technology. The best I can hope for now is that the power of CORBA and the clear benefits of HTTP scalability can be merged into an elegant solution.

John Heintz

> > Comments on this Article
> >
> > 1. Please refrain from turning technical articles into
> a
> > platform for passive-aggresive non-MS technology
> bashing.
> > Implying that CORBA/IIOP is an inherently flawed
> > technology because low skilled "system architects" and
> > "designers" didn't know how to use it correctly is just
> > plain foolish. In any case, where does MS's previous
> > contribution to this space (DCOM, COM+ etc) factor into
> > this ? Perhaps MS should critique its own technologies
> > rather than taking the opportunity to try and discredit
> > its competitors.
> >
> I must say, I sense a lot more MS-bashing in your comment
> than there ever was non-MS bashing in the interview.
> Anders was talking about an idea more than a specific
> technology in the section about remote objects. The
> example he gave was CORBA, but it was clear to me as I sat
> there that he was talking about all remote object
> architectures. He also didn't bash them in all situations.
> He said that up to a local area network they work well,
> they just don't scale to the internet. He may have had a
> secret plan to bash CORBA, but I saw it as an appropriate
> example given that CORBA was the big dream for
> inter-language inter-operability. Now web services seems
> to be that big dream.
>

Regarding your response to Point 1:
DCOM was MS's answer to CORBA. It was marketed as a "big dream" language independent and interoperable technology from day one. They even tried to port it to UNIX to counter CORBA's OS independence. Try entering "DCOM language independence" or similar into Google if you doubt it was spun this way.

Regarding your response to Point 3:
Most applications have simple requirements. Providing open ended flexbility for something as pivotal to application/service development as state or session management is simply a long piece of rope that inexperience developers will hang themselves with. Next we'll be hearing that you don't get security or transactions either for the sake of flexibility - oh wait ...

It is well understood that client/server reference counting or heartbeat etc based mechanisms are problematic - but not providing any basic mechanism at all isnt a "flexible" solution. To me it's a glib way of saying "its not our (MS) problem".

Regarding your response to Point 4:
Like all things, his answer is open to interpretation. My question, therefore, is what other interpretation could be placed on the statement "All of these O/R mappings usually live and die by whether they are flexible enough in their caching policies, and most of them are not.". I ask you, what "most of them" was he referring to? Most non-trivial O/R products (both commercial and Open Source) I have dealt with provide significant cache control and configuration options. What about the dedicated object-cache applications, both commercial and open source, that are readily available and generally very easy to use? My issue here is that the statement he made is factually incorrect and is being used to flog the .NET donkey yet again.

Furthermore, I was astounded by the statement "On a middle-tier, for example, you quite often don't care about caching, because you're just serving some incoming HTTP request that's immediately going to go away thereafter. So why cache?. Well not to dwell on it but one good use of caching on the middle tier is to construct .NET Petshop "best practices" applications that cache everything in order to demonstrate .NET's superior performance and scalability compared to J2EE (/sarcasm off).


My personal anti-MS sentiment does not govern the content of my comments on this article (although it obviously colours the tone of it somewhat). As an unbiased technical article, factually incorrect sweeping generalisations should not go unchallenged.

Many less experienced architects, developers and designers get their information from articles just like this one. Do you want them to come away with the idea that "only .NET has flexible caching" or "only .NET understands that stateless service models are scalable" etc ?

If the overall message we are supposed to get from this article is that local/remote/distributed transparency is largely impractical then I guess we all "got it" - but then again I thought this was well understood 20 years ago ...

> Regarding your response to Point 3:
> Most applications have simple requirements. Providing open
> ended flexbility for something as pivotal to
> application/service development as state or session
> management is simply a long piece of rope that
> inexperience developers will hang themselves with. Next
> we'll be hearing that you don't get security or
> transactions either for the sake of flexibility - oh wait
> ...
>
That's a valid point about inexperienced developers, but just because the protocol doesn't have state doesn't mean inexperienced developers have to create their own session management from scratch. Most often I expect developers, inexperienced or experienced, would use an API layer on top of the protocol. You have session state on this website when you post to this forum, but I didn't write the session management. That's provided by Java's servlet API and I just use it.

> Regarding your response to Point 4:
> Like all things, his answer is open to interpretation. My
> question, therefore, is what other interpretation could be
> placed on the statement "All of these O/R mappings
> usually live and die by whether they are flexible enough
> in their caching policies, and most of them are not.".
> I ask you, what "most of them" was he referring to?
> Most non-trivial O/R products (both commercial and Open
> Source) I have dealt with provide significant cache
> control and configuration options. What about the
> dedicated object-cache applications, both commercial and
> open source, that are readily available and generally very
> easy to use? My issue here is that the statement he made
> is factually incorrect and is being used to flog the .NET
> donkey yet again.
>
What O/R mappings do you mean by "most of them" would have made a good follow up question there. I'm curious now and will see if I can get an answer from him on that. But once again, I didn't percieve his comments to be flagrant .NET promotion. When he was talking about O/R mappings in general, he really seemed to be talking about O/R mappings in general.

> My personal anti-MS sentiment does not govern the content
> of my comments on this article (although it obviously
> colours the tone of it somewhat). As an unbiased technical
> article, factually incorrect sweeping generalisations
> should not go unchallenged.
>
And I appreciate posts that point out such unsupported claims. I sometimes neglect to ask a follow up question to challenge a unsupported claim in these interviews. Sometimes I don't notice the claim during the interview. Other times I notice it but just don't have time to ask a follow up because I need to move on to other topics. I generally don't publish blatant competitor bashing. I just leave those parts out. But when unsupported claims get through, I've found that one of the roles these forums play is that readers will object to them here. And when I'm lucky, the interviewee actually responds in here, so the conversation can continue.

> Many less experienced architects, developers and designers
> get their information from articles just like this one. Do
> you want them to come away with the idea that "only .NET
> has flexible caching" or "only .NET understands that
> stateless service models are scalable" etc ?
>
No, I don't. But first of all I'm not convinced that most people will come to the same conclusions that you did. What a reader takes to an article has a lot to do with what they get out of it. Second, you are right that this article, and the Anders series in general, is biased towards C# and .NET. It does include words that could serve to promote those technologies.

But similarly, the James Gosling interviews are decidely biased towards Java, Guido van Rossum towards Python, Matz towards Ruby, Bertrand Meyer towards Eiffel, and Bjarne Stroustrup towards C++. I don't think this phenomenon arises soley because it may be in a language designers economic interest to promote his own language. The language design reflects the opinions of the designer. I try to get at those opinions, the ideas behind the language, but I go through the language to get to them.

For example, I don't believe that Bertrand Meyer thinks that Design by Contract is good because it's in Eiffel. Rather, I think Design by Contract is in Eiffel, because Bertrand Meyer believes that it is a better way to develop software. And I wanted to know why. So that's what I asked him, and in his answer Bertrand talked all about Eiffel. He likely wanted to take the opportunity of the interview to promote Eiffel a bit, but he was also just answering the question I asked him. And in the process, he provided some insights into the motivation behind the concept of Design by Contract. That's the kind of thing I was trying to do in this article. We were talking about web services, which is something very central to .NET, but what I really was trying to get at was what was this lesson he said we'd all learned about stateless interactions.

Hejlsberg's logic is truly messed up.

First he tries to explain that certain abstractions aren't good (i.e. Distributed Objects and O/R Mapping). The he proclaims web services as the right solution since it uses HTTP which we've seen to scale. However, he completely side steps the point that all of Microsoft's tools to support web service support the abstraction of distributed objects. That is grab a WSDL file and then call a stub that reaches out to a distributed object that is some other machine. Now if he talked about REST then he'll be at least correct.

Okay, he rightly observed the caching problem with O/R mapping, however any reasonable robust O/R mapping tool (i.e. TopLink or Hibernate) allows you to selective turn off the cache! All he did was criticize an implementation detail however fundamentally he didn't expose the problems of O/R mapping.

Oh well, some people just have to promote the party line!

Carlos

It's not always a bad thing to abstract away the fact that some procedure calls are remote. It may be inappropriate in some circumstances, when building applications of a particular scale, on a particular platform, yadda, yadda, but I really feel uncomfortable on making an ultimate decision on the matter.

It really depends on your audience. If you expect an advanced software group to be building on your distributed application, then certainly it may be a poor idea to hide the fact that it's distributed. The ten or so famous fallacies about network programming ("the network is reliable", etc.) come to mind. But at the same time, if you're writing a small application that operates in a distributed manner, and you expect reasonably new programmers to just pick up serialization, marshalling, remote exception, command-line-generated stubs and skeletons, unserializable this-and-that, you may be surprised to find that a lot of people are confused silly.

The problem is that, ultimately, you need to think about what you're trying to accomplish, too. It's nice to worry about remote exceptions, data transfer objects, and layers of wrappers wrapping wrappers, but when it's like one line of useful code per fifty boilerplate, one begins to wonder, "What do I have to do now, before I can do what I'm trying to do, and what was it that I was trying to do?" It's a natural inclination to want to abstract that stuff away. It may not be *safe*, but the wildfire of fully distributed, rapidly developed super-web-apps isn't going to catch unless the code makes sense to people that have to read and maintain it. So there's the conflict.

Though I agree that the network should probably not be fully hidden from the application, especially one that intends to integrate with it after all, it'd be nice if maybe we could at least turn down the volume on it a little bit.

And I don't see SOAP as being a step in the right direction. At all. XML is earning a reputation as a really verbose language. Let's not build a really verbose standard on it and then pretend we can sling around giant packets of tags of tags of tags per-function-call and live some utopian distributed dream. I'm not one for premature optimization, but that one's a premature bulkification. CPU speeds are really speeding up, but do we really have to fight back *that hard*?

I’m currently evaluating persistence solutions and here are my big three requirements:

* Can I navigate through an object graph and have the graph loaded lazily?
* Will the solution perform the book keeping tasks associated with the create and removal operations that occur to an object graph such that when I commit my transaction, my new objects get inserted into the DB and that stuff I trimmed away gets deleted.
* Is the solution manageable by a small team(3 heads) with object models in the 1000s of classes?

Ander's comments on object identity are spot on and frankly I can take it or leave it, but I’m not sure he’s got all the facts when it comes to caching. Who knows, maybe he does and his attacks should be interpreted as Microsoft’s entrance into the Object Persistence market. ;-)

But really, what is the Microsoft story on object persistence? Is there one? Is it ObjectSpaces? WinFS? ADO?

Roy T Fielding describes the trade-offs of sessionless protocols matter-of-factly in chapter 3 of "Architectural Styles and the Design of Network-based Software Architectures" http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm (2000)

He would be an interesting person to interview :-)


3.4.3 Client-Stateless-Server (CSS)

The client-stateless-server style derives from client-server with the additional constraint that no session state is allowed on the server component. Each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is kept entirely on the client.

These constraints improve the properties of visibility, reliability, and scalability. Visibility is improved because a monitoring system does not have to look beyond a single request datum in order to determine the full nature of the request. Reliability is improved because it eases the task of recovering from partial failures [133]. Scalability is improved because not having to store state between requests allows the server component to quickly free resources and further simplifies implementation.

The disadvantage of client-stateless-server is that it may decrease network performance by increasing the repetitive data (per-interaction overhead) sent in a series of requests, since that data cannot be left on the server in a shared context.

Martin Fowler also discusses the issue of having to design differently for distributed versus local in Patterns of Enterprise Application Architecture.

I think it is great that this article stirred so many comments to the validity of its content. I think it speaks to this formum's abilities to get comments out for review.

This article and the comments stir up all kinds of emotions about the validity of the MS technologies and about whether there is anything new in them. We all know that historically, MS has took good technologies and tried to sell them. Sometimes their implementations have failed to be attractive. Othertimes, they've found some niche as well as large target audiences.

Our abilities to adopt, support and utilize the right technologies is what matters. While the .NET platform may provide some new concepts, or even a packaging of well known concepts that is convenient for some, we all get to decide whether it is the answer or not.

I've decided on Java because of operating system portabilitiy. That is the predominate issue for me. I'll never know when MS will either cease to exist, out-price my checkbook, or out-license my nerves. Redhat changed the roadmap for redhat linux users, overnight. Other OS vendors, such as Apple and IBM can do similar things. Having a programming platform that is available on a wide choice of OSes and hardware is vital to my business and sanity, and thus MS is not in my deck of cards.

The MS technologies are so interwoven and so mystically complicated by the history of the windows platform that I just can't stand around and wait for real solutions from them any longer.

MS will continue to try and hire away great brains to work in their research labs. Great ideas will come out of those labs for really interesting technologies.

But, the problem with the MS platform is not that it needs a few new technologies. What it needs is a completely new base.

Apple figured this out for their OS. They replaced it, and kept most of their programming API, and look what it got them...

The .NET technologies are what MS is trying to make into their answer. But, that changes the whole programming paradigm. Their legacy API was insecure, broken and essentially irrepairable. It is so intertwined with the OS as to be inseparable. So, they have to create a new programming API for the application layer so that they can move to a new user API to OS interface that closes the doors on security, complexity and instability.

Let's see where MS is in another 2 years...