Posts: 135 / Nickname: fsommers / Registered: January 19, 2002 7:24 AM
Sources of Java Errors
May 16, 2008 8:00 AM
|
While the Java VM shields most developers from having to think about the memory-management aspects their Java objects, the VM does not completely manage other types of resources automatically, says Gwyn Fisher, CTO of Klockwork in this interview with Artima:
http://www.artima.com/lejava/articles/javaone_2008_gwyn_fisher.html To what extent do you agree with Gwyn Fisher that great Java developers need to think about the interaction between the JVM and their environment, and about how the JVM interacts with Java objects? |
Posts: 14 / Nickname: aeoo / Registered: April 12, 2006 7:02 AM
Re: Sources of Java Errors
May 16, 2008 11:10 AM
|
> To what extent do you agree with Gwyn Fisher that great
> Java developers need to think about the interaction between > the JVM and their environment, and about how the JVM > interacts with Java objects? I don't agree 100%. I think a great developer should focus solely on the language definition and not worry about the JVM. Sometimes there will be a situation where knowing something about the JVM will be necessary, and then and only then should the developer study such interactions. It should be the burden of the language designers to provide VMs that behave so well that all you have to know is the language definition to use the language. You shouldn't have to know anything about the implementation. Of course because the world is not ideal, a good developer should be prepared to sometimes do what "shouldn't be done" if the need arises. But it shouldn't be a default stance of almost any Java developer to worry about the JVM, in my opinion. The only exception I can see here are the developers who are writing new languages to run on top of the JVM. |
Posts: 20 / Nickname: raoulduke / Registered: April 14, 2006 11:48 AM
Re: Sources of Java Errors
May 16, 2008 0:04 PM
|
yeah, well, i could have more of an opinion if they actually had a Mac OS X version. lost a sale there...
|
Posts: 9 / Nickname: nn / Registered: July 11, 2004 6:19 AM
Re: Sources of Java Errors
May 20, 2008 10:35 AM
|
The first thing that came to mind after reading the article was:
"All non-trivial abstractions, to some degree, are leaky." ( http://www.joelonsoftware.com/articles/LeakyAbstractions.html ) Some people think that programming with modern languages and libraries makes suddenly everything easy. Less laborious and cumbersome? Yes. Less complex overall? Not really. By the way, the modern Python idiom for this: http://docs.python.org/whatsnew/pep-343.html |
Posts: 4 / Nickname: hobb0001 / Registered: December 28, 2004 9:02 AM
Re: Sources of Java Errors
May 16, 2008 10:23 AM
|
Meh. None of it is really news to me. I may be different, though, since I came up through the ranks via C and C++, so I know all too well that memory is only one of the resources that needs to be released. Also, contrary to what is claimed in the article, most implementations will close sockets and JDBC connections during finalize() when the object is garbage collected. The problem with relying on finalize(), though, is that it is only invoked when available memory runs low, not when the number of available sockets or JDBC connections run low.
|
Posts: 14 / Nickname: aeoo / Registered: April 12, 2006 7:02 AM
Re: Sources of Java Errors
May 16, 2008 11:04 AM
|
> Also, contrary to
> what is claimed in the article, most implementations will > close sockets and JDBC connections during finalize() when > the object is garbage collected I disagree. Most implementations in my experience isolate the code that opens and closes a resource into a framework or a reusable class, and resources are closed in the finallyclause of the try { ... } finally { ... }block. |
Posts: 4 / Nickname: hobb0001 / Registered: December 28, 2004 9:02 AM
Re: Sources of Java Errors
May 16, 2008 0:01 PM
|
> > Also, contrary to
> > what is claimed in the article, most implementations > will > > close sockets and JDBC connections during finalize() > when > > the object is garbage collected > > I disagree. Most implementations in my experience isolate > the code that opens and closes a resource into a framework > or a reusable class, and resources are closed in the > finally clause of the try/finally block. Sorry, I should have clarified. I was referring to implementations of the java.net.Socket and the java.sql.Connection classes. Usually, it's Sun's implementation in the standard runtime, but some J2EE containers have been known to provide custom versions of those classes. |
Posts: 25 / Nickname: rp123 / Registered: January 7, 2006 9:42 PM
Re: Sources of Java Errors
May 17, 2008 1:24 AM
|
> I disagree. Most implementations in my experience isolate
> the code that opens and closes a resource into a framework > or a reusable class, and resources are closed in the > finally clause of the >try { ... } finally { ... }> block. Resources should be closed in finally clause. You see a lot of code, even from renowned authors, that insufficiently tries to handle exceptions in the catch clause. From the Hibernate documentation e.g.:
|
Posts: 98 / Nickname: achilleas / Registered: February 3, 2005 2:57 AM
Re: Sources of Java Errors
May 19, 2008 1:04 AM
|
Interesting quote...should a VM have only a memory GC or a memory GC coupled with a socket GC, handle GC etc?
This is an idea that I haven't seen anywhere, but it seems to me as critical as memory. A run-environment should have N garbage collectors, whereas N is the number of different type of resources needing to be managed by the run-time environment. This means the language should have pluggable GC architecture... |
Posts: 4 / Nickname: hobb0001 / Registered: December 28, 2004 9:02 AM
Re: Sources of Java Errors
May 19, 2008 7:45 AM
|
> Interesting quote...should a VM have only a memory GC or a
> memory GC coupled with a socket GC, handle GC etc? You would only need one GC if it was done right. Instead of kicking off a global collection only when memory runs low, the GC could call finalize() on an object the moment it is no longer reachable. For example, when a function returns, all of the local variables would be collected right then and there (as long as they aren't contained in the return value). The problem is that bulk-mode GC is usually more efficient than per-object GC. |
Posts: 20 / Nickname: raoulduke / Registered: April 14, 2006 11:48 AM
Re: Sources of Java Errors
May 19, 2008 1:07 PM
|
> You would only need one GC if it was done right. Instead
> of kicking off a global collection only when memory runs > low, the GC could call finalize() on an object the moment > it is no longer reachable. For example, when a function > returns, all of the local variables would be collected > right then and there (as long as they aren't contained in > the return value). > > The problem is that bulk-mode GC is usually more efficient > than per-object GC. Ja! A while back I kept trying to explain that idea to one of the Alice ML folks who seemed to have a different take on it all, that the program author can use higher order hoodly-doodly to make their own RAII type stuff. I appreciate all the talk of how letting things happen in libraries is great and all, but it can't be a truism for all features, otherwise there would be no core language ;-) and I sorta feel this RAII-GC thing such that should be in the language. What I think roughly I don't really know what I'm talking about want is something like: (a) a type to be of (multiply inherit / interface) which tells the GC that you have an RAII-finalize method. (b) the GC keeps a stack of allocation references in parallel / inside the regular call stack (or whatever else is appropriate, e.g. if you are talking about closures on the heap). that stack is only for types from (a), not every possible type. (c) the GC then runs the finalization, and configurably optionally the actual memory GC to boot, to close the resource when its scope exits. ?!?!? (d) profit. I guess, alternatively, if your language let people register plug ins for when the VM hits certain points e.g. all stack popping, it actually could be a library. |
Posts: 2 / Nickname: aleksf / Registered: August 16, 2006 5:02 AM
Re: Sources of Java Errors
May 19, 2008 2:48 AM
|
> Interesting quote...should a VM have only a memory GC or a
> memory GC coupled with a socket GC, handle GC etc? > > This is an idea that I haven't seen anywhere, but it seems > to me as critical as memory. It's actually quite old and known idea: http://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization > A run-environment should have N garbage collectors, > whereas N is the number of different type of resources > needing to be managed by the run-time environment. > > This means the language should have pluggable GC > architecture... RAII should be built in as it is in e.g. C++. What should be pluggable (i.e. optional) is memory garbage collection (in Java GC sense). |
Posts: 25 / Nickname: rp123 / Registered: January 7, 2006 9:42 PM
Re: Sources of Java Errors
May 19, 2008 9:05 AM
|
> RAII should be built in as it is in e.g. C++.
In C++ RAII only works for objects on the stack. IMO, deterministic resource management is the desideratum - for stack- and heap-objects. But we will see this feature only in the successor of both, Java and C++. |
Posts: 18 / Nickname: mlybbert / Registered: April 27, 2005 11:51 AM
Re: Sources of Java Errors
May 19, 2008 2:56 PM
|
> > RAII should be built in as it is in e.g. C++.
> > In C++ RAII only works for objects on the stack. IMO, > deterministic resource management is the desideratum - for > stack- and heap-objects. But we will see this feature only > in the successor of both, Java and C++. RAII works by creating handles to objects and placing those handles on the stack. The objects being managed are often on the heap, in shared memory (in which case the handle must use an inter-process reference counting mechanism) or may have nothing at all to do with memory (e.g., semaphores and file locks). The end result is deterministic resource management. Now that I've worked with C# some, I think there's a lot to be said for C#'s "using" blocks. However, I think that for the most part a garbage collector that knows about more than just memory would be perfectly fine. For instance, a garbage collector that allowed the user to (1) register file descriptors, (2) state that there is a maximum of XX file descriptors available, and (3) request garbage collection whenever open file descriptors reach 80% or more of the limit. This would be the same collector that would run if available memory fell too low or if some other resource became constrained. In the event that the garbage collector runs and is not able to free up file descriptors, then the garbage collector would essentially be saying that all open file descriptors are being used by live objects. If there are no more descriptors (or memory, or ...) available, there isn't much to do except wait for some of those objects to die or determine if you're in a deadlock. But there wouldn't be much to do in an alternate world with an alternate system. |
Posts: 26 / Nickname: cpurdy / Registered: December 23, 2004 0:16 AM
Re: Sources of Java Errors
May 22, 2008 11:27 AM
|
> > > RAII should be built in as it is in e.g. C++.
> > In C++ RAII only works for objects on the stack. > RAII works by creating handles to objects and placing > those handles on the stack. The objects being managed are > often on the heap, in shared memory (in which case the > handle must use an inter-process reference counting > mechanism) or may have nothing at all to do with memory > (e.g., semaphores and file locks). The end result is > deterministic resource management. .. for any application so simple that reference counting is sufficient. (We do use RAII with our C++ implementation. It is not sufficient.) Peace, Cameron Purdy | Oracle http://www.oracle.com/technology/products/coherence/index.html |
Posts: 25 / Nickname: rp123 / Registered: January 7, 2006 9:42 PM
Re: Sources of Java Errors
May 23, 2008 11:35 AM
|
> (We do use RAII with our C++ implementation. It is not
> sufficient.) Not sufficient in what respect? RAII "merely" encapsulates and automates resource management. Of course, like any idiom it can be misunderstood and used wrongly. But I'd like to see an example where RAII is 'not sufficient' for resource management. |
Posts: 8 / Nickname: indranil / Registered: November 7, 2004 8:29 AM
Re: Sources of Java Errors
May 23, 2008 11:45 AM
|
>
> .. for any application so simple that reference counting > is sufficient. > > (We do use RAII with our C++ implementation. It is not > sufficient.) > Last I heard all VMs use reference counting in their garbage collectors. What other techniques are you referring to? |
Posts: 25 / Nickname: rp123 / Registered: January 7, 2006 9:42 PM
Re: Sources of Java Errors
May 23, 2008 0:58 PM
|
RAII != reference counting && RAII != GC
|
Posts: 8 / Nickname: indranil / Registered: November 7, 2004 8:29 AM
Re: Sources of Java Errors
May 24, 2008 5:21 AM
|
> RAII != reference counting && RAII != GC
Indeed, my reply was directed at Cameron's comment which mentioned ref counting. Reference counting is a technique that is used in RAII libraries like std::shared_ptr and for garbage collection in the JVM and the CLR. The scoping based rules like RAII + reference counting can be used powerfully to manage system resources. Sometimes you dont need ref counts. I like to use scope guards to guarantee a function will always execute(deterministically) when a reference goes out of scope, much cleaner and more flexible than Java finally or C# using blocks. Reference counts come into their own when you're passing handles to resources around. Sometimes even this is not enough, which is why we have weak_ptrs and WeakReferences I would like to hear from Cameron where techniques for resource management in C++ have let him down. And how does he solve these problems in Java. |
Posts: 8 / Nickname: sebastiank / Registered: September 19, 2005 2:05 AM
Re: Sources of Java Errors
May 24, 2008 0:11 PM
|
Before crying for new language features, why not use good old indirection? e.g.:
instead of... <code> CloseableResource rs = new CloseableResource(); try { rs.foo(); } finally { if(rs!=null) { rs.close(); } } </code> ...use... <code> DerivedResource cr = new DerivedResource(); rs.access(new ResourceAccess() { void access(CloseableResource rs) { rs.foo(); } }) </code> The access method will do the dirty work. Remember, it's the power of OOP that let's us improve things by <i>deriving</i> (by inheritance or encapsulation) instead of <i>changing</i> things all the time. |
Posts: 8 / Nickname: indranil / Registered: November 7, 2004 8:29 AM
Re: Sources of Java Errors
May 24, 2008 2:55 PM
|
> <code>
> DerivedResource cr = new DerivedResource(); > rs.access(new ResourceAccess() { > void access(CloseableResource rs) { > rs.foo(); > } > }) > </code> > I guess I'm missing something. Where is the resource getting closed? |
Posts: 8 / Nickname: sebastiank / Registered: September 19, 2005 2:05 AM
Re: Sources of Java Errors
May 25, 2008 1:14 AM
|
Sorry, I clicked the "post message" button to early.
the access method would look something like the this: class DerivedResource { //... public void access(ResourceAccess ra) { CloseableResource rs = null; try { rs = new CloseableResource(); ra.access(rs); } finally { if(rs != null) { rs.close(); } } } } The whole acquiring and closing mechanism is delegated to DerivedResource now. That way, thare's no way to forget to close the resource. Another advantage is that the boiler plate closing routine does not get duplicated over and over in the project. If you've ever had the pleasure to fix hundreds of occurrencies of all possible variants of closing resources, you'll probably value that approach. |
Posts: 8 / Nickname: indranil / Registered: November 7, 2004 8:29 AM
Re: Sources of Java Errors
May 26, 2008 3:28 PM
|
Thanks Sebastian. I'm genuinely interested in the best practices for writing robust code in Java these days.
At first sight the ClosableResource version seems to involve just as much typing as the first version. I'd also like to see how this coding style works when there are multiple resources to be managed. Could you rewrite this snippet of code using your style? Would also be interested in seeing what the RAII gang come up with.
|
Posts: 25 / Nickname: rp123 / Registered: January 7, 2006 9:42 PM
Re: Sources of Java Errors
May 27, 2008 9:47 AM
|
> Could you rewrite this snippet of code using your style?
> Would also be interested in seeing what the RAII gang come > up with. ...
You often see this style in books and articles but it is not sufficient. Goetz demonstrates a proper solution with nested try/finally blocks here: http://www.ibm.com/developerworks/java/library/j-jtp03216.html (see esp. Listing 4). I would close all resources in the try block and write a special re-usable cleanup() function to be used in the finally block in case an exception occurs. With RAII you don't have those problems because cleanup is defined once per class, not each time the resource is used. If Java had RAII you could spare 95% of all try/catch/finally blocks in the code. Lack of RAII is responsible for most of Java's proverbial verbosity. |
Posts: 37 / Nickname: miata71 / Registered: March 29, 2006 6:09 AM
Re: Sources of Java Errors
May 27, 2008 10:51 AM
|
For multiple closes in the finally block,
One option is to use org.apache.commons.io.IOUtils.closeQuietly(x). It checks for x==null, and eats any exception, which makes sure that everything will get closed (if possible). Another idea (haven't see or testing this in practice) is for the wrapper class to collect a List of java.io.Closeable, and at the end to go (backwards) through this List closing things. But not all the JDBC still implements Closeable, only the java.io stuff does. The tricky issue is what to do if close() throws an Exception. Apache ignores it, which, IMO, is about as good as you'll get. If the preceeding IO failed, you definitely want to ignore the close() exception, cause it's semi-expected, and you want to throw the "real exception", which is something in the preceeding IO. If it's just the close() that failed, ???. I've NEVER seen this happen, and whatcha going to do about it anyway? I wrote my own IOUtils-like class (cleverly called Finally, it was a ton of methods to call in a Finally block) that returned any Exception, so the caller had the option to do something. But in practice I never checked the result, or at most just logged it and ignored it, so I started using the more standard Apache stuff. |
Posts: 25 / Nickname: rp123 / Registered: January 7, 2006 9:42 PM
Re: Sources of Java Errors
May 27, 2008 11:43 AM
|
> For multiple closes in the finally block,
> One option is to use > org.apache.commons.io.IOUtils.closeQuietly(x). > > It checks for x==null, and eats any exception, which makes > sure that everything will get closed (if possible). closeQuietly() merely eats one exception but, as Goetz notes, "[n]early everything can throw an exception" in Java (http://www.koders.com/java/fid219DBE7E24ABBA5AA65548B0305BD6786EF5DF22.aspx?s=package+org.apache.commons.io+IOUtils.java#L17). |
Posts: 8 / Nickname: indranil / Registered: November 7, 2004 8:29 AM
Re: Sources of Java Errors
May 27, 2008 0:10 PM
|
Looks like there are many libraries that provide a closeQuietly method for cleanup.
http://www.google.com/codesearch?q=lang%3Ajava+closeQuietly&hl=en This goes someway to reduce the error handling boilerplate code that you have to write in Java. But the onus is still on the developer to call closeQuietly for all closable resources everytime they are used. I'm looking for an abstraction that looks after resource cleanup with a simple interface that doesnt get in the way of the business logic of the function at hand. Roland has mentioned RAII, but I'd really like to see code (in the language of your choice) that does the same job as the snippet I posted earlier. |
Posts: 37 / Nickname: miata71 / Registered: March 29, 2006 6:09 AM
Re: Sources of Java Errors
May 27, 2008 4:05 PM
|
> closeQuietly() merely eats one exception but, as
> Goetz notes, "[n]early everything can throw an exception" > in Java It eats the declared (and possibly expected) IOException. As for eating any shockingly unexpected RuntimeExceptions, it's not clear if you want to do that - depends on the application. For many, that's a crash, you definitely want to throw the exception and basically exit. Cleanup of resources is LESS important that getting a clean stack dump. YMMV. |
Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
Re: Sources of Java Errors
May 29, 2008 10:58 AM
|
> With RAII you don't have those problems because cleanup is
> defined once per class, not each time the resource is > used. If Java had RAII you could spare 95% of all > try/catch/finally blocks in the code. Lack of RAII is > responsible for most of Java's proverbial verbosity. I'm not going to say RAII isn't useful but there are other approaches to address the issue. I've written a library that addresses this issue in Java for databases. A database query might look something like this: Query query = Manager.getQuery("select bar from foo"); query.execute(new Handler() { public void handle(Row row) { // do things with row } }); The resource cleanup is handled by the manager. |
Posts: 5 / Nickname: darkmx0z / Registered: March 11, 2008 11:38 AM
Re: Sources of Java Errors
June 1, 2008 6:47 AM
|
example of RAII in C++
using this class:
Any File instance will be closed at the end of its scope or when an exception forces the stack to be cleaned. f will be deterministicly closed even in an exception was thrown. |