Summary
I've noticed a lot of wrong statements about how java source files must be in a directory tree mirroring the package name. In fact, eclipse gets it wrong, and the developers refuse to fix it!
Advertisement
I have been trying to find a new Java IDE to replace my discontinued favorite, Kawa. While doing this, I have tried to take my largest Java applications, 400-600 class files and add them into various IDEs to see how they deal with large projects.
I recently downloaded and tried this with Eclipse and was surprised to find that it flat out ignores the package statements in my source files! I fired off some postings to the eclipse.org news groups and was greeted with a bunch of comments from individuals that believe that you have to put your source files in to directory structures that mirror the package names!
I went out to the Java specification and quoted parts of 7.3 and 7.4 related to the fact that the package statement in the source file is the package that the class belongs in. There is text elsewhere that states that the class files need to be distiguishable by package so that the JVM can find a class easily. There is also text about storing class files in a database, which is not a file system.
YOU DO NOT HAVE TO STORE YOUR SOURCE FILES IN A PACKAGE STRUCTURE!
NO, REALLY YOU DON'T!
Anders Hejlsberg in [http://windows.oreilly.com/news/hejlsberg_0800.html] even said "In Java, the packaging is also your physical structure, and because of this a Java source file has to be in the right directory...", so he doesn't understand this issue either.
Your class files are in a structure that allows them to be located by the JVM, what ever its requirements are. Your source file locations can be arbitrary!
How will the compiler find source files
Many people ask things like, "How will the compiler find the appropriate source files to compile if it can't go to the file based on the package name?" I tell them the compiler should, and must compile the list of source files that I give it. It should use the class path to resolve class files that there are no sources for, and it should write the class files out based on the compilers arguments and the vendors implementation.
The javac compiler has the '-d dir' argument that causes the compiler to write the package structure out start at the indicated directory and going down from there.
Imagine that if the compiler only compiled source files that had explicit references in them. What would happen for classes referenced only by Class.forName()? They wouldn't get compiled! That would be bad!
Others have told me that when the source and the class file are in a package named directory tree, that incremental builds go faster because the compiler can check dependencies and find the source file to open and recompile if it detects such a need. The problem is, that as soon as you introduce a SCC (source code control) system, you can not count on source file dates to drive incremental builds. You have to take into account that some SCC operations might get an older version of a source file that someone wants to build with, and a simple date check will not detect that the source file needs to be recompiled.
For this very reason, IDEs should support either rebuild all, or compile this file, and that is all.
There are several other scenarios where smart compiler tricks with dates and paths do not work either. If an IDE wanted to keep MD5 checksums of every source file, and then compile based on the checksum changing, that would create a nearly bullet proof incremental build.
What's up with Eclipse?
The Eclipse problem really grates on me. The funny thing is that when I compile my large projects, I see messages from the compiler stating that the package statement does not agree with the directory. And I have to ask "So why not just use the package statement?"
Have you made changes to your package structures? Do you think the IDE should force you to structure your source files in some particular way? Do you want to add random source files from other projects to your current project so that you can debug an interaction more readily?
Does your IDE stay out of your way, or force you to work in a particular, contained way?
First of all, I think you are technically accurate on your observations, and that there is a lot of misunderstanding on this point. Java does not enforce package structure on source files.
This does not necessarily mean that you therefore shouldn't keep source in a package structure. The language doesn't discourage that practice. Some people might infer that the practice is encouraged, though I think that's stretching things a bit. I'm sure many people find this practice to be very beneficial, however.
IDEs should support either rebuild all, or compile this file, and that is all.
Although it has many merits, I disagree with this as a blanket statement. It can be useful to allow compilation based on time stamps if the SCC updates time stamps to "now" when getting the latest copy. Also, when not updating from an SCC system, it's more convenient to have only the necessary files be recompiled.
However, the javac compiler has incomplete information and sometimes does not recompile things that should be recompiled. Extrapolating on your suggestion, an outside build system (perhaps Ant or an IDE) could probably do a complete job of recompiling files only out of necessity based on dependencies. (This assumes the SCC system playes by the "now" rule mentioned above.)
> IDEs should support either rebuild all, or compile this > file, and that is all. > > Although it has many merits, I disagree with this as a > blanket statement. It can be useful to allow compilation > based on time stamps if the SCC updates time stamps to > "now" when getting the latest copy. Also, when not > updating from an SCC system, it's more convenient to have > only the necessary files be recompiled.
Yes, while it can seem advantageous, it is problematic enough that I am just not comfortable with it. I have seen this bite people too many time because many SCC environments let the users choose how the time is updated, and it never seems that it is consistent.
> However, the javac compiler has incomplete information and > sometimes does not recompile things that should be > recompiled.
I am not sure what you mean by this. I have always used
<code>javac -d <dest> ...files...</code>
for compiling source files, and I've never recalled getting unexpected results. What experience have you had with unexpected results? What arguments were you using and what version of the compiler?
> Extrapolating on your suggestion, an outside > build system (perhaps Ant or an IDE) could probably do a > complete job of recompiling files only out of necessity > based on dependencies. (This assumes the SCC system > playes by the "now" rule mentioned above.)
The biggest problem is Class.forName(). You just can't really detect all the referenced files and how they are used to know if the modifications on one file imply a recompilation of another file, without a complete introspection of the class files in question. By the time you do that, recompilation of all files is probably going to be done.
On my 400-600 file projects that generate 1.2MB jar files, the build lasts just over 1 minute under ANT. If I just do javac -d with the IDE, it seems like it takes about 40 seconds. I can wait that long occasionally to make sure I have a complete build!
>> However, the javac compiler has incomplete information and >> sometimes does not recompile things that should be >> recompiled. >I am not sure what you mean by this. I have always used >javac -d <dest> ...files...
Example: A.java depends on B.java which depends on C.java
Files B.java and C.java change. I believe if you compile with:
javac -d <dest> B.java
C.java will also compile.
But technically A.java is out of date, as well, and should be recompiled. For example, if the change to B.java is to the value of a static final variable that A.java used, and if the compiler chose to optimize things so that A.class had the value of that static final variable locally, then you could get very unexpected behavior.
I have a vague memory of one other issue, but I can't remember it right now. But certainly periodic complete recompiles are crucial for sanity.
>The biggest problem is Class.forName().
I don't quite understand how Class.forName() is a problem. There isn't really a static compile-time dependence, so isn't this outside of the realm of compilation, anyways?
>the build lasts just over 1 minute under ANT
The fact that machines are as fast as they are now certainly supports your position. If complete recompiles are timewise relatively inconsequential, then that is definitely the best way to go. I'm not positive this is always the case, but it's getting to be moreso as time goes on.
And so, have you found an IDE that works the way you think it should?
Also, how are your projects arranged? I have mine in something like:
HelloWorld ->\bin ->\src
And eclipse works fine with it. Took me a bit to figure out how to do this but I found a good tutorial on it. Couldn't you define a structure as well or is eclipse not "flexible" enough to handle larger projects?
> Actually I used to hate the way visual age maintained > it's own file structure. That was the workaround then. > I actually find it better like it is now.
Yes, visual age was even more difficult because it used its own storage environment and you could not use your SCC with it unless you checked everything out, and did an import. At the end of the day, you would have to do an export and then check everything back in. That made it tough for teams in many cases.
> And so, have you found an IDE that works the way you think > it should?
Well, I am partial to Kawa which is very similar to the simple Symantec Cafe that I started using in 1996 or so :-)
> Also, how are your projects arranged?
I have my projects arranged so that directories are specific to a part of the project that is not exactly in line with packages. For instance, in one project, all the interfaces are in the same directory. That makes it very easy to visually inspect something you need history on in the SCC browser for instance. Basically, I really hate have the top three layers of my source tree empty because of package structure. So, I typically do not have large projects arranged in package structures because it makes them really hard to navigate by directory. I do this on a regular basis using command line on linux, and explorer on windows for a number of reasons. I do it often enough that the extra 10 seconds of typing each time starts to add up on the wall clock.
> And eclipse works fine with it. Took me a bit to figure > out how to do this but I found a good tutorial on it. > Couldn't you define a structure as well or is eclipse not > "flexible" enough to handle larger projects?
Eclipse, I think, has a number of problems about not letting the developer be in control. I can't really validate my initial feelings though because of the fact that its compiler ignores the package statements in my source files. I'd like to use it on my large projects long enough to make a decision without going through a whole day or so of rearranging the source tree, fixing tools that have directory names in them etc. These projects are more than 5 years old, and there is a lot of historical information in the SCC tree, as well as work in the tools involved in the builds and documentation. If eclipse would read the package name, like the language spec mandates, all would be well for the next step it seems.
I don't know how everyone else works, but I spend a lot of time working in single source files and pushing F7 in kawa to compile that file. If I get to an interface dependency change or some other globally visible change, I use 'rebuild all' to make sure that everyone still agrees on signatures, types etc.
I'm not sure how others work, or how the eclipse team works, but the IDE environment in eclipse feels very confining and restrictive of my ability to work quickly.
You have the technical facts right but the human interface wrong. The thing to remember is that the source file hierarchy is not primarily for the compiler - it's for the other people who read your code. When I'm looking for the source code for the class named org.whatsit.foo.Bar, I look in the corresponding place in the source tree. I don't want to have to search around for the source file. Being "creative" about arranging your source tree just ensures that your source code is that much harder to read. So I'm happy that IDE's enforce the conventional way of doing things, even if it's not strictly required by javac.
Having empty directories at the top used to bug me too, but I got used to it, and now I wouldn't have it any other way.
Although I don't have a problem with arranging code in directories according to their packages, (and this is how I do it,) I do think that an IDE should just be there to assist.
You dont expect your GUIs to implement business logic, why should the IDEs implement compile-time checking logic ?
I used to use Kawa as well and found it very good and light. On the other hand Ive found VAJ a bit of a dinosaur. Huge footprint and stuck to the past ! I couldnt even change basic parameters, such as the JDK/JRE
Im know using TextPad (!) and Ant.
You might think that this is a step back, but I can write my code, press a button IN TextPad and have an Ant script of my selection run and all output get printed IN my TextPad window. I can pass parameters to the Ant script and just run a build or run a full build with javadoc, testing and deployment.
It is not very strong in the SCC area, but Ant can be used for that as well in most cases.
> You have the technical facts right but the human interface > wrong. The thing to remember is that the source file > hierarchy is not primarily for the compiler - it's for the > other people who read your code. When I'm looking for the > source code for the class named org.whatsit.foo.Bar, I > look in the corresponding place in the source tree. I > don't want to have to search around for the source file. > Being "creative" about arranging your source tree just > ensures that your source code is that much harder to > read. So I'm happy that IDE's enforce the conventional > way of doing things, even if it's not strictly required > by javac. >
I'm in agreement with this too. I use Eclipse every day at work for PHP development, and don't feel constrained by it all. I work with a team of 3 other developers so I have to be able to easily find code written by others. If the IDE forces EVERYONE to locate files in a specific place, then I can be sure where things are located.
I used to develop with Kawa -- then moved to vi again by failing to find another IDE. Now using Eclipse, I don't mind the directory thing; I used to put source in truncated package structures -- javac doesn't care, however, javadoc does. If you want to use javadoc (certainly back in 1.2.2) it forced you to used the directory structure.
These days I think it's worth keeping it, just because it's what is expected (both by other developers and numerous tools). It's not THAT annoying.
> Your class files are in a structure that allows them to be > located by the JVM, what ever its requirements are. Your > source file locations can be arbitrary!
Assuming: a) Your code is sometimes read or edited by others, and b) You agree with the general coding principle of "least surprise".
I'd be inclined to go with the flow on this one and place the source code where others would expect to find it, rather than somewhere arbitrary.
Equally, I'm no fan of the Java conventions for placing brackets or capitalizing variable (and other) names but life is too short to plough my own furrow on these issues.
Try IntelliJ. It does the best incremental compilation I've ever seen. When it compiles your project, it saves the exact timestamps of the source files it's using, so the SCC problem of older timestamps is no problem. And it persists dependency info to disk as well, so it's really quite intelligent about which classes and files truly need to be recompiled.
At my previous company I was working on a large codebase with perhaps 25 other developers; a full javac would take several minutes (on my Pentium 4, maybe 1.4GHz). I found that in IntelliJ, I almost never had to "build all", even after syncing up (if memory serves); the incremental compile just got it right.
Flat View: This topic has 19 replies
on 2 pages
[
12
|
»
]
Previous Topic
