Summary
I'm curious, how many developers spend time consciously designing and managing relationships between physical entities?
Advertisement
One of the most challenging aspects of software development is managing and enforcing design and architecture. An elegant conceptual design can quickly evolve into a tangled mess during implementation, where not one developer grasps how the high level vision has manifested itself in code. Attempting to understand and manage design and architecture by manually synchronizing documentation with a constantly evolving code base is a lost cause.
The problem grows worse for large systems developed by large teams. Attempting to manage the logical dependency relationships between thousands of classes is fruitless. While TDD can help, this is even difficult because of the wide array of experience found among the development team. Basically, some developers are better at TDD than others, and it's difficult to police everyone on a large team. It might be possible to manage relationships between packages, but even this can be difficult for large systems where you might have hundreds of packages. And logical dependencies really only address the maintenance issue of software, but do not help much to promote and facilitate reuse.
While we certainly cannot ignore the logical dependencies, for large systems we also need to manage the phsyical dependencies between the deployable units. In Java, these units are .jar files. Even for large systems, there usually exists a manageable number of .jar files. Understanding the physical dependencies between .jar files helps with maintenance since we'll know the ripple effect of change, and these physical units are also the foundation of reuse.
But anything done manually is open to error, and requires a high degree of discipline to continually manage. Since I program mostly in Java, I've been very keen on managing and enforcing these physical dependencies. Recently, I've been working on two projects to help manage physical dependencies. The first is JarAnalyzer, a utility that can analyze a directory of .jar files, and report on the physical dependencies between each. It's certainly in it's infancy, and I have plans to continue it's development, but I also think it's stable enough to share. The second is a pattern language to help manage physical dependencies, which can be found at Extensible Java. I'm interested in your feedback on either, and am also very interested in hearing how (and if) you manage your physical dependencies.
On my current project, the Ant script compiles different package sets of the codebase separately to enforce key dependency rules at compile-time.
The main reason behind it is that a substantial set of functionality was being added to an existing application because it was important to have an integrated appearance, but there was also a mandate that the new functionality be sufficiently decoupled that it could be used to construct a stand-alone application at some point in the future.
We needed tight integration and loose coupling between what were in many ways two applications.
Rather than leave it to chance, I defined the dependency rules (both module to module and module to third party library) for the modules and made sure that if they were broken, the automated build would catch it (since it shows up as a compile error). That way, all the developers have a safety net to remind them of the boundaries we're trying to maintain.
Calling jar files "physical" seems to be going a bit far. Maybe you've spent too much time behind the screen? Last I looked out the window there is still a *real* physical world out there, you know? :-)
You've hit on one of the patterns that I've been working on called a LevelizedBuild (http://foundations.extensiblejava.com/pmwiki/pmwiki.php/Heuristics/LevelizedBuild). It's a great way to enforce your dependencies and ensure that no undesirable dependencies creep in. On top of that, we also create a TestComponent that verifies the individual .jar files being built.
Intermodule dependencies is what you're working with.
I've found that large organizations generally need to treat all depended-on modules as third party modules and thus these modules have their own release cycles and version numbers. IOW, this is a release management issue rather than a developer issue.
I was thinking the same thing. I am always amused when people talk of things like a "virtual file" vs. "physical file." I think you can just throw out the "physical" modifier and no meaning is lost.
In this case Todd's suggestion of "Intermodule dependencies" works. Furthermore, I'd like to congratulate you Todd, on not suggesting "Cargo Cult Dependencies" ;-)
>>Intermodule dependencies is what you're working with.
I've thought about module, but like many things, I thought it was a bit too abstract. Is a module a Java package, a logical grouping of code, a .jar file?
>>I've found that large organizations generally need to >>treat all depended-on modules as third party modules and >>thus these modules have their own release cycles and >>version numbers. IOW, this is a release management issue >>rather than a developer issue.
There certainly are version and release implications associated with these "modules". However, developers still have to develop them, and think about the dependencies between them. So it's also a developer issue.
This problem is a real pain. I have gotten queesy when going to clients who have a million projects with their own lib directory with hundreds of jar files. I don't know how they manage it.
I like the maven style where I declare my relationships, and the jar files are found for me.
If a build on a project is getting large, I simply seperate the build, and have multiple projects. Each project now has a binary relationship, just like it does to any other physical dependency (e.g. this is useful if running your tests takes all year, or people are on eachothers toes too much, etc etc).
>I've thought about module, but like many things, I thought it was a bit too abstract. Is a module a Java package, a logical grouping of code, a .jar file?
Its a "clump" of code. Package it anyway you like.
Most commonly, its code not under your direct control.
In my painfully acquired experience - modules must be strictly organized in a directed acyclic graph. No mutual dependencies. Mutual dependencies imply unnatural splits in responsibilities and probably that boundary should be rethought.
When building a module, you must only consider what you depend upon. Give no consideration to those that depend upon you as knowledge of them will inevitably pollute your code.
A given module release is released against a version set of depended-on modules. Thus, if module C 1.1 is developed against modules A 1.5 and B 2.3, then A 1.5, B 2.3 is implied when you say you depend on C 1.1.
Strict adherence to this philosophy eliminates a multitude of pain.
>>Its a "clump" of code. Package it anyway you like. We need something a bit more concrete than just a "clump." That something is a .jar file.
>>Most commonly, its code not under your direct control. And that's when you want to peer inside to find out what it's dependencies are. But in any custom development effort, you'll also be creating physical entities that are under your control. First, you have to give consideration to the relationships, which is something I find not a lot of developers are doing. Second, it's helpful if you have some guidelines that help you massage the relationships, promote testability, and increase reusability.
>>In my painfully acquired experience - modules must be >>strictly organized in a directed acyclic graph. Yep. It's called AcyclicRelationships
>>When building a module, you must only consider what you >>depend upon. Yep. It's called LevelizedBuild.
And there are a lot more patterns that can be helpful. Check them out when you have a chance.
I was also a little surprised about the use of the word physical -- when I saw the title of article (before reading it), I was expecting somebody from a manufacturing industry background making a comparison between design of physical things and design of software.
It seems like the main reason dependencies get out of hand in software is that most of tools used for implementation (even tools that support automatic refactoring) are still just manipulating text files without visualizing the structure in a way that makes the dependency graph apparent.
On the other hand, if you're designing something like the engine of the car, the real physical dependencies: matching sizes and positions of parts that plug/screw into each other or the positioning of components that need to align (like wheels connected by a belt) would be more apparent because the structure is something very visible and tangible.
In software, it's much easier for a stray dependency to sneak in because the problem isn't so obvious.
(there are tools in this space, but they aren't used nearly as much as the text-oriented kind and those I've used have serious limitations in terms of actually doing things instead of just looking at them)
>We need something a bit more concrete than just a "clump." That something is a .jar file.
If you want to limit your world to Java - that's just ducky. OTOH, schemas have versions too, so do other resources like shared libs, your vm, OS, firmware, CPU, etc. It all matters. Saying its a jar file is taking a pretty limited view.
>that's when you want to peer inside to find out what it's dependencies are.
One would hope they are listed in the release notes. How else could you expect to make it work?
>First, you have to give consideration to the relationships, which is something I find not a lot of developers are doing.
On a large project, I would say this is the architect's job. Developers write code - architects set the structure and context. That makes it more of a management issue than a code issue.
>And there are a lot more patterns that can be helpful. Check them out when you have a chance.
I did a quick search of these "patterns" you mention. Google finds a definition in exactly one place out of 8,058,044,651 web pages. I don't really think they are patterns so much as basic industry practice. Seems a bit like trying to patent the process for making ice.
>>If you want to limit your world to Java... Right now, I am limiting my discussion to only Java. The physical entities (or modules) that we program we must design and we must understand. While the ideas are not limited to Java, using Java and it's constructs for the sake of example helps make the discussion a bit more clear through the use of concrete examples (IMO).
>>One would hope they are listed in the release notes... And if these "release notes" could be generated automatically, we'd be certain that they are correct and up-to-date.
>>On a large project, I would say this is the architect's >>job. Developers write code - architects set the structure >>and context I really don't like the separation we tend to make between developer and architect. Quite simply, I think there are more and less experienced developers. Those with more experience tend to label themselves architects, I guess. However, if we can make lesser experienced developers more aware of the physical and logical design issues, the resulting code will be more robust and consistent. A great architecture doesn't result in great code and great code doesn't mean great architecture. A great system has to have both.
>>I did a quick search of these "patterns" you mention. >>Google finds a definition in exactly one place out of >>8,058,044,651 web pages. I don't really think they are >>patterns so much as basic industry practice. If they're basic industry practice, a search should have returned a higher hit count. Maybe that answers my question and we aren't giving adequate consideration to our physical dependencies.
Flat View: This topic has 19 replies
on 2 pages
[
12
|
»
]
Previous Topic
