|
| TooMuchMagic |
UserPreferences |
| JicarillaWiki | FrontPage | Overview | Articles | GetStarted | Sourceforge Project Page | RecentChanges |
This wiki is in 'slumber' mode, just like its associated sourceforge project. Edits are disabled, the content is potentially stale and is not maintained. That said, it contains some really useful stuff still. Enjoy!
This work is licensed under a ![[WWW]](/oldwikiskin/img/moin-www.gif){kind=small}
Creative Commons License.
Programmers are, generally, smart people. Also, generally, they try to solve the general problem rather than the specific one. This leads to the holy grail: reusable software. Not reusable in the sense that you can store it on a hard drive and type a new letter with it the next day, but reusable in the sense that their software will be generally usable in a broad scope of software products. This will, generally, mean that there is less code for them to write in subsequent software projects. Since programmers are generally very lazy, this is generally considered a good thing.
Generally, the complex minds of smart programmers result in complex solutions. Also, generally, complex solutions will be slower than simple counterparts, will be more prone to containing bugs, will be more difficult to debug, more difficult to maintain, more difficult to explain, and more difficult to learn. Which are, generally, bad things.
(you may have noticed my slight overuse of the word "general" in the above two paragraphs. Do not worry: that's intentional, and typical of the programming style this article ridicules. Other "magic words" to be aware of include "dynamic", "pluggable", "runtime", "bytecode", "scriptable", "fully configurable" and more).
This paper assumes you are a smart java programmer with lots of knowledge about smart java features. If you are not a java programmer, but very smart anyway, it may still make some sense to you, or it may not. Let me know.
"Magic" is what wizards use to make really impressive (to lesser minds, seemingly impossible) things happen, often automatically and transparently, sometimes distributed, and usually without asking about it first.
"Magic" is anything that makes a piece of software miraculously do things that otherwise seemed impossible (or at least very difficult).
"Magic" is inherently vague, and difficult to define precisely. What we can do is show examples.
Let us enumerate several magic tricks in common use throughout the java world.
Anything that uses java.lang.Class and the associated java.lang.reflect package. While a simple this.getClass().getName() may not be very shocking, the consistent use of reflection for decoupling classes and applications can result in mindboggling complexity. The use of reflection is often unavoidable in ugly beasts such as "generic application frameworks"; reflection is what allows them to be generic.
In common use ever since jdk 1.3 started providing built-in support (but before that available through bytecode engineering) dynamic proxying is a true black art. Using dynamic proxies, one can replace an instance with just about anything else. This allows for all kinds of dynamic features, like pluggable, runtime-configurable, xml-policy-based instrumentation, component hot-swapping, and a whole lot more. It also makes it utterly impossible to be sure the instance you're currently holding, is, in fact, an instance. It results in quite significant overhead (ranging from a factor of 10 to a factor of 10000000 depending on what kind of magic stuff your dynamic proxy does for you), looks quite ugly in a debugger, and can be used to implement all kinds of other magic.
Libraries such as BCEL and CGLIB can be used to read and write java classfiles directly. That's great fun of course, as we can circumvent or rewrite many of the java language rules (ugly things like access modifiers really are no sweat when you're doing bytecode hacking). Combined with smart classloaders, we can even defer all this rewriting until runtime.
Let's face it, classloaders are the nightmare of the java world, analogous to the DLL hell of the windows world. It has taken many smart programmers many years and five product redesigns to get classloading inside Jakarta Tomcat figured out. And all Tomcat does is run servlets. Anything more generic than Tomcat has a classloader architecture that's even more difficult to get right.
One way to circumvent all those issues is by linearly transforming them into something else, by providing custom classloaders. Classloaders that aggregate, that reload classes, that support bytecode engineering and modification, that have special provisions for dynamic download and upgrading of support libraries...I've seen them all. Combine a few of them together and it becomes utterly impossible to tell what version of what classfiles from what locations on what disks are linked to one another. Version mismatches are almost guaranteed!
Aspect-Oriented Programming, AOP for short, is not magic in and of itself. Its (part of) a programming methodology. Unfortunately, java doesn't have AOP support. It is easily added though, provided one uses enough wizardry. AOP can be implemented using interceptors, dynamic proxying, and similar features (ie, Nanning), using byte code engineering (ie, AspectWerkz), using a precompiler (commonly done for instrumentation purposes, ie clover) or even using a language extension and appropriate compiler (ie, AspectJ).
An idea that gathered steam by being a core feature of the .Net platform, Attributes are a way for associating arbitrary data (like text strings or even arbitrary java objects) with arbitrary parts of a classfile (like the class itself, some field, or the fifth argument to the method of the base class).
Of course, java doesn't have built-in support for attributes either. It does have javadoc tags, which can and are abused to model attributes. Libraries for doing so include XDoclet, QDox, jakarta-commons-attributes, and others. Of course, each library does it not just a little, but often quite differently.
But how do we get the javadoc tags into a place you can do something with it? Well, we'll have to add a compiler, a storage format and a runtime reflection-style support library. In some applications, multiple attribute libraries are used, so that means multiple compilers, metadata files, and runtime libraries as well.
Attributes are all the rage today, especially when combined with AOP, some form of xml and some kind of smart containment framework.
Of course, the use of proxies, AOP, attributes, and more, may not be enough to satisfy our dynamic needs. In that case, we can always design our own pseudo-language that does the things we cannot figure out how to do in java. Common examples include xml-based configuration tools (like the XML version of Jelly or XMLBeans) and so-called declarative-style programming (ie, Avalon-Merlin).
As an alternative to a special-purpose language, one also has the option of using a generic scripting language. These are often interpreted at runtime, most always lack strong typing or access checks, and have at least as many pecularities as java itself. Furthermore, their java-based implementations always seem to lag behind their native counterparts. Examples of popular java-enabled scripting languages include Jython, BeanShell, Groovy, and more.
Surging on after adding metadata and scriptable configuration, we can move away from 'basic' java and start using some kind of framework. These frameworks usually require some application restructuring to get into position, and then slowly saturate every one of your classes until it is no longer usable without the framework. What is often seen as a big advantage to these frameworks is that they include a whole lot of magic for you to use "out of the box", in a "uniform, general way". Popular frameworks include J2EE, WebWork, Eclipse Platform, Avalon, HiveMind, Spring, and others.
There are many other kinds of magic besides those listed here. If you know of an especially good example, you're encouraged to add it.
Many programmers have a tendency to be Too Smart. Being Smart, they introduce various kinds of Magic into their software. This will usually lead to a software design that is more generic, dynamic and flexible than is needed, has more features than is needed, and is hence much more complex than is needed.
(additions welcome! This is a wiki!)
The only way to really avoid magic is to be constantly on-guard against it. But here's some tips anyway:
Of course, if you absolutely have to add some magic:
Programmers are smart. We can think of many smart things. We have a tendency to overuse smart things, making software too complex. We really shouldn't.
Comment from Leo Sutic (2003-11-25): Since I wrote commons-attributes, I guess I should own up to my sins here. Truth is, I'm not using it (commons-attributes) myself. Why? Because it is a non-standard way of doing things. The gain of using them isn't big enough compared to the risk that I'll look at my attribute-enabled code later and wonder "ok, how did all this work?". That said, I still think attributes are really really nice - no insights I share here has so far kept me from proposing one attribute-based scheme after another at dev@avalon. But we need (a) a Java-wide standard, widely adopted way of using them and (b) at least some "best practices". Face it, any attribute-based frameworks you have today will be bleeding edge, and risk becoming obsolete with JDK1.5. So you get the instability of a technology that has just been born, coupled with the insecurity of a technology that's about to die.
Comment from Leo Simons (2003-11-26): Some interesting discussion regarding this essay to read on avalon-dev: http://nagoya.apache.org/eyebrowse/BrowseList?listName=dev@avalon.apache.org&by=thread&from=545942
Comment from Leo Simons (2004-04-03): Berin Loritsch has written an essay about magic and convenience: http://d-haven.org/modules/news/article.php?storyid=17
