Growing a Single Giant Fruit is a Dumb Thing for a Tree to Do

If a fruit tree put all its effort into generating a single giant fruit in its entire life time, then I’d consider this a complete failure. Nature has learnt that producing offspring is a numbers game when there is luck and odds involved. So the tree would do a bad job at preserving its legacy. The few pips would likely be ignored, lost, damaged or remain too close to home. The tree itself would have a tough life as well. Putting all effort into a single fruit would mean working against the natural restrictions of its environment. Seasons would have to be worked around, water and other resources would be put into a single vulnerable spot. Carrying a giant fruit would required stronger branches, hence a change to a design that took millions of years. (The tree wouldn’t have fun, either, as the chances of pollination would be minimal. But I think I’m at the point now where I should move on from this metaphor.)

What inspired the metaphor was the dealing with self-imposed pressure to achieve big things. Me personally, I get this pressure as a natural consequence of being a creative mind, combined I guess with the self-assessment that all this creativity doesn’t count until I’ve produced at least one artefact that will count as my legacy. Evidently,this pressure comes with the expectation that the artefact would be of considerable size to be noticed by my environment. There are of course many other motivations to try and grow a giant fruit. Many individuals work hard for some form of fame and fortune and media events such as cooking, dancing, singing, travelling, surviving and inventing competitions spark the run for the long way to the top even more.

I did come to realise that what I occasionally envy great achievers for – the step to lock themselves into a room for an entire career and to focus on their oeuvre only –  is indeed an extremely selfish, unnatural and plain dumb move. What I mean are the artists, scientists, politicians, business men etc. that sacrifice bonding with their children, marriages, fun and adventures in order to put all resources into growing a giant fruit.

If you want to achieve something great, aren’t you doing it for yourself ? Who can judge more than yourself what hard yakka and long journey it will have been ? Do you really need to produce something as giant as whatever you think matches the benchmark of a TV Master Chef jury, Pulitzer award, national elections or fat payout ?

I’m making this up but maybe there’s a guy out there who could write Nobel price material if he spent a life working towards it. Instead, he teaches kids the beauty, fun and usefulness that can be found in science. He is a healthy and very effective fruit tree. He is continuously creating generations of positive outcomes in his environment, a legacy that will spread and trigger even more amazing developments. He is (hopefully) keeping enough spare resources to put effort into other aspects of life, which guarantees an ongoing experience for all involved. Oh – and he is less likely to fail in his mission.

My point is that continuously producing small achievements within the immediate environment, and without creating an imbalance in where resources are being spent, is not only more sustainable, but also promotes evolution and eventually has a longer, broader and more positive impact than a single giant fruit that by this stage of my blog is already starting to smell rotten.

Legacy happens via distribution and long term value to some community. It does not guarantee for names to be mentioned in news flashes or history books. But it does turn individual actions into ongoing motion and is likely to be rewarded with a lot of karma points. Most importantly, the rewards are genuine and guaranteed.

Growing a Single Giant Fruit is a Dumb Thing for a Tree to Do

W.H.Y. exactly are we cutting carbon emissions ??

Carbon emissions are one of the factors that are likely to have triggered climate change. Ok, I get that part. But how did we come to the reverse conclusion that simply by reducing carbon emissions we will be able to tackle the complex challenge of climate change ?

With all the effort and finance put into it, I would assume that there is good scientific reasoning behind such strategy. I just fail to find any concise and convincing documentation for it.

Can you help me finding the chain of deduction that leads from “greenhouse gases are causing climate change” to “carbon reductions will lower the negative impact of climate change to human mankind” ?

Here are the explanations I’ve tried so far:

Hypothesis 1: “Carbon caused the green house effect, so reducing emissions will revert it.” Anyone with minimal understanding of so-called non-linear dynamical systems (‘chaos theory’) will have to ditch that hypothesis into the very-very-very-unlikely-bucket. Earth’s climate with its interactions to the rest of the planet is an extremely complicated system and certainly not ‘reversible’ in this sense. There can’t be scientific evidence for this approach as no model could provide such accurate prediction. (I may have to elaborate on this soon).

Hypothesis 2: “Carbon caused the green house effect, so reducing future emissions will make it less bad.” So we have proof that reducing emissions will slow down climate change ? Again, if not via climate models, how else could such statement be deduced ? (And if this was our rationale – why are we hand-balling the problem to our children instead of preparing them ?)

Hypothesis 3: “Carbon reduction is a new market with huge potential“. This one actually sounds logical. But surely this can’t be what’s happening, as various non-profit movements would not be supporting the carbon cutting for such capitalist motivation. Moreover it would leave the question open if this approach does fight climate change or not. (Nevertheless, it would be a pretty clever master plan, I have to admit: re-directing the environmental movement towards generating a new market – even with own stock exchange – and getting the politicians to promote it for free. Why did no one use this business opportunity when acid rain was still talked about ?)

Hypothesis 4: “To be honest, we know that we can’t tackle the climate challenge this way – but it’s a good pretext for improving air quality  and general environmental awareness“. Sounds logical. It’s a scam – but for the better. I approve and move on.

W.H.Y. exactly are we cutting carbon emissions ??

Remote Java Debugging via SSH Tunnel Only Works if Local and Remote Ports are Identical

Yepp. At least that’s what I found.

Summary

Remote debugging a Java application from Eclipse (or other IDE)  is a great tool. Even if the source code is not available and decompilation due to legal restrictions a no-no: IDEs will allow you to step through the code, showing a strack trace and variable values. This is an often underrated tool to figure out what the hack is going on, especially in someone else’s code.

If your Java application is not only running in a different JVM, but on a different host (such as a virtual machine or a cloud instance), then SSH may be the only secure way to connect to the host. The only way I got the combination of SSH and remote debugging working however, is if I create a SSH tunnel where local and remote port are identical. This may be due to exotic network settings or the actual nature of technology – don’t care, don’t care, don’t care.

How To

If you’re not sure how to set it up:

Start your remote Java application with the following extra options:

-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=80

Create the SSH tunnel (Ubuntu example):

sudo ssh username@hostname -L 80:127.0.0.1:80

Then create a remote debug configuration in Eclipse, pointing to localhost and port 80. Evidently, the choice of port 80 is arbitrary, any port that is not taken on both, local and remote host will do. Also note that if the default ssh port 22 is closed on either side, then the SSH server can be configured to listen to e.g. 443 or another open port. In this case, add ‘-p 443’ to the ssh command.

Remote Java Debugging via SSH Tunnel Only Works if Local and Remote Ports are Identical

Let’s Build the Next-Gen Enterprise Integration Framework

Enterprise integration frameworks are based on a single and mandatory data format, such as XML. Adapters are used to enforce format compliance, meaning that information stored in native formats (such as databases and file) must be converted to XML before taking part in any integration process. Content transformation is necessary to solve the integration problem, I agree. But imposing format transformation (as frameworks do) is not justified.

The approach of modeling integration artifacts in a technology-independent, generic way is canonical and valid. The ‘common denominator’ approach leading to concepts such as ‘messages’, ‘adapters’, ‘services’, ‘orchestration’ and ESBs makes sense – as long as it remains conceptual. These entities are suitable to model the gaps that are to be filled when integrating enterprise systems.

There is no reason however, as to why these concepts need to be mapped 1:1 when implementing systems integration. ‘Messages’ for example can be formatted as XML – but also in files, database rows, even datastreams.

Forcing all business data to be physically transformed to and from XML is not necessary.

It is in fact causing significant cost overhead:

  • Facades are required to make existing enterprise systems format compliant (SOAP, WSDL, XSD, …).
  • Projects get stuck in performance issues as out-of-the-box adapters do not match expected volume and latency requirements
  • Data format detours are to be implemented and maintained.

Even worse, the technology is bound to fail when systems integration requires exchange of:

  • Large files
    • used for data exchange with external legacy systems
    • multimedia: movies, MP3s, podcasts
  • Large row sets retrieved from databases
    • search results
    • batch data counting as single transaction
    • geo-coded data
  • Data streams such as
    • live data produced by communications, telemetry, monitoring, …
    • low-latency real-time data
    • large volume data streams, e.g. from/to FTP servers.

Only a technology stack capable of dealing with native formats, while supporting a consistent, abstract representation in its model, will provide an efficient and convincing tool for enterprise systems integration in future systems.

(I could build such technology and several people I know could too. If you happen to be an investor, let’s talk, then we flock it off to Oracle. I’m serious.)

Let’s Build the Next-Gen Enterprise Integration Framework

Runtime Enhancement with JDO and Datanucleus

Now there’s a nice discovery I recently made in the world of JEE development: Datanucleus is a persistence API that does not depend on the underlying persistence mechanism. Meaning you won’t have to change your POJOs if you decide to move from RDBMS to Amazon S3, XML, JSON, MS Excel or any other of the various supported mechanisms. The specifics on the persistence are kept in a configuration file,  entities can be annotated via JDO’s @PersistenceCapable. In my view this absolutely rocks. I will prefer it over JPA from now on.

The way it works is that annotated classes are ‘enhanced’, meaning their bytecode is modified to include all the necessary functionality. This can be done at build time: Ant tasks and a Datanucleus Eclipse plugin are part of the package.

There’s a small catch: I happen to develop JEE applications via Eclipse, with hot deployment to an application server and EJB modules split across different projects. After a while I got annoyed by:

  • Noticeable delays caused by enhancement: with Eclipse set to ‘build automatically’, each recompilation causes the enhancer plugin to run. This is the case even if the class compiled is not an entity. It takes long enough to be noticed by the impatient.
  • Hot deployment failures due to synchronisation issues between enhancer and deployer (Glassfish Eclipse in my case).
  • I’d prefer to have a single process for enhancement, instead of having to update my Ant scripts and using the Eclipse plugin as well.
.

The elegant solution provided by Datanucleus is enhancement at runtime, not build time:

    JDOEnhancer enhancer = JDOHelper.getEnhancer();
    enhancer.setClassLoader(this.getClass().getClassLoader());
    enhancer.setVerbose(true);
    enhancer.addPersistenceUnit("my-persistence-unit");
    enhancer.addClasses(BlueEntity.class.getName());
    // add entities to be enhanced to this list...
    enhancer.enhance();
    Properties properties = new Properties();
    properties.setProperty("javax.jdo.PersistenceManagerFactoryClass",
         "org.datanucleus.jdo.JDOPersistenceManagerFactory");
    properties.setProperty("javax.jdo.option.PersistenceUnitName",
         "my-persistence-unit");
    PersistenceManagerFactory factory =
         JDOHelper.getPersistenceManagerFactory(properties, 
            this.getClass().getClassLoader());
    PersistenceManager persistenceManager =
        factory.getPersistenceManager();
.

Important notes: there’s a few catches:

  • This only works within a web application, for classes (entities) deployed in WEB-INF/classes. This is because the enhancer will attempt to overwrite the original *.class files with the enhanced *.class files. If your application server explodes the EAR into a directory structure, this will succeed. If the original classes were bundled in a *.jar file, then the enhancement will fail on write. Consequently, ejb-jars or utility jars within the EAR can not be enhanced at runtime.
  • Make sure the enhancement process has sufficient file permissions to write into WEB-INF/classes and its content. Sounds obvious if you think of it. Costs hours debugging if you don’t.

Conclusion: There is a way to implement runtime ‘enhancement’ for Datanucleus.  No change to the build process is required. It is limited to the web module of a JEE application, but allows to introduce the beauty of JDO/Datanucleus in the most non-invasive way. Hopefully, the enhancement process will be improved to support enhancement on the fly, without the attempt to overwrite *.class files. This would allow to use JEE structures where entities reside within a separate ejb module…

Runtime Enhancement with JDO and Datanucleus

Every Geek Needs a Pet (Project)

Geeks always want to play with new toys. Fair enough, this is what makes ’em tick. But is the day job the right place for such playtime ? Bad idea, I think, for employer and employee: it changes the focus from the goal to be achieved to the path how to get there. Personal interest starts to interferes with objective decisions, deliverables may be ‘fancy’, but not meet customer expectations. And it can backfire for Geek as well: if playtime doesn’t happen, then frustration will follow.

Sooo – why not play at home !? Create your own perfect world where you can do everything the way you want – with the toys you choose, no time pressure, no distraction.

This may sound rough, especially coming from a software developer himself. But between me and you, let’s be honest for a moment, Geek:

You wouldn’t hire an electrician just because he’s using the latest & greatest tools and certainly wouldn’t pay him to read the manual on the job. You expect him to choose the tool that fits the task best, to know how to use it – and certainly not to play around instead of installing what you are paying for.

For some reason, this expectation hasn’t really made it into IT. Technology choices are very often driven by personal interest, delivery cost include a significant amount of ‘learning on the job’, engineering focus moves from the goal to the path.

Sure, work has to be satisfying and one way Geek achieves this is by trying to get as much playtime with new toys as possible. If he’s a lucky Geek, this aligns with his task, he can play all day, spend paid overtime self-educating and adding skills to his CV. He spends much time at work, manager is happy.

But there’s a few traps (and this is what made me write this blog):

  1. If the technology is given – and a boring one, then Geek will soon get very frustrated. The core of what makes him tick is entirely absent. Geek will become demotivated, frustrated and cynical.
  2. If Geek actually gets to choose his toy technology but then looses interest, he will want to walk off or change again half way through the task. Manager will say no, Geek gets back to point 1.
  3. If Geek enjoys shiny toy excessively, he will become inefficient due to sleep deprivation and lack of focus on what needs to be achieved. The deliverable will be very neat – but not what Mr. Customer asked for. Manager will make sure this doesn’t happen again, Geek is back to point 1 again.

To secure Happy Geek Time, I strongly suggest to start a little pet project at home – if you haven’t already. There’s plenty of freedom, no hassles, total power and limitless opportunity for perfection. Choose to build something that will give you a great sense of achievement once it’s in the box – preferably something that will be used by others and make them happy. Open source collaboration, little tools for your friends and family, prototypes or small freelance jobs to earn extra pocket money…anything beyond the ‘Hello World’ throw away code will do the job.

The payout is massive: your Geekdom will be satisfied enough to survive dreadful days of boring coding on the job. You will acquire new skills and feel the rewards of achievement – even if it’s something small. You will also learn about work discipline, task & time management, development & release processes, pragmatism and maybe even commercial aspects of software development. Sounds scary ? Wait – maybe one day you will even understand what’s driving your boss’ decisions !

Good coding, let me know how you go…

Every Geek Needs a Pet (Project)

Out Smart the Code Monkeys

The IT industry is facing the challenges that come with an established sector competing in a global market. Competition is getting tougher, the ‘did it, done it’ has become a weak argument to secure market position. How can IT companies provide value that differentiates their capabilities from the main stream ? Not as hard, I think, here are a few thoughts…

The IT Industry Faces the Laws of Global Main Stream Markets

It’s no news any more – the IT industry is facing the challenge of a changing world: for decades, most IT consulting and software companies could safely rely on the fact that they were the only kid on the block. Clients were desperate to find someone who could fix their computer, write software, design a web page or make applications and databases talk to each other. IT was a niche industry where some basic experience and a couple of references were enough to make a living.

Then came globalisation, Internet, freelance sites, outsourcing, crowd-sourcing…you know the story. More kids have come on the block and the block has virtually grown into the clouds (pardon my punt). The propeller head backyard shed has turned into a discount shopping centre. Finding a team of ‘code monkeys’ to write a business application is as easy as finding a builder to construct a new office site. Customers have a choice these days – and many have gained enough experience to choose wisely.

So how can your IT company increase its chance for survival in such an overcrowded market ? “Being different” – as many suggest – is certainly a good approach. But it needs to go further than printing groovy business cards, practising early adoption of new technologies and sticking to the rules of an incestuous IT industry. In order to be different, one has to leave the mainstream. Step back and think about how to out smart the competing code monkeys.

This is what I came up with (so far):

Idea #1 – Be Cheaper (by Being More Effective)

Time is money.  Cut cost by reducing human work.

IT costs are mainly labour costs. Offshore competition can cut these cost by 50%. You can counter offer with an extra 25% cut if you decrease the human work necessary to deliver. Building software 4x faster is actually not that hard, here are a few ideas:

  • Use The Best Tools & Team.

Ask a car mechanic – good tools are crucial to get a good job done on time. Ask an elite team – professionals are the secret to fast resolution. Get the best tools and team, don’t be greedy on the wrong place.

Get fast hardware – software ‘build cycles’ will shorten to half the time, more tests can be run, less regression means less risk of schedule blow out. Pay the extra to get mature, well supported, well documented and robust tools. Cheap tools will cause extra work expenses due to faults, functionality gaps, data loss and delays waiting for support.

Pay the 50% extra salary for a talented programmer – he will cut code at 2x the average speed (if not more), effectively saving you time, thus money.

  • Develop Better Tools.

Identify the bottle necks in your delivery process and invest some time in removing them – as early as possible. Continuously extending your toolbox will not only make you more efficient (thus cost-effective) – it will also generate intellectual property. Be pragmatic about this, though, don’t get distracted writing the generic tool that does it all. Restrict yourself to reliably removing the bottle neck, with a design supporting flexibility and extension were needed. Design the solution so it can be reused in other projects – if this will be of value.

  • Automate Repetitive Tasks

Replace repetitive, time-consuming (and frustrating) human tasks with automated processes. Frustration is a big morale killer in IT. Automate the monkey tasks and challenge your team with something that requires human intelligence. Do the maths – compare the time it takes to manually execute a task N times with the time it takes to implement and execute the process. (Be aware that project progress wont’ be linear:  you are taking a time out to build the tools that will carry out the real work at the end.)

  • Build Robots

It’s 2010. Write software that writes software. The geeks will love this task and management will be amazed: robots code very, very fast and don’t eat, nor sleep.

Idea #2 – Migrate to New Markets

Traditional IT markets are densely populated. Move where there is no competition.

Most IT giants are active in tel-co, finance, supply chain, transport, government, … . Why not extend the ‘IT revolution’ to other industries ?

  • Go for Gold

Identify financially strong areas outside the traditional IT client industries. Where’s the honey pot that your competitors ignore ? What are the opportunities in Mining, Tourism, Small Businesses, Farming, Research, Education, Media, Entertainment, Hospitality, Catering, Sports, … ? Be aware of hypes and try to stay outside – hypes act like competition magnets. Instead, get a good picture of long term trends. Find a niche.

  • Do the Homework: Hands-On Research

Potential clients may not be aware of the capabilities of IT. This is why they haven’t called you yet. It is up to you to identify the applications that can be enhanced by or made possible with IT. Get in touch, go on site, sign NDAs, show interest. Think beyond CRM, ERP, location-based mobile apps and similar IT inbreed. What are the existing processes in this market ? Where are the bottle necks, problems, costs, inefficiencies ? What can be automated, how does information flow and how fast does it flow ? Where is the pain and how can IT help ?

Idea #3 – Extend Your Cross-Skills

Strong IT skills make you a good working horse. Crossing them with non-IT skills  makes you a unicorn.

The majority of IT solutions are processing data. Not many are interfacing with controllers and sensors, interacting with machinery and carbon-based lifeforms to the extend possible. Extending your capabilities to include such interaction as part of a business solution creates a rare skill set, hence a large opportunity.

  • Broaden Your Scope

Extend your capabilities to provide end-to-end solutions across several disciplines of engineering. Telemetry, remote controlling, industrial processing, engineering, construction all have the potential to interface with IT. But not many people can see how, as there is not much dialog between these disciplines and geeks. If you can take ownership of an umbrella solution that covers IT and non-IT components, then you have a very unique offer.

  • Leverage of Your Employees Background

What are your employees education and previous experience ? You may find a valuable source of non-IT skills – and networking contacts. Make non-IT skills an important criteria in the hiring process.

Idea #4 – Use your Enemy

Be different by extending your approach, rather than changing it. Complement your unique features with your enemy’s strengths.

Out smarting the code monkeys does not necessarily mean that you have to compete against them on all levels. You have chosen a more efficient approach, changed market and extended your skills set. Use your smarts to delegate as well as execute. Take advantage of all the effort your competitors put into their own work.

  • Delegate to Your Competitors

Avoid getting stuck in the ‘startup’ situation of having one main income stream only. If new opportunities arise, you may find yourself short of resources to successfully generate income streams. Accelerate this growth by sub-contracting to code monkeys. Delegate small pieces of work, with very limited and clear scope. Make sure you do not become dependent on the monkeys. Offload the tiring and frustrating work, keep the intelligence and key decisions with your own employees.

  • Harvest Expert Knowledge

Get experts to temporarily work with your team. Hire them to transfer their skills, rather than just apply them in a dark room. Organise workshops, get the experts to teach your staff by doing project related tasks together.

Out Smart the Code Monkeys