Integration best practices – Service robustness

My previous blog post on integration best practices gave some general thoughts on service development. When designing services one of the most important parts is resilience of your services and make sure that failure of one services doesn’t collapse your entire world.
There are several key features of a service which combined determine the fault tolerance of your service. In this blogpost I will explain which features contribute to creating better services.  Continue reading

External Resources Are Unreliable

Developing software is one thing, keeping it up and running in production is a whole different challenge. When your software is running in production it will be used in ways you never thought about, and it will behave in ways you’ve never expected it to behave. A common culprit of problems in production is the use of external resources.

What are external resources? Everything that happens outside of the safety of your runtime environment, e.g.:

  • Database connections
  • Consumed webservices (whether it’s REST or SOAP)
  • Inter-process communication
  • Network access in general

Treat external resources as slow and unreliable: it should return well formatted XML within 2 seconds, but it might decide to return the latest The Avengers sequel at 50kb/s which takes 5 hours to receive. It seems like an unrealistic scenario but sooner or later, it will happen. To prove this point, a few examples I’ve seen through the years:

  • Time-out on a database connection (network problem)
  • Broken NFS-mount (after 30 minutes) thanks to a misconfigured firewall
  • HTTP contract is not honoured:
    • Content-Type says ‘application/json’ but it returns plain HTML
    • Content-Length is incorrect leading to half-received responses
  • Incorrect XML entities: Microsoft Office generates invalid XML entities in PDF metadata leading to unparsable XML
  • An external webservice responding with 10kb/s leading to time-outs and rolled back database transaction on our end
  • An external webservice honouring only the second (even) request
  • An external webservice crashing on parallel requests (we had to synchronise access on our end)
  • A VPN shutting down every night after a few hours of inactivity

There are some grey areas like file access; a local disk could be fine. But hey, it might be an NFS-mount (Network File System) which makes it unreliable since again, networks are unreliable (even though Wikipedia says TCP provides ‘reliable transport’, but we know better). In the end it pays to be protective; don’t trust stuff from the outside.

Much of this awareness has been acquired through my personal bible of software development; Release It!, by Michael T. Nygard. Anyone writing software which is meant to go to production (which probably most of us do) should have read it.

Knowledge-driven mobile forms

Wouldn’t it be great if our mobile data collection would be rule-driven to deliver exactly what is needed at the right time? Think of a case where a building inspector would receive dynamic forms on his device that are specific to the context he’s in and to the forms already filled in on-site. This would greatly increase efficiency.

Continue reading

Messaging with Be Informed 3.11 and a Full javascript stack

This blogpost is about creating an integration concept with the newest Be Informed version (3.11) and a modern application stack, including React and node.js.

Be Informed is a business process platform which allows for modelling an application instead of coding it, see their website for more information. React is an javascript framework for building user interfaces created by Facebook. Node.js is a javascript application platform. Combining the two works perfectly.

Continue reading

A closer look at our Confluence Add-ons

About Confluence

Collaboration, sharing, discussions, documents, knowledge, wiki, intranet. These are some of the keywords people use when talking about Confluence. Confluence started out as a wiki-like system but it goes a lot further than that these days. We’ve seen it (or installed it ourselves) in many different forms, like (social) intranets, public websites, knowledge bases or just a place to store documents. With Confluence it is easy to create content, to share it, to find it and to breathe life into it. Besides that, it’s easy to customize or extend Confluence and it’s openness allows you to do just about anything you like with it.

Continue reading