Friday, March 18, 2011

Never write you own Message Queuing Framework

Software development is fun. Especially true if you are developing an Enterprise Application. There will be many challenges, world class challenges! High throughput, efficiency and low roll out cost and many sophisticated and complex business logic and workflow that goes beyond what you have learned and practiced in the past. If you are not ready for them, you better look elsewhere.

Very common in enterprise solutions to have distributed application support that require high response time and yet reliable communication and services to deliver the requests. Secured, durable and reliable messages encapsulating business operational messages that get processed at the distributed end, usually the server tiers, are common. These are normally done with many ready made Messaging Framework, such as MSMQ, AppFrabic Service Bus, Enterprise Service Bus, WebSphere MQ. There are also a couple opensource messaging framework like RabbitMQ, ZeroMQ, AMQP.

We have more than enough varieties in MQ selection however, there always some techy developers or architects would like to invent one them-self within their own team. Often, the discussion is surrounding the fact of freedom and lightweight modules that suits best into the solutions, teams or the company. It is always good to have the experience of developing such framework in your resume, but that is often bad for the team, and company.

Never invent one there is a ready made solution.
It takes several years for a framework to be matured and tested against many genuine requirements within the businesses and made many fixes, enhancements, simplification, optimization, customization, usability and security tests while accomplishing the needs of being orderable, durable, reliable and securable. Developing one yourself is a short sighted move. You are ignoring the facts that a framework takes millions of hour and efforts to be matured and testified. You are thinking that one man view is always better than the effort of a group of people. You might be right for the first few months, but as your solution becoming more matured and many more challenges coming onboard, you will quickly find that you were shooting at your own feet for the decision you made.

You often find that your invented MQ framework does not scale as good. Has many shortcomings in throttling, failed-safe, and security. Does not have a dead letter policy supports. Loosing the sight of dynamic expansion of queues. Priority overwrites and poison message control. Worst when you have no idea to maintain/support disastrous recovery. Worst when you find your own MQ framework does not support distributed transaction, transaction flow (distributed but integrated transaction model) and does not log properly and not autonomous. Your solution may ended up just a distributed but serialized messaging gateway that processes messages from different sources in a single queued fashion and starts to suffers when a business starts growing large.

Other than business factors consideration, you owe your team or your company at large by requiring many innocent teammates to endlessly support and fixing defects that were common but long fixed in the matured framework. You start to find yourself heading to where other frameworks is doing by doing more work and getting less throughput. You burn many pricey man-hours for the company to get less. You tricked your team members with more long working hour to support and defect fixing to deliver sub-standard solution to the users. You make your product less competitive to the competitors and you shifted the team focus from product development to platform components that are no better but long available. You sincerely given your competitors a chance to close up your leading role or gap in the industry. You put your team to loose enthusiasm. You bid your company and the team to the losing end to only build a good resume in you.

From the business, team, and product perspective, there are no reason for one to develop your own messaging framework. Unless you would want to compete with the giant in the market, this is not your cup of tea. Just stay away from losing sight, you should and must maintain your business focus and continuing your industry leading role.

Final words, don't reinvent the wheel if this is not your strength and does not record in your business roadmap. Do what you do best and leave the worries to the experts.

No comments:

Post a Comment