Building an IoT / M2M Platform

Sharing some thoughts and philosophies that have gone in towards architecting Osmosis platform.

  • Device data capturing – this is perhaps the most important piece of any IoT system. How effectively and securely we can get the sensor data on to the cloud determines the success and failure of the platform. Some of the key guidelines we have followed here are

o    Minimal footprint – Since M2M data is small and frequent, it is very important that we have minimum overhead when getting the data from the sensor network. We have therefore defined our REST protocols to have bare bone headers while not losing on the context and security. We are also building interfaces to support TCP / UDP as well as MQTT and other communication protocols to leverage some of their inbuilt strengths.

o    Failover – Quality of Service is another very important aspect. Making sure that the sensor information is tracked and delivered effectively, is quite critical as some of the applications might have high sensitivity and losing that information can have serious implications. We have built our system to make sure that each piece of information is captured and necessary triggers activated in case there is a delay or loss through the network. This is achieved through well defined response codes as well as support for “heartbeats”

o    Security – A lot has been said about the importance of security and IoT and the lack of it. We have seen numerous instances of security breaches causing some major problems in early stage IoT deployments. While the standards around security are still evolving we have taken a simple but fool proof approach so that no one can impersonate or sniff on the data being sent from the sensor networks. The approach is to achieve this through our custom security protocols while there is some standard that emerges. Thereby making sure that there is no compromise on the customer’s data security

o    Interoperability – While a lot of the existing platforms claim that they are truly interoperable and they can take data from any or every device in the world. The reality is that given the state of art and the state of standardization, it is impossible to achieve it “Out of the the Box”. Here again, we have taken a simple approach of keeping the architecture modular and simple. So that the effort and time required to interface some other devices and platforms is limited to a couple of days if not a few hours while not trying to code of each and ever protocol that exists in the world. Integrators can actually use our APIs and SDK for writing their own handlers and upload to give them full flexibility to build their devices and IoT systems that suit their needs

o    Flexibility – We understand that while building IoT systems, the data that comes from the sensor network can change multiple times and each domain has their own mechanism for grouping the devices / sensors and sensor data. In a sense, the data coming from sensor networks is truly “unstructured”. We have therefore defined our APIs and protocols to give a full flexibility to the system developers and integrators to generate their own hierarchies and send data. Using our state of art rules engine, you can then create your own business process workflows and triggers to notify and actionize specific events or trends.

o    Business rules development and management – Being able to act at real time on data coming from the wireless sensor networks is the crux of any IoT deployment. Given the importance of this, we have put a lot of effort in building a flexible and scalable business rules management system that acts on combination of device and system configuration and sensor data. Users can build simple and complex rules and alarms that can be put into the system in real time and modified as needed to “act” on the real time information and trigger system defined and custom actions. This, we believe is one of the most important design pieces of our system and makes it hugely useful.

  • Device management – With multitude of devices and across different domains and verticals being able to administer and manage them effectively plays a big role in the success as well as Total Cost of Ownership – TCO of the IoT deployment. For that reason, we have invested in a very scalable but simple device management system that does not assume a particular structure or approach. The user interface is also search driven and devices can be identified, grouped and searched through custom meta tags. The customer has full flexibility to define these meta tags, attach them to group of devices and create views and actions based on these groupings. This makes the device management and workflow a fun and easy activity. The entire UI is built on responsive and web 2.0 concepts and technologies that ensures that the simplest and most complex of use cases can be adopted and actors can perform tasks on the go.


  • Analytics and Reporting – Some of the basic concepts we have adopted for reporting and analytics is that there is no report that fits all customers. Each customer would have a unique requirement that would not fit in the model that the analysts would have thought about. At the same time, having simple and uncomplicated insights should come out of the box. Using this philosophy, we have adopted a two pronged strategy.

o    One is a standard customer dashboard that gives specific insights in as visual manner as possible. These are based on the type of device.

o    The other is to build necessary infrastructure to quickly churn out custom reports based on customer needs. This infrastructure plugs into both the structured as well as unstructured data and provides ability to apply domain specific implementation to get the necessary insights.

o    While that is all good, we also understand that our customers will have a number of other systems and ERPs in their ecosphere. Therefore all our data is accessible through intuitive REST API that can be used by the integrators to extract information from the Osmosis platform and integrate with other systems.

  • Cost – As I had mentioned in a previous blog on outsourcing and IoT, costs and ROI will start playing a very important role in the adoption of IoT as we get on to the serious side of the business. Keeping that in view, we have made sure that the overall cost of the platform is maintained at the bare minimum without compromising on the security or scalability or resilience of the system. The impact on cost has been a constant thread that has governed our decisions on technology, design and usability of the system. And it is not just the cost of implementing and IoT solution but also the long term cost of ownership. Given our small, lean and agile approach, and the distinct advantage of being in Hyderabad, India we are confident that the platform we have built can be really competitive on cost front.

To sum it up, it has been an interesting learning curve building the Osmosis platform and are happy with what has come out. Its still under evolution but we are confident that it has reached a stage where we can reach out to potential customers and start using it for production deployments.



Outsourcing / Off shoring in Internet of Things – Next big opportunity – Part 2

As promised, this is the second part of my view on Outsourcing / Off shoring and IoT.

Since IoT is still in an early adoption stage, there are mostly pilot projects being done in serious business applications. Given the research-oriented nature of these projects and the maturity [or lack of it] of the IoT platforms being used for them, most of the work is getting done in-house or by the platform vendors. As these dimensions are not conducive for outsourcing being successful. Cost is also not a major factor because the finance guys are also getting their financial models in place to ascertain the ROIs on such investments.

When IoT adoption moves to the next phase [after having proven their worth in pilots] to actual deployment across different divisions and across the globe, there will be a huge need of resources across different technologies including platform specific insights. Cost and availability will play an important role in that phase. Outsourcing and IT services firms, need to and are preparing for such eventualities.

There are a few factors to evaluate on how / where should the services organization focus on to prepare for this opportunity.

  • Technologies – IoT is a coming together of multiple technologies namely
  • Hardware / PCB – design and development. Gateway
  • Embedded development – including knowledge of protocols like MQTT, 6LoWPAN, CoAP, XMPP, etc, IoT Gateways play an important role in this
  • RF related development – including ZigBee, Z-Wave, Bluetooth / BLE, etc
  • Cloud development and deployment – Data streaming, complex event processing, etc
  • User interface for managing devices and newer workflows that emerge from these deployments. This includes mobile-based development and development platforms.
  • Big data and analytics – for generating intelligent business insights from the large data coming from the sensor nodes
  • Last but not the least, integration of the IoT systems with the existing enterprise solutions deployed

There are certain areas where existing IT services vendors are quite proficient in and can quickly scale up. Though, other areas will be very new for them and they will have to either tie up with specialized vendors or start building their competence on.

An obvious question here will be whether there will be a need of resources who have knowledge of all these technologies and their interaction or having competence in individual areas would suffice. This question is still out in the open, but my view is that a combination of the two will be needed. This entire system being too new to a majority of the folks, not having deep understanding of how these operate and the challenges around them will not make them very effective. Keeping this in mind, Axelta has recently launched a weekend [2 days] training cum hands on workshop to get the engineers experienced with all aspects of these technologies and their interaction.

  • Domain – I mentioned in my previous blog on this topic, domain will play a more important role than ever in successful deployment of IoT solutions. I had brought about this point in my first blog post IoT – connection between real and virtual world where I had mentioned that vast gap between real and virtual world is getting bridged through IoT. Therefore, people who have worked in the real world are deep subject matter experts in that space will play a very critical role in success of IoT. Which means, that besides the consultants and analysts who have built a reasonable understanding of the domains / verticals through study and consultation, folks who are embedded into the verticals will be needed. That is an opportunity for the industry people but a challenge for IT services organizations
  • Product – IT Services firms have till now depended on just the services. Which has been very linear model and over last decade, there have been a lot of cost pressures on them. There have been many attempts to move up the value chain and build products that can move them away from the linear model. Though, the success in such investments have been very limited. Main reason being that the entire management structure and the overall engine has been geared up for providing IT services. Given that this is an entirely new area and there is a space for multiple new players, IoT throws a very good opportunity to these firms to make the transition. The challenge though being that they still have the same engine that is geared to get more head counts and more billing. A very good option for these firms could be to dig into their large cash piles and buy out some of the IoT start ups in India or in the western world. Give them funding and freedom of operation, nurture them like CISCO has done in the past and like Microsoft Ventures does and then absorb them over next few years

Focusing on product / platform is also important at this point in time because that is where most of the action is happening right now and will happen over next couple of years at least. Till the time a few companies emerge as the leading contenders like SAPs and Oracles and Googles of the world.

  • Managed Services – This has been a trend over last decade which has now picked up momentum. Where industries outsource their entire IT or IT enabled services to outsourcing firms. IoT will need much more of these managed services. It will also enable such outsourcing through larger automation and bridging the gap from the real world. IT services firms that gear themselves up for providing managed services leveraging their or external platforms have a lot to gain from IoT over next 5 to 10 years.

To summarize, IoT and M2M is a brilliant opportunity for IT services firms for ensuring their growth and profitability over next decade but they will need to get their strategy right as well as be ready for changing their proven models to succeed in the IoT outsourcing world of the future.


Working with CouchDB – My experience of moving from Sql to NoSql

I had heard about NoSql and read about it for sometime. It was a fascinating areas but had not stepped in it till the time we had to build an IoT [Internet of Things] platform. We decided to have CouchDB as our data store for the device data and MySql as relational store. The top reasons to chose CouchDB against other document stores like MongoDB, DynamoDB from Amazon and others were its simple architecture and support for Map Reduce as well as cluster support.

Our fist struggle started with trying to make it work on AWS [our cloud deployment platform], as AWS does not provide it out of the box. We deployed it on a windows server machine and had challenges in accessing the http url from the webserver deployment. At the end, it turned out to be a silly issue related to windows firewall blocking http request on the port used by CouchDB.

The biggest challenge has been though change in mindset from thinking sql / relational to nosql. We have got so much used to looking at data in a relational way that it becomes really difficult get meaningful results and visualize information when dealing with NoSql. After struggling for more than couple of weeks, we now have got a hang of it and able to make some reasonably complex map reduce programs that gives us meaningful insights.

As of now, I am pretty impressed with the speed at which CouchDB is able to process the information and the power it provides and am confident that going for CouchDB has been a good choice. Our next struggle will be to generate analytics by extracting information from both MySql and CouchDB and collating them together. I can already see some challenges on that and we had to migrate some of the set up / configuration data from relational to object DB to extract max power of the in built CouchDB engine. Whether that is the right choice only time will tell.

NoSql / Big Data is going to play a big role in successful deployment and application of IoT and for developers who have grown in the relational world, it will be an interesting though tough challenge to make this transition.

We are using NVD3 for visualization of the analytics being drawn from CouchDB and it has been a good experience till date. The learning curve has not been a significant one and we have been able to get simple to medium complex reports easily from the system. We might have to move to D3 for more complex visualization but as of now we are happy with NVD3. More on the decision to go for NVD3 instead of end to end reporting frameworks like jasperreports for a later date.

Outsourcing / Off shoring in Internet of Things – Next big opportunity

Outsourcing today, especially IT outsourcing, is something which is considered a part of any project plan, development lifecycle. It has become a part of IT landscape, adding huge value across the globe. I distinctly remember a time when large numbers of articles were written about how outsourcing will fail when it comes to complex processes etc. All those scenarios failed to deter the outsourcers and today anything and everything (almost) is welcome to be outsourced.

As Business in IoT starts to boom and comes out of labs and R&D centers, how can outsourcing be very far behind. As Internet of Things has to do with both physical devices and software components with a decent sprinkling of networks, communication protocols and clouds thrown in, outsourcing will be challenging to begin with but will surely happen. I believe outsourcing in IoT to reach similar levels as that of traditional IT program over the next 3-4 years. Already beginnings are being made around the world, especially in India, to make that happen. Most Indian IT services companies have started labs/ R&D centers in their campuses focusing on developing these skills.

Though, there is a significant difference in what tradition software/BP outsourcing and IoT outsourcing will entail, especially on development side.  As there are hardware components and intense domain skills involved in developing devices, analysing the data produced, I believe the smaller Indian IT companies, with specific domain and technical skills will gain a good chunk of this market. This is the chance that these small companies should utilise to grow big.

There are 3 distinct areas in which companies need to focus to get ahead of the curve and get outsourcing deals.

  1. There are several domains in which capabilities need to be developed. IT companies will have to hire agriculturists, doctors, radiologists, scientists, environmentalist and meteorologist etc. to be able to get IOT deals. Large numbers of opportunities are going to open up in water management, security, environment, power, transportation, logistics and healthcare.  Companies which have deep skills in such areas will be having distinct advantage to get outsourcing deals. Analytical skills w.r.t technology (Big data) as well as domain could make a real difference and get you deals.
  1. New technologies, protocols, hardware’s need to be mastered. Protocol’s like MQTT, IPv6 (6LoWPAN) , ZigBee, GSM, RF need to be mastered. Companies, start-ups which are developing their own devices, gateways will be in a good shape to get outsourcing deals as they would be in the groove. Start-ups which are developing their own low cost solutions like device management platform  can easily get into integration related outsourcing deals.  As large number of companies are being set up by domain specific people around the world, Indian companies with deep tech skills will be able to get the deals to actually develop the software, hardware components needed. Getting to Customize solutions for particular business needs and catering to different markets might also be a good business model.
  1. Traditional businesses like call centres will also get a chunk of business in catering to monitoring devices, alarm handling, processing data, device selling, device on-boarding etc. As the number of connected devices continues to explode, equivalent rise in such business to outsourcing companies will happen. IOT trained staff will be key to get such deals. Large Indian BPO/Call centre companies will have a distinct advantage in getting such deals due to their scale and reach.

The Golden period of outsourcing is yet to come as IoT in itself will present a huge outsourcing opportunity globally. It has already started and trickle is all set to become a deluge in coming years.

China has already taken a lead in both building the electronic components and actual adoption in IoT, but an interesting development is newer product development companies / start ups coming in India that are focusing on product design and innovation. Likes of Gecko are a case in point. I see innovation off-shoring besides the typical cost arbitrage as the next big opportunity for Indian companies as there is a huge runway available and the innovation talent available here can play a significant role in future. More about this in my next post.