Where Does Data Management Software Belong?
It is interesting to see how some of the developments in the IT space are governed by intradepartmental realities. I see this most pronounced in the storage team’s perspective versus the rest of the IT team. Storage teams are exceptionally conservative by nature. This makes perfect sense – servers can be rebooted, applications can be reinstalled, hardware can be replaced – but if data is lost there are no easy solutions.
Application teams are aware of the risk of data loss, but are much more concerned with the day-to-day realities of managing an application – providing a valuable service, adding new features, and scaling performance. This difference in focus can lead to very real differences in viewpoints and a bit of mutual distrust between the application and storage teams.
The best example of this difference is where the storage team classifies storage array controllers as hardware solutions, when in reality most are just a predefined server configuration running data management software with disk shelves attached. Although the features provided by the controllers are important (like replication, deduplication, snapshot, file services, backup, etc.) they are inherently just software packages.
More and more of the major enterprise applications are now building in the same feature set traditionally found in the storage domain. With Oracle there is RMAN for backups, ASM for storage management, Data Guard for replication, and the Flash Recovery Area for snapshots. In Microsoft SQL Server there is database mirroring for synchronous or asynchronous replication and database snapshots. If you follow VMware’s updates, it is easy to see they are rapidly folding in more storage features with every release (as an aside, one of the amazing successes of VMware is in making managing software feel like managing hardware). Since solutions at the application level can be aware of the layout of the data, some of these features can be implemented much more efficiently. A prime example is replication, where database level replication tools tie into the transaction logging mechanism and send only the logs rather than blindly replicating all of the data.
The biggest hurdle that I have seen at customer sites looking to leverage these application level storage features is the resistance from the storage team in ceding control, either due to lack of confidence in the application team’s ability to manage data, internal requirements, or turf protection. One of the most surprising reasons I have seen PCIe SSD solutions selected by application architects is to avoid even having these internal discussions!
As the applications that support important business processes continue to grow in their data management sophistication there will be more discussions on where the data management and protection belong. Should they be bundled with the storage media? Bundled within the application? Or should they be purchased as separate software – perhaps as virtual appliances?
Where do you think these services will be located going forward? Let me know below in the comments.