Cost of Warehousing

Despite the utility of data warehousing, the technology traditionally had a high total cost of ownership, including direct and indirect costs. The total cost include the costs of software, hardware and personnel, all of which must be addressed before a business can begin analyzing the first row of data.

Software Costs

For the software needed to run a data warehouse, licenses must be purchased for both end-user analytics tools and server software, along with any add-ons or optional packages, on the basis of the company’s need. Even if the initial cost is only a few thousand dollars, it can quickly increase from there. The power of insight that a data warehouse can provide is not instant; in fact, a business might have to run through several terabytes of data before it can realize the value of the data. As expected, the volume of data to be examined will increase in size gradually over time, but the costs of the enterprise data warehouse can grow exponentially.

Hardware Costs

Acquiring the powerful hardware required to run complex queries can be a very expensive affair. A data warehouse needs servers to run software and huge data centre space to store the servers also storage for the data to be examined is required. A high-speed computer network to access the data warehouse is also needed. Redundant power to ensure maximum uptime is also needed. Not only are all these assets required up front before you even begin data analysis, but they will incur future costs for the expected growth of the data warehouse. Additional storage devices, servers and network hardware will be needed for growth.

People Costs

The collection of software and hardware that makes up a data warehouse does not run itself.  Trained human effort is needed for this purpose. Consultants and contractors may be required to assemble and maintain the data warehouse environment, while the enterprise’s internal IT department and regular users must be trained to effectively use this expensive investment. Just as software and hardware cost money, so do the people who will end up constructing and using it.

Cost of Data Warehouse on the Cloud

In the past, building a data warehouse required expensive hardware, software licenses, data integration, and consulting to make it all work together. Today, with the advent of SaaS products and cloud software delivery, it’s much easier, faster, and cheaper to spin up a data warehouse from scratch. Here’s how you do it for as little of each as possible:

1.    Choose your cloud data warehouse technology. There are three good choices today: Amazon Redshift, Google BigQuery, and Microsoft Azure SQL Data Warehouse. Each of these is fast to provision (think 10 minutes, not 10 months), available for a low monthly fee (typically starting at around $200 / month), and extremely performant & scalable. Note that when you buy a cloud data warehouse, there’s no hardware to manage: once you choose your provider, all of the provisioning details are handled for you.

2.  Choose your data integration technology. There are several good choices today: RJMetrics, FlyData, and Fivetran are all available for less than $1,000 / month and can be set up quickly and easily. Each of these services makes it simple to connect to many different data sources and stream that data into your data warehouse. RJMetrics, in particular, makes this process fast: our process is completely self-serve and can be completed in five minutes.

With the combination of these services, it’s possible to get a data warehouse up and running for around $200 / month, and within about 30 minutes. These costs increase as your number of data sources and dataset size grow. This is a dramatic decrease in cost vs. ten years ago when a data warehouse cost hundreds of thousands to millions of dollars to deploy. This decrease in cost is causing many more organizations to deploy data warehouse technology.