Why Consumption Pricing Doesn't Reward Efficiency

Consumption pricing is cost-effective for prototyping, not scalability. That's why we built Falcon the way we did.

If you are managing enterprise technology stacks, you no doubt are familiar with SaaS, IaaS, and other consumption-based pricing. It can be liberating for some scenarios and a prison in others. We built and license Falcon to break free of the traps in consumption-based pricing — and even the hidden cost traps of open source projects. With consumption pricing, the vendor is incentivized to increase customer usage regardless of efficiency. With capacity pricing, Haevek is incentivized to free up customer resources for innovation.

Consumption pricing is well-suited for early-stage work. You pay for what you use, run experiments when you need to, and stop paying when you stop running. For a team in prototyping mode, that model is genuinely useful, because costs stay proportional to actual activity and you don't pay for what you don't need.

That advantage disappears once workloads go to production and are always on. The ability to dial the meter up and down, which made consumption pricing the right choice during development, is gone. What remains is a recurring cost that increases with data volume and has no guaranteed ceiling.

Consider a data team with a $1 million annual infrastructure budget that can affordably prototype a new analytics pipeline for $10,000. At production scale, that same pipeline may run $200,000 a month. By May, the budget is gone.

Uber's CTO described exactly this dynamic to The Information earlier this year after the company burned through its entire annual AI tooling budget in four months: "The budget I thought I would need is blown away already."

The cloud compute costs that were controllable at low usage become impossible to predict when adoption runs at production scale. Maintaining workloads requires far more budget than planned. Continuing to run at scale eats through money but shutting it down disrupts production users who have come to depend on it.

The flexibility that consumption pricing offered in development costs transforms into overhead that grows as you scale up.

The Vendor Incentive Problem

At production scale, the incentives built into consumption pricing create a windfall for the vendor and work against the people paying for it.

Under consumption pricing, vendors earn less revenue from the customers who create and benefit from efficiency improvements. If their software runs twice as fast, the customer pays half as much in compute time. That is a direct hit to the vendor's revenue.

A vendor in that position has two options. Accept the lower revenue or recapture the loss through pricing adjustments, pushing the aggregate costs back up to approximately where they were before the gain.

This is a structural property of the consumption pricing model, like a team billing by the hour earning more by working slower. Cloud compute and consumption-based data platforms work the same way. When a customer needs more throughput, the vendor's incentive points them toward adding more compute. When a customer needs to cut the bill, every optimization they make reduces what they pay the vendor.

Haevek has watched this play out on programs where Spark was the processing framework. Engineering teams had already optimized those pipelines as far as the model allowed. Configurations were tuned, code was clean, and costs increased in step with growing data volume. Every available cost saving tool within that pricing structure had reached its limit. The only remaining options were to spend more or find an approach that ran the same workloads at lower cost.

Firsthand experience like this is the primary reason Falcon's pricing model is built the way it is.

A Pricing Model That Rewards Efficiency

Falcon's pricing model is built around capacity rather than consumption. A reasonable analogy is broadband internet. You buy a bandwidth tier, not billed per byte transferred.

Falcon licenses work on the same principle. You commit to an annual tier of concurrent compute cores. That covers everything you run on Falcon for the year. When Falcon ships performance improvements that make pipelines run faster, your licensed capacity handles more throughput, which means additional workloads fit within the same license at no additional cost.

Under the Falcon pricing model, the benefit from performance improvement flows to the customer automatically.

As Falcon's efficiency improves, it reduces cloud infrastructure consumption without affecting Falcon license costs. They complete the same work in less time, using fewer cloud resources, and the infrastructure savings stay with them.

Our pricing model is predictable, making it easier for our customers to plan and innovate. The infrastructure budget for the year is a known number from day one. Data volume growth does not translate into bill surprises. Our customers can confidently experiment and build knowing that there will not be unexpected costs caused by growth in data volume.

What the Falcon Pricing Model Looks Like in Practice

A data analytics company was spending several million dollars a year on Databricks and AWS. They were ingesting multiple terabytes of sensor data per day, transforming it, running analytics, and providing insights to their customers. Their infrastructure cost was growing at the same time their customers were asking for more complex analytics capabilities, and the existing workloads were consuming the budget that would have funded those capabilities.

The migration started with batch and streaming workloads. On batch processing, Falcon showed 80% cost reduction against the equivalent Databricks workflow. On streaming, Databricks was running 26 servers carrying both Databricks fees and the underlying AWS infrastructure costs. Two servers running Falcon have replaced those 26 servers, cutting AWS EC2 costs by 90%.

Eight days after signing the contract, they had their first pipeline in production and were seeing the savings accumulate. Within 90 days, they had expanded their license by 4x, finding more workloads where the same economics applied.

The savings in those 90 days more than covered the annual cost of the Falcon license.

Databricks is still in their stack. It handles data science workloads, notebook environments, and dashboarding. End users who rely on those capabilities see no change. The underlying Delta Lake tables are unchanged. What changed is where the batch and streaming compute runs, and what that costs.

They escaped the consumption pricing trap; new feature investment no longer demanded steep budget commitments for consumption-based pricing before there was revenue. The freed up infrastructure budget went to developing robust analytics capabilities their customers were asking for.

Comparing Falcon, Databricks, and Spark Costs

Falcon is built to be the most efficient distributed compute engine possible so businesses can be more strategic and innovative. Consumption pricing does not support our goal. Under consumption pricing, every efficiency improvement Falcon shipped would reduce Haevek's own revenue. Our pricing model, based on cores running our Kubernetes based Falcon platform, had to reflect what the engine is actually built to do.

The other challenge was competing against open source software, and free is a difficult baseline for a pricing conversation. Apache Spark is free to download. The infrastructure it runs on is not, and often charged on a consumption pricing model.

Falcon is on average 80% faster than Spark on equivalent workloads. This translates into needing 80% less cloud infrastructure to do the same work, which, like the Databricks scenario above, saves more than the cost of the Falcon license. The license tiers are structured so that 50% to 80% of those infrastructure savings stay with the customer after the Falcon license cost is covered, with larger tiers returning a higher share of the savings.

Finding the Right Tier with the Falcon Test Flight

Before any license commitment, the Falcon test flight runs on the customer's actual workloads and collects benchmarks. From that data, Haevek can model a full-scale infrastructure scenario; from 10% of the pipeline we project the full workload and determine the best fit Falcon license tier. A team that needs to justify a budget decision can choose based on measured performance against their own data, rather than a projection built on assumptions.

Haevek wants the customer's first dollar to be backed by evidence and be on a pricing model that rewards both parties for running workloads as efficiently as possible.

The Falcon test flight runs your actual workloads on your own infrastructure so you can see what those numbers look like before any commitment is made. If you're carrying the infrastructure cost of a Spark-based production workload and want to know whether our pricing model works for you, get started here.