scx.ai logo

Media

Our Take: AI Inference Doesn't Need New Data Centres

Following David Keane's Certified Strategic interview, we look at why efficient inference can scale inside existing data centres, what Node 2's repeatable deployment model changes, and the questions buyers should ask before waiting for new capacity.

By SCX.ai Team5 min read

Certified Strategic host Michael Cooney recently interviewed SCX.ai co-founder and CEO David Keane about Node 2, purpose-built inference chips, AI agents and sovereign infrastructure. The resulting headline was direct: AI inference doesn't need new data centres.

That headline deserves both emphasis and one useful qualification.

This is not an argument that Australia should never build another data centre. New facilities will continue to play an important role. The point is that new construction should not be the prerequisite for every additional unit of AI capacity. If efficient inference systems can operate inside existing facilities, organisations can add useful capacity without waiting for a new building, a new power allocation or a new water connection.

That distinction changes the infrastructure conversation. The question shifts from "How many buildings are being announced?" to "How much useful AI can we deliver from the infrastructure already available?"

Inference Is a Different Infrastructure Problem

Training and inference are often bundled together under the label of AI compute, but they place different demands on infrastructure.

Training creates a model. Inference puts that trained model to work, responding to a customer question, reviewing a document, generating code or completing one step in an agent workflow. Training can be a large, concentrated project. Production inference is a continuous service where latency, throughput, cost, reliability and jurisdiction all matter.

That difference creates deployment options. Inference does not always need to replicate the architecture of a large training cluster. Purpose-built systems can be deployed in smaller blocks, connected across suitable sites and operated as one service.

For customers, the building is not the product. A secure, reliable and economical response is the product. Infrastructure decisions should start with that outcome rather than with a target for square metres or megawatts.

Efficiency Creates Room to Grow

The interview highlighted two features of SCX.ai's approach: SambaNova's application-specific chips and air-cooled systems. Their importance is not simply that they use less energy. It is that they widen the range of facilities where inference capacity can be deployed.

Lower power density and air cooling can allow systems to fit within existing facility envelopes rather than requiring a specialised new hall. Air-cooled racks also avoid adding a chip-level liquid-cooling requirement. That is not the same as claiming an entire facility consumes no water, but it removes one potential barrier to deployment.

David also discussed the performance comparisons covered in the interview: two and a half to five times greater energy efficiency and four to six times faster performance. As with any infrastructure benchmark, results depend on the model, hardware and serving configuration being compared. The architectural lesson is broader than any single figure: more useful output from each kilowatt means more capacity can fit within constrained power and cooling envelopes.

Efficiency therefore becomes a deployment advantage, not just an environmental one. It can shorten the path from available facility space to production AI capacity.

Node 2 and the Value of Copy-Paste Infrastructure

In the interview, David described Node 2 as a "copy-paste from Node 1". It is intended to double SCX.ai's computing capacity, and he said it was on track for later this year.

Copy-paste may not sound ambitious. In infrastructure operations, it is a strength.

A repeatable node starts with a known bill of materials, a known power and cooling profile, an established deployment process and an operating model that has already been tested in production. Each deployment adds evidence for the next one. Teams spend less time redesigning the system and more time bringing capacity into service.

This is also where a distributed model becomes useful. Smaller blocks at multiple suitable sites can be linked into one cluster, rather than making every expansion dependent on a single large development. The aim is not distribution for its own sake. It is repeatable capacity that can be delivered where customers, data and connectivity already exist.

Agents Make Inefficiency More Expensive

The demand side of the interview matters just as much as the infrastructure side.

A chat interaction can involve a prompt and a response. An agent can retrieve documents, call tools, write and run code, check its own output and make repeated model calls before returning a result. One business task can therefore generate a much larger inference workload than one chat session.

As agents move into production, inefficiency compounds with every step. Higher energy use per token, avoidable latency and unpredictable serving costs are repeated across the whole workflow.

This is why efficient inference is becoming an economic requirement. It affects the cost of delivering an agent-powered product, the number of workloads that can run within an existing footprint and the price predictability a provider can offer customers. More demand does not remove the need for discipline. It increases it.

Five Questions for AI Infrastructure Buyers

The practical lesson from the interview is not that every organisation should use the same architecture. It is that buyers should test whether proposed capacity is deliverable, efficient and appropriate for the workload.

  1. Is the capacity live? Ask what is serving customers today, not only what is planned for a future quarter.
  2. What is the measured efficiency on your workload? Compare useful output, latency and energy consumption on representative models rather than relying on a generic chip benchmark.
  3. What facility changes are required? Establish whether deployment depends on new power, liquid cooling, construction or other long-lead upgrades.
  4. Can the deployment be repeated? A standard operating model can make expansion across suitable sites faster and more predictable.
  5. Who controls the service? Understand where prompts, outputs, logs and model weights are handled, and which organisation and jurisdiction are accountable for them.

None of these questions is answered by a rendering of a new building. All of them require operating evidence.

Delivered Capacity Is the Milestone That Matters

The choice is not between using existing buildings forever or building new ones at any cost. It is between treating construction as the measure of progress and treating delivered AI outcomes as the measure of progress.

Existing data centres offer an immediate opportunity when efficient systems can operate within their power and cooling envelopes. New facilities can then be planned where they add genuine long-term value, rather than becoming the default answer to every increase in demand.

The real milestone is not breaking ground. It is a customer receiving a fast, secure and affordable response from capacity that is already in service.

Read the original Certified Strategic article and listen to the full interview, explore our global inference network, or talk to our team about your inference workload.

Source

SCX.ai is Australia's sovereign AI infrastructure company, providing efficient AI inference for enterprises, government agencies, developers and researchers.

Related Topics

AI inferencedata centresDavid KeaneCertified StrategicNode 2energy efficiencyair coolingSambaNovasovereign AISCX.ai
Our Take: AI Inference Doesn't Need New Data Centres