When an operator decides to get serious about operational data, the first proposal on the table is usually a central historian. Consolidate every asset into one process data platform, put the analytics on top, done.
We scored that option against two alternatives during an architecture assessment on a multi-vendor estate.
| Option | Score | Where it fails |
|---|---|---|
| Federate the existing historians | 2 / 10 | A second system of record, in a different technology, for a subset of the data, whose output still has to reach the warehouse |
| Expand the incumbent SCADA platform into a hub | 3 / 10 | Inverts data gravity onto a control system, and forfeits format portability |
| Broker-native ingestion into an open table format | 7 / 10 as a target | Depends on inventory and access work that was not finished, which is why the same review scored it 4 as something to start immediately |
Those numbers are ours, from one assessment, and reasonable people would score them differently on a different estate. The third row carries the caveat worth repeating: 7 is the score for where you are going, 4 was the score for starting on Monday. The reasoning behind the scores travels better than the scores do.
What historians are genuinely good at
Worth being fair, because the historian is not a bad product. It is a product built for a specific situation, and it is excellent in that situation.
Process historians were designed for a single plant, or a small number of similar plants, running a homogeneous control system. High-frequency data, thousands of tags per unit, engineers who live in the trend client all day. Swinging-door compression is a genuinely clever piece of engineering that stores a deadbanded process signal at a fraction of its raw size while bounding the error, and a warehouse gives you nothing equivalent for free. The asset models are mature. The engineering workflows built on them, from trend analysis to golden-batch comparison, are decades refined and hard to reproduce in SQL.
If you run one big facility with one control system and your primary consumers are process engineers doing time-series analysis, a historian is very likely the right answer and this post does not apply to you.
The acquisition estate is the opposite case
An upstream operator assembled through acquisition has close to none of those properties.
The assets are geographically scattered and operationally independent. The control systems are heterogeneous, because each asset arrived with whatever its previous owner chose. Tag naming is inconsistent across platforms and often inconsistent within one. The consumers are not process engineers in a trend client, they are production engineers, allocation analysts, and reservoir teams who want to join operational data to commercial data, which is a warehouse workload.
Most importantly, the analytics destination usually already exists. The company already has a cloud warehouse, already has business data in it, and already has people who know how to query it.
Proposing a historian in that context means standing up a second system of record, in a different technology, with a different query language, for a subset of the data, whose primary output has to be replicated into the warehouse anyway before anyone can join it to production or economics. You have added a hop and a license, and the analytical question still gets answered in the warehouse.
That is the shape of the 2 out of 10. Not that the technology is bad, but that it is answering a question this estate is not asking.
Why expanding the SCADA platform scores little better
The other common proposal is to take whichever SCADA platform has the largest footprint and make it the hub. It is already licensed, the team already knows it, and it already has a historian module.
We scored this 3, and the reasoning is worth separating from the historian argument because they fail differently.
Putting the incumbent platform in front of the warehouse inverts the data gravity. Operational data lands in the SCADA layer, gets historized there, and is then extracted. Every downstream consumer is now dependent on a system whose primary job is control, whose availability requirements are operational rather than analytical, and whose upgrade cycle is governed by the automation team.
It also forfeits format portability. Data historized inside a vendor’s store is in that vendor’s format, and getting it out later is a migration project. Landing the same data as open columnar files on your own object storage lets the warehouse read it in place with no ingest compute, and leaves every future option open. It is not free, since storage, catalog and compaction are real, but it is cheap and it is yours.
And for the foreign assets, the ones on other platforms, routing them through the incumbent gateway adds a hop without unblocking anything. The access problem for those assets is commercial, not architectural, and no amount of hub configuration solves it.
There is a real nuance here that the score obscures. The incumbent platform paired with a publishing module is frequently the right edge publisher for the assets that actually run it. That is a good use of the product. The failure is promoting the edge publisher to estate hub.
Existing historians are read-sources, not expansion targets
Most estates already have a historian somewhere, usually inherited with an asset from a larger operator.
The recommendation is not to rip it out. It is to stop treating it as the direction of travel.
Read from it. It holds history you cannot get anywhere else, and that history is valuable. Pull it into your landing zone, normalize it alongside everything else, and let the engineers who like the trend client keep using the trend client.
Do not extend it. Do not add the newly acquired assets to it. Do not build the canonical asset model inside it. Every tag you add is a tag whose licensing, availability, and portability you have handed to a vendor, in a system that is not where your analytics live.
The distinction between “we read from it” and “it is our platform” sounds small in a meeting and determines the next five years of cost.
What we actually recommended
The option that scored 7 is unglamorous: publish from the edge where the assets support it, poll where they do not, land everything raw as Parquet under an Iceberg table on the operator’s own object storage, normalize in the warehouse, and let the warehouse serve the consumers who are already there.
The open format is doing real work in that sentence. It means the landing zone is not owned by whoever you bought a license from this year, and it means the warehouse can read the data without a second ingest and a second copy. It is the same reasoning as the abstraction layer above your SCADA vendors, applied to storage rather than to the tag model.
The normalization work is unchanged either way. Tag mapping, unit conversion, quality flag translation, asset contextualization, all of it still has to happen, and it belongs in the silver layer of a medallion regardless of what you ingested from. Choosing not to buy a historian does not save you that work. It only stops you from paying a vendor to do a different version of it in a place you cannot see.
The migration nobody budgets for
One consequence of the historian route deserves its own paragraph, because it is invisible at purchase and expensive later.
A historian accumulates history. That is the product. Five years in, it holds the only copy of five years of process data, in a proprietary store, at whatever compression settings were chosen during commissioning by somebody who has since left.
Getting that out is not a query. Depending on the product it is an export API rated in tags per second, or a bulk extract tool with its own licensing, or a support engagement. Operators who have done it describe it in weeks. And the compression is not reversible: what comes out is what the historian decided to keep, which for a deadbanded signal is not the raw measurement.
None of that is an argument against historians in the environments they suit. It is an argument for knowing, before you start adding tags, what the exit looks like and roughly what it costs. The same question applied to an open table format on your own storage has a boring answer, which is the point of asking it.
What this does not solve
The honest part, because a recommendation that solves everything is a sales pitch.
It does not give you the trend client. Process engineers who work in a historian interface all day will find a warehouse a downgrade for their specific workflow, and telling them otherwise is how you lose their support. Budget for a visualization layer, or accept that the historian stays for that audience.
It does not solve sub-second data. If you have genuine high-frequency requirements, vibration analysis, electrical transients, anything where the interesting behavior lives below one-second resolution, a warehouse-first architecture is the wrong tool and a historian or a purpose-built store is the right one. Most upstream surveillance does not need this. Some does.
It does not solve access. Everything in this post assumes you can get the data out of each asset, which on a heterogeneous estate is a commercial question before it is a technical one.
And it does not come with the maturity a historian has. Compression, interpolation semantics, and asset modeling are solved problems inside those products and problems you will be solving yourself in a warehouse. That is a real cost, and it should be weighed rather than waved past.
The question that decides it
Ask where the analytical question gets answered.
If the answer is “in a trend client, by a process engineer, on one plant,” buy the historian and do not let this post talk you out of it. If the answer is “in a query joining operational data to production, allocation and economics across every asset we own,” then the platform already exists, it is the warehouse, and the remaining work is getting data into it in a shape you control.
Most operators we talk to have already answered that question without noticing. They built the warehouse three years ago and staffed it. The historian conversation is usually a proposal to build a second one.