Data Engineering & Analytics
One set of numbers, and everyone agrees on them.
Most companies this size do not have a data problem. They have a reconciliation problem: four systems that each hold a defensible version of the truth, a month-end that runs on spreadsheets, and one person who knows why the finance number never matches the operations number.
not in scoperegulated lending decisions · replacing source systemsScoring and decisioning
We build the layer that ends the argument — the warehouse, the pipelines that feed it, and the reporting on top — so that a number in a board pack, a number in a dashboard and a number in the source system are the same number, and you can show where it came from.
What this silo does not cover, stated in full rather than left to the rail: we do not build models that make or materially influence a regulated credit, lending or insurance-pricing decision, and we do not replace the source systems that produce your data — we model and pipeline what those systems produce. SeeScoring and decisioning.
What this usually looks like before we start
- Month-end close takes days longer than it should, and most of that is assembling and checking data rather than doing accounting.
- Two reports about the same thing disagree, and the meeting becomes about which one is right instead of about what to do.
- The critical reporting logic lives in a spreadsheet, a saved query, or one analyst's head. When that person is on leave, reporting stops.
- Somebody asks a reasonable question — margin by category and channel, exposure by segment, shrink by store — and the honest answer is "give me three days."
- You bought a BI tool. It is showing the same numbers as before, more attractively, because nothing underneath it changed.
Reporting nobody trusts is worse than no reporting, because you pay for it and route around it.
What changes
- The close gets shorter and less manual.
- Data arrives on a schedule, reconciles automatically, and fails loudly when it does not. Your team spends month-end reviewing exceptions rather than rebuilding the same workbook.
- Every number has a traceable lineage.
- For any figure on any report, you can follow it back through the model to the source system and the load that produced it — what makes reporting defensible to an auditor, a lender, an acquirer or a board that has been surprised before.
- Definitions live in one place.
- "Active customer", "net revenue", "gross margin" are defined once, in the model, and every report inherits that definition. Changing a definition becomes one reviewed change rather than a search through forty workbooks.
- Your analysts stop being a bottleneck.
- A modelled, documented semantic layer means the people who know the business can answer their own questions instead of queuing for someone who can write SQL.
What we build with
We work in the Microsoft data stack, because that is where this client base already is and because it keeps the estate on one identity, one security model and one bill:
- Azure SQL Database / Azure SQL Managed Instance / SQL Server — warehouse and mart storage.
- Azure Data Factory — orchestration and ingestion.
- Microsoft Fabric and Azure Synapse — where the scale or the workload justifies it. Often it does not at this company size, and we will say so rather than sell a platform you will pay for and not use.
- Power BI — semantic models, reports, row-level security, and deployment pipelines.
- Tableau — where it is already your standard, or the right fit.
- Python and dbt-style transformation patterns — tested, version-controlled transformation logic.
For companies moving off ageing SQL Server infrastructure, this work is frequently the second half of a migration project — see Azure & Microsoft Migration. Migrating the database and modelling the warehouse are adjacent purchases, and doing them in the wrong order wastes money: rehosting a database nobody trusts just relocates the problem.
In finance
Reporting that has to be defensible, not just informative.
- Management reporting and board packs built from a single reconciled source, with a lineage you can show.
- Portfolio, exposure and segment reporting with history preserved as it was recorded.
- Regulatory and client reporting where the number must reconcile exactly, every time.
- Row-level security so people see the accounts, branches or portfolios they are entitled to see.
In retail
Reporting that has to arrive in time to act on.
- Sales, margin and inventory across stores, channels and categories, in one model rather than three that disagree.
- Point of sale, e-commerce and ERP data unified — usually the hardest and most valuable part.
- Stock, replenishment and shrink reporting current enough to act on.
- Seasonality and peak analysis using history you can trust.
What this silo covers
How we work
Understand the questions. We start with the decisions the reporting is supposed to support, and trace where today’s numbers come from and who adjusts them by hand.
Assess the sources. What systems hold the data, what condition it's in, and where the quality problems are — including the parts that need cleaning up at the source.
Model it. A warehouse designed for the questions you actually ask, with the grain, conformed dimensions and history you need.
Build the pipelines. Scheduled, monitored, idempotent loads that alert on failure and never half-write or duplicate.
Report on it. Power BI or Tableau — usually whichever your people already know — plus a semantic model your team can extend.
Hand it over. Documentation, source control, a deployment pipeline, and working sessions with the people who will own it.
Built for
- Finance and retail companies, 15 to 500 employees, where month-end or reporting depends on manual reconciliation.
- Teams with real analysts already in place who need an engineering layer underneath them, not a replacement for them.
- A company running a few million rows across a handful of source systems — a modelling and pipeline problem, not a big-data problem.
- Companies already on, or moving to, the Microsoft data stack.
Not a fit
- You need a regulated credit, lending or insurance-pricing model — see Scoring and decisioning for exactly where that line sits.
- You want a report count or a dashboard count delivered — a hundred dashboards nobody opens is not a successful project, and we will say so.
- Your data volume or streaming requirements genuinely need a lakehouse platform on day one — the assessment will tell you if that's true, and often it isn't.
- You want the visualisation layer fixed without touching what's underneath it — a BI tool is a window, and a new window doesn't change what's behind the wall.
Access to your data during this work follows the same standard we hold ourselves to everywhere else — named individuals, least privilege, logged and time-bound elevation.
Questions buyers actually ask
We already bought Power BI and it did not fix anything. Why would this be different?
Because a BI tool is a window, and the problem is usually behind the wall.
If four systems disagree, if there is no shared definition of "customer", and if the numbers require manual adjustment before anyone believes them, then adding a visualisation layer produces prettier versions of the same disagreement. That is the single most common state we find, and it is why the tool gets blamed.
The work that changes things is the modelling and the pipelines underneath: one reconciled source, definitions agreed once, loads that test themselves, lineage you can trace. Once that exists, the reporting tool you already own usually turns out to be fine.
Do we need Microsoft Fabric, or a data lake, or a warehouse at all?
Probably not all three, and quite possibly none of the fashionable ones.
A company with a few million rows and half a dozen source systems is well served by a properly-modelled Azure SQL database and a scheduled pipeline. That is unglamorous and it works, it costs a fraction of a lakehouse platform, and your team can operate it without specialist skills you would have to hire.
We recommend the heavier platforms when there is a reason — data volume that genuinely does not fit, semi-structured or streaming sources, or a real-time requirement. Not because it is on a roadmap slide. The assessment tells you which situation you are in, and "you need less than you think" is a common finding.
How long before we see anything useful?
We aim for a first genuinely useful report early, on a narrow slice — one subject area, end to end, in production, being used — rather than a long build ending in a single reveal.
There are two reasons, and only one of them is about morale. The first is that a working slice is the only reliable way to discover what the data is really like; source system documentation is optimistic almost everywhere. The second is that it gives you a real decision point early, at low cost, about whether to continue.
Who owns the models and the code?
You do, without qualification.
The warehouse is in your Azure subscription. The pipeline and transformation code is in your repository. The Power BI or Tableau content is in your tenant, under your licences. Documentation is written for your team's use, not as a lock-in device.
We do not build things only we can operate. If we vanished, a competent data engineer could read the repository and the documentation and carry on.
Our data is a mess. Should we clean it up before we call you?
No. Assessing the mess is part of the work, and most attempts to tidy data before a project are wasted because nobody yet knows which parts matter.
What does help before a first call: a list of your source systems, a rough idea of volumes, the reports your business actually relies on today, and honesty about which numbers currently get adjusted by hand and by whom. That last one is the most useful thing you can bring, and it is the one people are most reluctant to say out loud.
Can you work with our existing analysts?
Yes, and it is usually the better outcome.
Your analysts know the business, which is the part that cannot be outsourced. What is often missing is the engineering layer underneath — modelling, pipelines, testing, source control, deployment. We build that, and work alongside your people so they can use and extend it.
Where a team wants to learn the practice rather than just inherit the output, we will work that way deliberately: pairing, review, and documentation aimed at handover from the start.
Start with the assessment, not the migration.
Book an assessment call30 minuteswith the engineer who would scope the work[PLACEHOLDER: written-summary turnaround, not yet committed]