Insights / Integration
Inside an integration project: from discovery to monitoring
Connecting systems is less about the connection than about everything around it: mapping, translating, testing and watching the result once it is live.
4 min read · KYH Systems
An integration project connects software that was never designed to work together, so that data moves between systems securely and automatically instead of through exports, re-keying and spreadsheets. It is an engineering process with distinct stages, and knowing them makes it much easier to scope, budget and judge.
1. Discovery and API mapping
Before anything is built, architects audit what is already there: legacy databases, ERP systems and cloud SaaS endpoints. They catalogue the data each one holds, and find where communication is slow or breaks down. The data flows are then mapped, using an enterprise service bus or REST and GraphQL interfaces, before any code is written.
2. Data transformation and middleware
Systems rarely agree on formats. One keeps rigid SQL tables while another exchanges asynchronous JSON. Middleware sits between them, translating, cleansing and validating data as it travels. It is also where security is enforced, including the encryption requirements of regimes such as HIPAA and GDPR.
3. Testing and deployment
Testing covers automated unit tests, peak-load tests and end-to-end validation in a sandbox, aimed at the problems that only show up under real conditions: latency and corrupted data. After launch, centralised logging and dashboards watch the message queues and alert the team to failed transactions or timeouts, so a problem is found by you and not by a customer.
The stages at a glance
| Stage | What happens | Reference point |
|---|---|---|
| Discovery and mapping | Audit the systems, document endpoints and data flows | TOGAF enterprise architecture framework |
| Middleware development | Build the transformation engine and validate payloads | GDPR and HIPAA encryption protocols |
| Testing and deployment | Load and sandbox testing, then real-time monitoring | ISO/IEC 25010 software product quality |
Questions people ask
What is the goal of an integration project?
To connect platforms that hold data in silos so it flows between them securely and automatically.
How long does one take?
Often three to nine months, depending on how many systems are involved and how complex they are.
Why do quick custom scripts so often fail?
They tend to lack error handling, retry logic and scalable middleware, so they break the first time one of the source APIs changes.
What security measures are used?
Typically mutual TLS, OAuth 2.0 token verification and end-to-end encryption of the data being exchanged.
Have a project like this?
Send us the problem, not a brief.
Write to usNext article