Most digital products we create, whether that’s a website, mobile app or web app, tend to have at least one software integration. No two integrations are ever the same, even if you use the same 3rd party platforms and systems. Every organisation has a different set-up and their own, unique needs and requirements to integrate their systems. There are however some common approaches and challenges, which we’ll explain in a little more detail.
We hope that this article gives you the ability to ask the right questions and enable you to confidently approach your integration project.
- What is software integration and what can it do for you?
- What is an API?
- What is an SDK?
- Direct data access
- What are the common challenges with software integration?
What is software integration and what can it do for you?
Software integration enables two or more different systems to communicate. They can be one-way or two-way. And they allow you to more easily add functionality provided by 3rd-party software vendors to your systems.
The list of possible integrations is limitless but a few common examples include:
- Storing and updating records in a CRM or database e.g Salesforce, PipeDrive, ThankQ
- Enabling payment processing e.g. Stripe, SagePay, PayPal
- Enabling new communication channels via email, SMS, Push notifications e.g. MailChimp, Send In Blue, Twilio, Firebase
- Displaying data provided by a 3rd party
- Transferring data between systems
- Accessing proprietary functionality provided by another company
What are the different types of software integrations?
There are three different ways you might integrate with a digital product:
- API – Application Programming Interface
- SDK – Software Development Kit
- Direct data access
Sign up to get helpful articles, case studies and the latest news straight to your inbox.
What is an API?
An API, or Application Programming Interface, is a defined way for multiple pieces of software to communicate with one another. This can be used to pass data between the systems or trigger functionality or behaviour remotely.
API’s can also be thought of as a contract that one piece of software sets up with another. Using an API, a development team knows how to request the results it wants but doesn’t have any say over how it is done. This is delegated to the 3rd party.
An API is commonly used across both apps and websites and might include features such as handling payments or donations or integrating with a CRM to pass account data.
We use this approach regularly for our partners, including the New Wine website and the Try Dry app for Alcohol Change UK.
What is a REST API?
A REST API is the most common type of API. It’s designed to work well within the constraints of the web, is easy to document and familiar to developers.
What is a SOAP API?
SOAP is an older standard that is now mostly found in older, legacy API’s. It is harder to document, harder to understand and can take longer to integrate with.
What is a webhook?
Sometimes software integration isn’t as simple as requesting something and immediately getting a response. You may send data to a 3rd-party to perform some processing. And this could be completed sometime after your initial request. It’s also possible that the 3rd party needs to initiate some process without you having first made a request e.g. receiving an SMS.
A webhook is the answer to both these scenarios, providing new options for software automation. The 3rd party essentially sends a message to tell your system that a particular event has occurred. Typically this is just a notification and it is then up to your system to fetch the relevant information from the API.
We used this approach to develop John Charcol’s bespoke SMS platform.
What is an SDK?
Not all integrations happen over the web. Especially when working with apps, it can be useful to add specialist functionality from a partner. This is where an SDK, or Software Development Kit, comes into play.
An SDK is a special package of code that can be added to an app to provide certain functionality. Using an SDK, a development team just needs to know how to trigger the desired behaviour without needing to know how to achieve it themselves.
An SDK can be almost anything. We’ve recently helped clients provide in-app ID verification via Onfido and remote technical support via integrating SDK’s.
Direct data access
Most apps and websites store data in one of a small number of database platforms. Whilst it can be tempting to integrate two systems by giving both access to a common database this is almost always a bad idea and more trouble than it’s worth.
By directly communicating with a database you bypass any business logic that is present in your app which can cause issues around data integrity and validation. It’s also difficult to provide appropriate access to multi-tenant systems as database access is essentially a firehose without the ability to filter by organisation.
Unlike an API, there’s also no direct contract between the two systems. This means that if one needs to make changes to the database structure it may break the integration, resulting in a more fragile solution that requires more maintenance.
Performance can also be an issue as poorly optimised queries could be run directly which may impact the responsiveness and user experience of regular users of the system.
What are the challenges of integrating with third-party providers?
Documentation
Documentation quality differs massively between API providers. Large SaaS providers (like Stripe, Salesforce or Twilio) often have really complete and easy to understand documentation, which makes it much easier to integrate with. Smaller organisations may not have the same resources to dedicate to their documentation which can make a software integration project more trial and error. Typical challenges found in documentation is only covering the success cases and not providing details of errors. Or not outlining all possible response options.
Unexpected/inconsistent behaviour
Bugs exist in all systems and it’s possible that a 3rd party SDK or API may not behave exactly as advertised. If not managed correctly this can cause issues to appear on your site or app.
Scale
Ensuring an integration can scale as you do is critical. API’s often have rate limits and these need to be understood to make sure they won’t cause bottlenecks in your product as you grow. Increased usage of commercial API’s will likely also have financial implications.
Security
When passing data between systems it’s essential that it is done securely. It’s important to make sure appropriate encryption and authentication methods are available and that data is stored and processed adequately by the 3rd-party.
Monitoring when errors occur
As software integrations are often processes that run in the background without a user interface (UI), it can be a challenge to know when something has gone wrong. Good monitoring of integration tasks is essential, leveraging tools such as Rollbar to highlight issues. It’s also important to consider what information would be needed in retrospect to debug any issues that might arise. This may require storing more data than is required for the task to enable problems to be diagnosed retrospectively.
Synchronisation
If data is being changed in multiple places, it can be challenging to reconcile this without getting lost updates. In an ideal world, data would flow one way through your system but if this isn’t possible a strategy is needed to handle synchronization.
Performance
When dealing with a 3rd party service you have very little control over the speed at which they process your requests. This can sometimes cause users of your system to perceive a reduction in performance or the UI being less responsive. Common examples of this might be when a user interaction requires integration to send an email, or generate a document. It’s therefore important to consider whether a queueing architecture might be appropriate. This will separate the user interactions from the actual integration points but can require a change in design to workflow or your UI.
How can software integrations help with legacy systems?
It can be expensive and time-consuming to rewrite or upgrade a legacy system, especially if it is well adopted and business-critical. That can often be stifling to progress and prevent improvements in user experience or automation. Developing an API for a legacy system and enabling data to be securely transferred in and out can reduce this technical debt.
It’s also commonplace for legacy systems to contain many functions in a single codebase. We often see systems that contain business-critical back-office functionality mixed alongside the marketing website. Usually, this means that the website becomes dated and is impossible to update as the cost of replacing the back-office system is prohibitive.
The overlap between these systems is usually slim. So we’d recommend separating functionality and enabling API access for any overlapping features. This creates a more agile environment and reduces your risk. With this approach, you can continue to use the good aspects of a legacy system whilst also benefiting from new tools and automation opportunities.
We were able to help Toilet Twinning separate its back-end system from the website by using this approach. You can read more about it in our case study.
Integrating your systems requires careful consideration, and it can be a daunting experience, especially if you’ve not managed one before. But partnering with the right team to do it the right way, can bring huge efficiencies, cost savings and a new way of working.
If you’d like to discuss how integrating your systems could aid digital transformation and the art of the possible. We’d love to chat, just get in touch. We look forward to hearing from you.
