5 lessons we learned building our enterprise-grade integration with Microsoft Dynamics 365

Microsoft Dynamics 365 is a well known CRM (Customer Relationship Management) & ERP (Enterprise Resource Planning) solution for large and enterprise-level organisations. As Drupal increasingly expands into the enterprise market there is a clear demand for integrations with Microsoft products, yet today there are very few learning materials and libraries to assist with this available. 

Historically, the two worlds of Drupal and Microsoft have had a very small intersection in terms of technology, philosophy and their developer communities. It makes the integration of Dynamics 365 with Drupal even more challenging.

When we started a large MS Dynamics integration project over a year ago, the only relevant Drupal modules we could find were unfinished or outdated. We were basically starting from scratch. 

The situation hasn’t changed much since then either. The natural complexity of CRM integration projects often causes delays and overspending, and when an integration is finally finished, everyone just wants to forget about all that pain and move on to something else.

At SystemSeed, we went through several CRM integration projects with our clients, but this type of work remains one of the most challenging and complex. In this article, I will share our learnings from the recent Microsoft Dynamics integration project for our long term client Concern Worldwide, Ireland's largest charity. Most of the lessons apply to large CRM integrations in general.

Lesson 1: Understand the structure and business processes of the organisation

The key to a successful CRM integration is to have a deep understanding of the structure and business processes of the client organisation. CRM solutions like Dynamics 365 or Salesforce offer some useful default settings, but every organisation will have their specific requirements, and you must fully understand them. In the CRM world, there is no “one-size-fits-all” solution, and each client implementation will have its own customisations. 

More than anything else, it requires strong communication & collaboration skills from the development team. You will usually see Agile development teams actively participating in project discussions, regularly sharing progress, flagging blockers, zooming in and out between technical details and high-level roadmap, and pushing for long term solutions rather than workarounds. A CRM integration project is a great test of all those skills for your team.

On a technical level, the data architecture skills of the development team are crucial for designing optimal data flows for the organisation. One may think that modern CRM tools already define all the data structures and workflows, so there is very little to amend. All you need, you’d think, is to make a few API calls here and there. 

From our experience, it’s the opposite: enterprise CRM solutions focus on flexibility so that organisations of different sizes and from different sectors can build their own CRMs on top of the “base” software. Understanding and actively participating in data architecture is one of the most important parts of successful CRM integration. Everything else will go much easier and smoother if you take this lesson seriously.

Lesson 2: Get access early 

One of the first things to do at the beginning of the CRM integration project is to set up user & API access for the development team. Believe it or not, it may be a very challenging task. CRM administrators tend to push back on setting up user accounts until the project is finished, but it’s really hard to develop and test the work without having full access to everything. This includes:

• Access to sandbox & production environments
• Developer access (logs, any developer tools, settings, etc)
• API credentials (keys, endpoints, documentation, logs) 
• User access for people who will be testing and accepting the work

The more you can get at the beginning of the project, the easier it will be to deliver functionality piece by piece instead of waiting till the very end of the project only to find out that half of the features don’t work as expected.

Once the access is obtained and tested, it’s time for a quick hands-on investigation of the admin UI and available API endpoints. Different versions of Microsoft Dynamics look and behave quite differently, so it’s worth exploring what’s available first.

Lesson 3: Document everything

CRM integration will inevitably require comprehensive documentation for CRM users and developers. We prefer to start early and keep expanding our documentation as the project evolves.

Well-structured and up-to-date documentation helped us to save time and avoid unnecessary confusion in many areas of CRM integration development. Good documentation helps with:

• Structuring and sharing the knowledge between developers
• Onboarding new developers when the project is big enough
• Supporting and upskilling the client who will be using the integration 
• Reminding the team why certain decisions have been made and helping them navigate the code
• Supporting CRM integration in the long run as it will most likely be around for the next few years

Comments in code and notes in the issue tracker won’t be enough. To make documentation easy to find and maintain, we recommend using a specialised documentation hub service. If you don’t have one yet, there are plenty of options available such as Readme.io, Notion, Gitbook, and BookStack.

Lesson 4: Upskill your team and the client

From our experience, the entire CRM integration project is a non-stop learning process. When you join two completely different worlds like Drupal and Microsoft new findings will appear every day. Without specialised knowlege, it may be hard to navigate new tools and make optimal technical decisions. So, how do you balance learning & doing to deliver the best results?

At the beginning of the project, it’s very useful to deliver CRM UI training for the whole team. Even if developers won’t normally interact with CRM UI, they need to understand how it’s supposed to be used by the client. As we already learned from lesson 2, developers should have their user accounts set up in the CRM and feel comfortable using them.

Drupal developers who have never worked with Dynamics 365 may try to reuse the usual mental model of Drupal development (AKA the “Drupal way”). To help them navigate the “Dynamics 365 way”, Microsoft offers free fundamentals course. If it looks a bit boring, there are a few paid courses available on Udemy and Pluralsight that might suit you better. Pick a highly rated fundamentals course relevant to your version of MS Dynamics to get up to speed with the basics and terminology.

As mentioned in the previous lesson, all new knowledge should be collated and documented. This way, by the time when the project is ready to launch, your developers will be already upskilled and ready to train other teams and clients.

Lesson 5: Add verbose logging and helpful UI elements

At SystemSeed, we deal with integrations on an everyday basis and often encounter modules and SDKs with very poor logging and debugging capabilities. For our MS Dynamics integration project, we made sure our logging system is the best in class. It has already proven itself as a great time saver during testing, troubleshooting and API version upgrades. 

The following recommendations should help developers build robust logging and monitoring for the CRM integration of any complexity:

• Log all requests and responses along with their data payloads in a human-readable format. Don’t limit your logging to errors only.
• Find or build a low-level SDK that makes it easy to run and debug API responses. For example, SystemSeed built the Drupal module for Dynamics NfP365 plugin for charities.
• It’s not enough to just log errors. Make sure they are alerted using systems like Rollbar, Sentry, etc.
• Even if 100% API uptime is guaranteed, make the integration resistant to temporary API downtimes. 
• Cover what’s possible with automated tests. We prefer end-to-end tests as they are less synthetic and prove that integration works. If you are new to automated testing, check out my intro article here.
• Finally, make sure non-technical users have a meaningful UI interface in Drupal to understand the status of CRM synchronisation and perform basic troubleshooting themselves if needed.

Some of the items above may look like an extra to the scope of the CRM work but our experience integrating Drupal and Microsoft Dynamics has proven how important they are. It’s worth reserving the necessary resources in the development plan for implementing proper logging and monitoring. It will help launch the project on time and keep everyone happy in the long run!

Final note

In this article, we shared some lessons learned during our recent Dynamics 365 integration project. Most of them are applicable to other CRM systems like Salesforce or ThankQ CRM.

If you need any help building or supporting a CRM integration, please contact us. It’s what we do!