An Insider View on How to Evaluate Software Providers
Over the course of my 25+ years in the finance industry, I have interviewed many software providers. More recently I’ve been on the other side of the table while marketing Titan, a back-office/fund administration platform. As technology overhauls are not something that people undertake daily (because who wants to switch software that often!), my experiences may be of value to those of you who may be about to tackle such a project. This article is meant to help those considering adoption of a large new or replacement software product, such as entire front, middle, or back office technology.
There are many different topics I could cover, but I’ve narrowed it down to the following:
- Keeping Up
- Special Requests
- Talk to the Programmers
- Operations Staff Input
- Play in the Sandbox (i.e., Test It Out)
- Quality Assurance
- Billing Model
If the software product has multiple different components, ask if these functions are integrated. There are various interpretations of “integration” with many marketing departments describing everything as integrated, even if their company has written separate programs to enable various functions to talk to each other. So, ask further whether these functions were designed in the same language and at about the same time so the integration exists from the ground up.
For example, many (many) years ago, I attended a cocktail/meeting hour hosted by a major software vendor that was trying to get my then-company to license their fund administration software. They also had a number of their programmers in attendance. I discovered that the portfolio accounting programmers, the partnership income allocation programmers, and the transfer agency programmers had all worked for different companies that had been acquired by this software vendor, and they had coded in three different languages. That company’s definition of “truly integrated” was very different from my own.
Hint: If you see a pricing model that has many add-ons (in the case of fund administration software, think additional charges for having futures in your portfolio accounting module or being able to account in multiple currencies) then there's a good chance that this functionality has been purchased separately.
That's not to say that this loose definition of integration and having various programs working together is a bad thing, but you definitely want to know what you’re getting. An upside is that these individual programs are probably written very well if they were originally one company’s sole product. But a downside is that the main company may not have the same control over or understanding of these third-party programs. Again, they won’t be truly integrated.
This may seem like an obvious one, but make sure to ask for a list of references and speak to at least three. In addition to the regular questions, ask if they’ve had any problems with the software. Instead of getting wrapped up in the fact that their problems existed, however, ask how the problems were dealt with and focus specifically on the provider's response, attitude, and turnaround time. Ask for an example of something the reference company thought the vendor performed well, and something they thought they could improve on. Ask if the vendor sought the reference company's feedback after the event. Consider asking the software vendor about specific negative experiences their references may have described and see how they react. Do they point their fingers back at the client? Or do they admit there were mistakes made and then focus on what they learned and what processes they’ve put in place as a result?
You may want to extract data to customize report preparation or to communicate with your portal. Ask the software vendor how they make their data available. Most modern software programs are built with external accessibility in mind. Most commonly, this is via a REST API. This API allows for automated input of data from an external system, ease of data extraction, and the ability to write on-demand, customized reports. The best thing about these API's is that they are incredibly secure, yet offer ease of use to developers within your organization. They also require little help from the software provider, given that they are intuitive and usually contain excellent descriptions of expected input and output along with all potential error messages.
To find out how committed the software vendor is to keep their product up to date, ask them how they’ve reacted to various recent industry developments. Some software vendors put more money into marketing than they do into development. You don’t want to sign on with a company that prioritizes their marketing budget over, for example, their programming for GDPR.
Ask the vendor how long it will take them to respond to any special requests you may have, whether those be around adding certain fields to their databases or designing a customized report for you. I would also ensure, when speaking with the provider’s references, that their answers are consistent.
Talk to the Programmers
I’ve learned invaluable information from conversations with programmers. Marketing people have a mandate: to present their product to you in the best light. On the other hand, programmers are very “black and white”, and their answers will be logical and technical.
Operations Staff Input
I’ve participated in many web demos where the attendees were all very experienced, brilliant decision makers who had a general understanding of their demands and great questions about our company’s strategy, corporate culture, and where we were in our business cycle. But I often wished their operations staff were there, too. Day-to-day operations staff have better questions about the finer details of the program and will ask for demonstrations. They will have a more in-depth understanding of the pain points of their current systems and will be able to tell if the new system will actually be better for them than their current one. They are the ones who know what inefficiencies and workarounds they have to deal with on a daily basis. Involving them in the process from the beginning will also be a positive contributor to their acceptance of the change.
Test It Out
The vendor should have the ability to let you “play around” in their program for a month or two. With newer, cloud-based programs this is as simple as the vendor sending you a URL. After NDAs are signed, you should also ask if they can upload your own data into their system so that you can test with data you are familiar with.
Modern programs, including the Titan Platform, use something called TDD: Test Driven Development. This means that before something is programmed, the tests for it are written. For example, the programmer would write tests associated with expiring an option. When this functionality is written into the software, these tests are run automatically by the ‘continuous integration’ server used by the developers. At the same time, all the other tests ever written for the entire program are also run. This means that we can be assured that if we make a change to existing code, nothing that depends on it within the platform is affected. This may seem intuitive, but it is common to add properties to data objects within the system. For example, before 2015, there was no requirement to know whether an investor had US indicia that would require that they file under FATCA regulations. When you add the “US Indicia” property to a contact, any feature that uses the investor may be impacted, from subscription transactions to reporting. Unexpected changes can often result in errors in unexpected places. By having every function automatically tested, any potential errors are caught immediately after the new feature code is added to the application. This way, you can be sure the production version of the software is fully functional.
Older programs don’t have this automatic testing incorporated into their software development life-cycle, which means that adding features can often affect totally unrelated parts of the program. When the programmers make any changes, they have to test everything manually, which uses a lot of resources, takes a lot of time, and is subject to error. Manual testing also means that older companies are subject to additional costs every time they work on an upgrade. When you are looking at various programs and the newer ones are cheaper, while this may at first seem too good to be true, don’t automatically write them off. It may just be that they are programmed in a more efficient way that doesn’t require an entire department for manual testing
As explained earlier, when it comes to software, lower costs don’t necessarily mean a lower quality product. There are larger, older software providers out there that charge higher prices to support their marketing divisions and manual testing divisions. Newer software providers may be cheaper simply because they aren’t incurring these costs.
Complex pricing models can indicate that systems aren’t truly integrated. When I was on the buying side, I remember looking at a pricing proposal for a fund administration system that was 12 pages long. If you were going to account for futures, that was an extra cost. If you were going to be multi-currency, that was an extra cost. This kind of pricing is suggests that the underlying software vendor has licensed these additional modules from a third party such that the various features of the software are not truly integrated.
Make sure you put the time into interviewing and getting to know your software provider. In-person meetings always give you a better sense of the culture of the company. If you’re licensing a program that is going to be integral to your operations, the relationship is going to be long-term. You’ll be working with your software vendor through onboarding, during staff training, for upgrades, and for special requests. You’ll want to make sure you know who you are partnering with.