Join our weekly blog

Beyond the Tech - Techniques for testing vendor alignment



When selecting the right technologies for your hiring process, there are two main components to consider: the technology and the vendor.


Evaluating the technology usually gets most of the attention. This is all the effort you put into ensuring the solution accomplishes what you need in the way you need it. It is all the steps to understand what it does, how it works, and how it will work within your system. Features and functions, usability, integrations, and adoption, are all part of technology.


The vendor, on the other hand, is commonly overlooked, but it shouldn’t be. It is the vendor that determines how you and the technology are supported now and into the future. A good vendor that is a good fit for your organization could be the difference between a technology that works in the moment, and one that has lasting if not evolving value for your organization.


So how can you tell if the vendor is a good fit?


Here are a few things to keep in mind when evaluating a vendor.


The true costs


For some vendors, cost of the technology is just part of the full price tag. For others, the technology is a ‘loss leader’ (the irresistible bargain that draws you in) to a much more expansive and costly portfolio of ‘gotchas’. When evaluating vendors, evaluate the full costs of doing business.


Other than technology, costs can lurk in places such as:

  • Set-up fees

  • Integration fees

  • Add-on feature costs

  • Add-on user costs

  • Configuration and/or customization fees

  • Maintenance and/or support package fees

  • Upgrade costs

  • Training and/or rollout costs


To get a true sense of costs,

  1. Request an itemized proposal for the full solution. The more specific, the better – detailing how you want the technology set up and rolled out along with who’s using it and how they should be supported. Include future plans or potentials (like more users to accommodate growth or more features to accommodate a new process in the future), so you can see how pricing may change.

  2. Ask for reference accounts. Request to talk to clients that are of similar size, complexity, demand, and/or technology infrastructure. Ask their clients about the true and continuing costs and where those costs are coming from.

  3. See the full price list. If you are not sure about the full solution yet and there are no reference accounts, at the very least understand what they charge and how they charge for it. Ask for a list of all potential fees and how they are applied.


Support


No technology is perfect. No organization is perfect. No individual is perfect. Therefore, there will be times where you or someone on your team using the technology will need help. This could be anything from getting stuck with what to do next, to something not working quite right, to an all out ‘sky is falling’ situation. For any of these confusions, inconveniences, problems, and catastrophes, you will need support.


Vendor support can vary widely. From highly responsive, 24/7, someone is always here for you support, to chatbots and ‘help pages’ that loop endlessly with no human in site. What you need depends on your team’s resources and expertise, but whatever level you need, make sure your vendor has it.


Support questions include:

  • Self-help materials (FAQ pages, user forums, chatbots, etc.).

  • Availability of human support – operation hours and method of communication.

  • Who has access to support.

  • If there are limitations on support usage such as extra costs, or limited hours.

  • Support structure including escalation paths for major issues.

  • Support responsiveness such as expected time to response based on severity of issue.

  • Support guarantees – what happens if you can’t get the support you’ve been promised.


Maintenance


Similar to support, all technology requires maintenance. Maintenance is the changes to the product to fix problems or maintain solutions to current standards such as changes to include the latest security protocols, adherence to the latest privacy laws, capacity expansions to accommodate increased demand, etc.


And, similar to support, Vendor maintenance can vary widely. From highly reactive bug resolution and rapid releases, to annoying bugs that linger and security holes that go unaddressed until disaster strikes.


To evaluate maintenance, look at:

  • Response rates to bugs and issues by severity (i.e. how bad the bug is).

  • Frequency of maintenance and patch releases.

  • Communication policy for known issues by severity.

  • Costs, if any, of maintenance updates.

  • Uptime guarantees.

  • Security and privacy policies.

  • Disaster recovery plans.


In addition to evaluating the materials and policies of the vendor, take the time to investigate through other channels. Look for frequent complaints or problems that have been posted or published to industry or technical forums and check with the better business bureau for official complaints. Keep in mind, not every complaint posted to a site is a sign of a problem. We’ve all seen complaints that are little more than venting of someone having a bad day or are planted as a dirty competitive tactic. Look for patterns and frequency. One complaint about a system being down may just be a fluke or purposefully misleading, but many is likely a window into an undesirable trend.


The product roadmap


Over time, your needs will change. Your organization is constantly evolving, growing, morphing, and reacting. How will your vendor fit into that journey?


Like you, your vendors’ products and technologies are also constantly evolving, growing, morphing, and reacting. From little changes to improve or expand features, to new offerings and technology expansions, to huge shifts in how they fundamentally solve problems. Knowing your vendor’s roadmap will help you evaluate if your journeys are aligned or if they will lead you in far different directions.


Product roadmap questions include:

  • Future plans and areas of investment.

  • Process for submitting product change requests.

  • Process for evaluating and incorporating client requests.

  • Frequency of incorporating client requests.

  • Frequency of minor and major releases and examples of what was in previous releases.

  • Costs for access to minor and major releases.

  • Methods to collect user feedback and advice.


Additional offerings


In addition to the technology, many vendors provide other ancillary services, many of which come at a cost. Depending on your needs, these services may be a necessity to the success of your organization’s implementation and adoption of the technology.


When it comes to additional offerings, start with what you will need and make sure the vendor meets those needs (either directly or through a partner community). Then evaluate everything else the vendor has to offer, as you never know if there’s a nugget that could help distinguish one vendor from another.


Common additional offerings include:

  • Configuration and/or project services.

  • Consultant services.

  • Customization / engineering project services.

  • Product Training – direct or train the trainer.

  • Content or discipline training.

  • User conferences / user forums.

  • Partnership marketplaces with discounts.


Customer experience


Just like with dating, it’s not just about the technical details. Everything can look perfect on paper and yet, the relationship still doesn’t work. Beyond support and maintenance, or products and offerings, there is experience.


How does the vendor treat you? How is the overall experience on the interactions? Are you being respected and heard, or are they rushing you off the phone with empty placations? Are your fundamental values aligned and your interests mutual? When things get difficult (and they will), is your vendor your partner or your adversary?


The challenge with evaluating customer experience is the timeframe. You’re generally evaluating the relationship while still courting (before you sign the deal), when the true experience often comes after commitment (after you sign the deal). Here are a few tips to adjust for this ‘best behavior’ timeframe:

  1. Know the values. Value alignment (or, at least, value understanding) can go a long way in predicting the experience. Values dictate behavior and decisions when things get difficult. Know your vendor’s values and don’t be shy about asking for proof of when they lived those values.

  2. Expect the best. If you’re having any issues during the courtship with responsiveness, respect, or value alignment, this could be a bad sign. If this is them on their best behavior, what happens when they have your money? Note: this is not about if the vendor is willing to accommodate your every request (which is a red flag for a different reason – see the vendor or villain blog), but rath