Vendor assessment checklist [by Jobresponse]

The vendor assessment stresses the importance of outlining your project specification beforehand. We’re discussing the approach to the project specification in the MVP Scope of Work [by Jobresponse] note. But approaching the vendor assessment would also require you to think strategically about your project. Namely, you should know your project goals. As we talk a lot about the MVP — the MVP itself defines its goals. It’s the project with pre-defined initial data, like three months Go-To-Market, built from scratch (no legacy code), and you don’t have a large team of stakeholders at the moment of the startup ideation. Overall this checklist fits most of the outsourcing and outstaffing scenarios, whether startup or established enterprise.

Because a vendor assessment is a strategic business goal, ensuring your checklist includes components that guarantee the results is vital. Any mistakes made here unfold into a long and brutal process later. Are you willing to take any tough calls? This checklist will give you the knowledge to stay out of trouble.

Again, you should think but not overthink at this stage. What’s good for a large enterprise is not always good for the startup MVP. You don’t need big names or vendors with complicated processes of software design and development. Simplicity wins, but there is always hard work behind any simple thing.

Scratch the surface

This is your homework exercise to surf around the internet and get a list of potential vendors that seem to be fit for your project. There are myriad software agencies, and you need to narrow down your search against your criteria. Your initial match should go on high-level inputs. For example, if you need vendors that develop MVPs in your domain, whether it’s SaaS or healthcare — look for vendors with this particular background.

Pro tip: Don’t brush off a vendor that hasn’t done anything specifically in your industry or domain. Lots of software development vendors are industry-agnostic, and they might have a successful blueprint that is widely adopted across industries. But surely, if your industry is quite specific and has sensitive parts, like patient data in healthcare, it is recommended that you look for vendors in your particular domain.

While this exercise won’t give you holistic reviews about potential partners, you can still find some useful information like customer reviews on websites like Goodfirms or Clutch, and assess the vendors’ profiles. Most vendors have case studies on their websites. Those can give you good insights you can also refer to during the interview.

Get your vendor list ready and send out the emails. It is recommended that by that stage, you have something to send out about your project, like MVP Scope of Work [by Jobresponse], but it is not required to start the conversation. The following section outlines the vendor interview and assessment in detail.

Vendor interview questions

Most of the time, the first meeting is the meeting with the sales. You are not going to see the whole team or meet engineers and designers ready to work on your idea. However, some vendors may bring engineers, but this is a red flag as they want to push you to make a purchase decision. In the best-case scenario, you’ll talk to the CEO with a technical background — this is great.

The vendor interview questions are important, but what’s more important is to read the non-verbal signals. Interview with a great sales is not a one-way street. They also assess the same things you do. If the sales team asks you the right questions, educating you and making you think differently — this is a strong signal you’ll want to account for in your assessment. It is in the partner’s interest to ensure their investment is worth it — yes, their time costs — and a great partner is open to sharing expertise in certain areas, like remote team management, organizing meetups, employee training, and employer branding, to name a few.

Questions

Introduce yourself and your team to get the conversation going. If you sent the project details beforehand, the other party should know the context, but a small project intro would help to get everyone on the same page.

• Ask a vendor to introduce their team and company. In the company profile, you want to know some basic information like headcount, location, domain expertise, and years in business.
• Did a vendor work on similar projects? What the process looks like? If the vendor didn’t work on similar projects, ask what they think the process of software development would look like.
• Dive into the project details. You can ask what some of the tricky parts of the project they recognized on the spot are. Are there any challenges? What tech stack do they propose to work with? How the acceptance testing goes? Talk about the parts of your project that matters most.
• What is the proposed team configuration? What roles do they think should work on the project? How and in what timeline a vendor allocates the team? Potential project start date? Are engineers and designers speak English? How does their assessment work? How do they hire engineers, and how do they make sure it’s a fit for the project? Are all team members full-time employees? Do they use contractors or freelancers?
• What potential project timeline?
• What potential ballpark estimate? Or how to approach the estimate and get the budget? What are the hourly rates per each role? What are the team’s monthly rates?
• How the project communication usually goes? What tools are they using? When are the sync-ups and standups?
• How the project accounting usually goes? What kind of reports are there? How does the billing work?
• Questions about the contract (good contracts seen include sections like intellectual property, rights to code, innovation, employee solicitation, etc.)
• What are the next steps in their process? How do we go to the next steps with our project?

These are basic questions to ask a vendor, but there are also some additional questions you may include in your conversation:

• Would it be okay if we treat the team like our team? Could we work together to effectively create incentives, perks, or even bonuses? Can we include the team on our investors’ pitch deck?
• Try asking a few tricky questions. What if we don’t like the engineer? What if we don’t see their commits pushed to the branch? What if something goes wrong, like the project takes a longer time to deliver, or some of the implementations wouldn’t work as intended?
• What is some other stuff that a company does? For example, some vendors run tech conferences, engineers give keynotes, develop engineering courses, open-sourcing software, build side projects and internal tech tools, and attend hackathons.
• Does the vendor provide training, knowledge transfer, and hiring assistance? Even if you don’t need these services in the first place, a great vendor can help with additional services to ensure your business gets max results.

Thank a vendor and ask if there is something that you can answer. Let the vendor know how your process looks, what and when to expect the next touch-points.

Reading signals

Let’s discuss how to read signals during the interview. Remember that the other party may not have all answers right away, like the team availability, timelines, or budget. Your goal is to listen to the answers and mark your assessment list with red flags.

• If the vendor calls the price or timeline right away, this may not be true. Definitely a red flag.
• If the vendor proposes to jump into the development phase right away — it’s a red flag.
• A vendor should have a process in place to prevent the wrong things from happening during the development phase or maintenance or ongoing development — if they don’t have an answer, it’s another red flag.
• If the vendor brings engineers to the intro call — it’s another red flag.
• If the vendor tells you that they have a project manager who will run all the communications — most likely, it’s a red flag as you won’t have direct access to the team.
• If the vendor assures you that the project won’t have any issues — it may be a red flag. All projects have issues, and the vendor’s job is to be transparent about how things are going at a given moment. If it feels like the team is not transparent enough, it’s a red flag.
• In some cases, contractors or freelancers on the team pose a risk. Red-flag this answer.

After completing the assessment process, you should have a good set of data to make a well-informed decision. Compare the data and discuss it with your team. Choose a winner and move forward with the project. It’d be great to let other vendors know that they didn’t make it to the next steps and thank them for their effort.

The next best step is Design Sprint [by Jobresponse], but don’t red-flag it if the vendor doesn’t talk about the Design Sprint. Instead, the vendor may operate with a different definition. If it still feels like the same concept, you’re safe to proceed.