Vendor Security Assessments Are Out of Control

Assessments and questionnaires are slowing down business, yet there’s no evidence they improve security

Your security team has spent hours this week alone on security questionnaires despite your organization’s unpatched systems, unfinished and untested disaster recovery plans, and unencrypted backups. Sound familiar?

Security analysts’ desks are awash these days with the spreadsheets, Word documents, and even custom apps organizations use to quiz prospective vendors about their security practices before buying a product. That product is usually an online app, known in the industry as software as a service (SaaS).

Why do organizations perform vendor assessments? They believe they have an ethical and/or legal obligation to protect their customers’ data. They also know that if the third party suffers a security incident that exposes that data, their customers (or regulators) might blame them, not just the third party.

There are already standards for information security that vendors can adhere to and get audited against, such as NIST SP 800-171, ISO 27001, and SOC 2. There are even readily available, battle-tested assessment tools, such as Cloud Security Alliance’s Cloud Controls Matrix (CCM) or the Higher Education Community Vendor Assessment Toolkit. These instruments are all meant to engender trust between organizations, and they’re good tools.

But despite these many yardsticks against which to measure a security program, many organizations promulgate their own methods of determining the answers to mostly the same questions. Do you have a security policy? An incident response plan? Encrypted, offsite backups? And so on.

As a result, every vendor now must answer innumerable distinct questionnaires if they want to close deals. Each customer organization has to review these answers and maybe even bicker with the vendor until the answers are acceptable. Whole teams exist at larger organizations for the sole purpose of sending out security questionnaires and reviewing the responses. Other teams exist to answer questionnaires from their customer organizations.

Many of us in the security space have managed both kinds of teams, often at the same time. Throughout it all, we ask ourselves: given the cost, how have security questionnaires reduced risk? Has a vendor assessment ever prevented an incident? From what I can tell, no research has been done on the topic. I asked you, my colleagues, if you had any evidence or even a reason to think this monotonous task is a better use of personnel than, say, running tabletop exercises to test your disaster recovery plan, or doing self-assessments to ensure your product observes the OWASP Top Ten.

No one has seen any such evidence. Yet we continue on this same circular path.

Why do we do this?

Standards and Regulations

The most common justification for vendor assessment programs is that standards and/or regulations require it. Most of the time, though, the requirement is a lot less rigorous than what we choose to implement. For example, ISO 27001 merely states that “[p]rocesses for acquisition, use, management and exit from cloud services shall be established in accordance with the organization’s information security requirements.”¹ There’s no mention of lengthy questionnaires, time-consuming reviews, or audits, unless the organization decides to include such measures in their process. Something much lighter-weight would suffice to fulfill this requirement.

Misplaced(?) Trust in On-Premise Systems

Other explanations for such rigorous assessments are rooted in human nature. We often suffer from the illusion that control is the same as safety. This fallacy can lead to a small- to medium-sized company, for example, believing that data in their own data center maintained by their own team is better protected than if entrusted to companies with immensely more resources, such as Google, Amazon, or Microsoft. Not all SaaS service providers are as well resourced, but the point is we can’t just assume all SaaS is inherently unsafe.

The only reasonable stance is to accept that both on-premise and SaaS approaches have their own risks and benefits, and to make an objective risk calculation for every use case.

The Psychological Comfort of “Doing Something”

A third reason extensive vendor assessments have persisted is that any situation feels safer after a thorough review and write-up. If you deliver a 50-page document on the security of a SaaS service before your company starts using it, and it later gets compromised and exposes company data, at least no one can blame you for the incident.

That reasoning may be sound, but it doesn’t take into account the many other times you reviewed SaaS services and made your company wait for your review before proceeding with business. So the feeling of security comes at an unquantifiable cost.

Areas of Proven Value

Some might accuse me of wanting to do away with vendor assessments altogether. But I’ll be the first to admit these processes do offer a few benefits:

1. A comprehensive inventory of cloud services. Such an inventory is especially useful when one of those services has a security incident, and you need to determine whether it affects your organization. It also causes you to keep security contact information for that vendor at the ready.
2. A chance to ensure the data on your end is properly safeguarded, before it goes into the cloud service and after it comes out. Many organizations seem to suffer from some sort of cognitive bias in this regard, intensely scrutinizing the vendor’s controls while neglecting to analyze their own. An important component of this view is the exit strategy—how you’re going to get all your data out and continue doing business when (not “if”) you part ways with the vendor.
3. A spotlight on any security features in the product. You can use them to help with #2 above, safeguarding the data, and you may be able to demonstrate to leadership why these (often paid) features are worth pursuing.

Beyond these specific areas, it’s hard to show the value of attempting to dig into a vendor’s security controls. But it’s important to know these areas of value because we can refine our processes. Get rid of the cruft and keep the stuff that works. You know, an evidence-based approach!

Evidence-Based SaaS Assurance

As a vendor, you can’t do much to shrink the heap of questionnaires coming your way except to hope your customers follow the guidance below.

But as a customer, you can reduce your vendor assessment workload to what we know works:

1. Check your requirements set by customer contracts and standards you certify against, and ensure you’re fulfilling them. Odds are the steps below cover them, but there may be special cases, such as how quickly you need to notify customers after a security incident.

2. Document. Maintain an inventory of SaaS services just like you would your servers, including

   • a vendor security contact
   • your organization’s use case
   • who in your organization has access to the administrative account
   • a backup for that person
   • what data types and classifications are to be stored in the service
   • an exit strategy (all relationships come to an end, after all) This information will really come in handy when you learn the vendor has had an incident. Update it at least annually.

3. Focus on security features, not controls you can’t validate. You don’t know if the answers they’re giving you are true or meaningful. And in the case of major providers, their safeguards might be better than you’d implement in your own data center. But do chase down security features offered by the service:

   • ability to integrate with your single sign-on system
   • multi-factor authentication
   • (if not the two above) brute-force alerting & prevention
   • log access and/or export
   • the ability to set retention periods for data stored in the service

   Revisit the list of available security features at least annually. Some of these may cost extra. I recently dealt with a company that charged extra for finite retention periods. The free version would have kept my data forever; I had to pay them to periodically delete it.

4. Ask the vendor for evidence of the good work they’ve already done, such as their ISO 27001 certificate, which should include a scope statement so you know what portion of their company underwent scrutiny. Ask again annually as that’s how often they’ll typically be audited to re-up the cert.

5. Ask your organization’s staff how they’ll protect the data before and after it goes into the cloud service. Are you harping on the vendor about encryption at rest when the same data is sitting in an Excel file unencrypted on your colleagues’ laptops?

With this lightweight, evidence-based approach, you can reduce risk without impeding business, and you may even free up some of your security team’s cycles for threat hunting, tracking new threats, or building tools to support the organization’s security program.