Vulnerability Remediation Tactics

You've found a vulnerability in a third-party component that your code uses. You're confident that it's really present and exploitable. You've done all your due diligence, and you want to get rid of it. How do you do that?

Generally, if a fix is available, there are two approaches you can take here:

1. You can patch the component in-place.
2. You can upgrade to a component version that does not have the vulnerability, by either:
   a. Pinning the vulnerable component to a fixed version.
   b. Doing iterated component upgrades until the vulnerable component has been removed from your dependency installation plan.

Patching In-Place

If you have a patch available for the version of the component that you use, you can always apply the patch in-place. This usually means taking a diff and applying it directly to the component's code as part of your build, and then building against the patched component.

Pros

• This generally requires the least amount of application-level changes, as long as the patch roughly preserves the semantics and behavior of the original functionality that you relied on.

Cons

• You need to have a patch available. This is pretty rare, unless you have a dedicated security team that also builds patches.
• Applying patches is finicky. How do you get the patch to persist between builds? Usually this needs some build process hackery, whether that's applying the patch as part of the build or checking in and vendoring the source code of the component, or publishing your own patched variant of the component.
• Patches sometimes require application-level changes anyway, especially if they impact the performance or semantics of the original vulnerable functionality, or if you were relying on implementation details.

Upgrading Components

If you don't have a patch available, you'll need to upgrade to a version of the component that is not impacted by the vulnerability.

Direct Dependencies

For direct dependencies, this is quite easy. Usually it involves changing your version constraint for the direct dependency to exclude vulnerable versions.

For example, let's say we have a project with a dependency named A that has versions 1.1, 1.2, 1.3, and 1.4. We want to remove a vulnerability found in A, version 1.3. Here is an example change we'd make to the project's dependency requirements:

{
  "dependencies": {
    "A": "^1.0"
  }
}

{
  "dependencies": {
    "A": "^1.4"
  }
}

The new dependency constraint excludes the vulnerable version of the package. Now, when you rebuild your project, your build tool will never pick the vulnerable version of A.

This is the key idea behind upgrades: to resolve a vulnerability in the current version of our components, we must change our builds to exclude all vulnerable versions of those components.

Aside: Dependency Compatibility

This gets a little more complicated if your direct dependencies depend upon each other. For example, let's say we're using the same dependency A as before, but now we're also using direct dependency B, and B depends upon a specific version of A.

{
  "dependencies": {
    "A": "~1.1"
  }
}

{
  "dependencies": {
    "A": "^1.0"
  }
}

{
    "dependencies": {
      "A": "^1.4"
    }
}

Now what change do we need to make to resolve the vulnerability in A@1.3?

{
  "dependencies": {
    "A": "^1.0",
    "B": "~1.1"
  }
}

We can no longer upgrade A without also upgrading B! Why? Because this is impossible:

{
  "dependencies": {
    "A": "^1.4",
    "B": "~1.1"
  }
}

This configuration is impossible because we are requiring a version of B that satisfies the constraint "~1.1". The only version of B that satisfies this constraint is B@1.1, which requires A@~1.1. But the requirements for A@~1.1 (coming from B@1.1) and A@^1.4 (coming from our project) are impossible to simultaneously satisfy.

So we must upgrade both A and B simultaneously:

{
    "dependencies": {
      "A": "^1.0",
      "B": "~1.1"
    }
  }

{
  "dependencies": {
    "A": "^1.4",
    "B": "~1.3"
  }
}

Notice a couple of subtle points here:

1. We could have also chosen to upgrade B to version 1.2. Even though B@1.2 depends on A@^1.0, the combined constraints of both A@^1.0 (from B@1.2) and A@^1.4 (from our project) exclude the vulnerable A@1.3. (Remember, the selected dependency versions must satisfy all constraints from the project and from all other dependencies.)
2. If our project had depended only upon B version 1.2, we would still be potentially vulnerable, because we would not have excluded A@1.3 from being picked.

When your build tool solves your dependency constraints in order to pick concrete versions of dependencies to install, the resulting list of (dependency, version) pairs is called your dependency installation plan (sometimes called an "install plan" for short).

A safe application install plan does not imply a safe component install plan, and vice versa.

Our example above shows that an application's install plan can be safe even when some of its components' install plans might not be. The reverse is also true for a different reason: the application build might pick a different component version, and different component versions can have different dependency constraints. Consider this alternative universe:

{
  "dependencies": {
    "C": "^1.0"
  }
}

{
  "dependencies": {
    "A": "~1.3"
  }
}

{
    "dependencies": {
    "A": "~1.4"
  }
}

If you build B@1.1, your build tool will probably resolve B's dependency constraints to the install plan: B@1.1, C@1.2, A@1.4.

Does this mean B@1.1 is safe to use in your application? No:

{
  "dependencies": {
    "C": "~1.1"
  }
}

{
  "dependencies": {
    "A": "^1.0",
    "B": "1.1",
    "D": "1.1"
  }
}

B@1.1
D@1.1
C@1.1
A@1.3

Notice that relying on a "safe" version of B still produced an install plan that included a vulnerable version of A.

To be truly safe, you need to know the actual install plan of your application. The existence of a safe install plan for a component does not imply that the component is always safe to use, because other components may add their own constraints to your application's build. Components are only safe if all of their possible install plans (including all possible install plans of their own transitive dependencies!) always exclude vulnerable components. Unfortunately, calculating such an exclusion requires enumerating install plans, which is NP-complete.

Upgrading Transitive Dependencies

Upgrading transitive dependencies is more complicated, because we normally don't set their versions or provide version constraints for them directly.

Adding a Dependency Constraint

One easy hack is to just add a direct dependency constraint on the transitive dependency. For example, let's consider a universe where a dependency A has versions 1.1 and 1.2, and A@1.2 is vulnerable:

{
  "dependencies": {
    "A": "^1.0"
  }
}

{
  "dependencies": {
    "B": "1.1"
  }
}

{
  "dependencies": {
    "B": "1.1",
    "A": "1.1"
  }
}

In this example, adding the direct dependency constraint on A causes the build tool to reject the newer vulnerable version of A. In this particular case, we resolve the vulnerability using a downgrade rather than an upgrade.

Some subtler points with this tactic:

• Adding dependency constraints can cause downgrades, but won't always. Sufficiently complex dependency constraint graphs often have many possible install plans, and build tools generally heuristically search (using backtracking) for the first install plan they can find. Sometimes, adding a constraint can result in the build tool going down a different path when searching for install plans, and can create a very different transitive graph that does not necessarily involve upgrades.
• This example adds a dependency constraint that specifies a concrete version, but you don't have to do that. You can also specify a version constraint that merely excludes the vulnerable component versions. Some tools may also allow you to disallow certain components altogether, and sometimes they will be able to find an installation plan that excludes every version of a component (usually because they select different versions of other transitive dependencies that have entirely different dependencies).

Pros

• It's really easy. Try it!

Cons

• You might be downgrading some components.
• It doesn't always work. Like we've seen before, some sets of dependency constraints can be impossible to satisfy together.

Overriding the Dependency Version

Rather than adding a new constraint, many tools allow you to override the version of a dependency. This behavior is different from "satisfy a new constraint,", because it allows you to pretend you've installed one version of a package, but have secretly installed another.

For example, NPM has overrides:

{
  "dependencies": {
    "A": "^1.0"
  }
}

{
  "dependencies": {
    "B": "1.1"
  }
}

{
  "dependencies": {
    "B": "1.1"
  },
  "overrides": {
    "A": "1.1"
  }
}

Other tools have similar equivalents. For another example, check out our blog post "Overriding Dependency Version and Using Version Ranges in Maven."

Pros

• This is also easy to do.
• This is always guaranteed to produce a build of some sort. It can never fail to produce an installation plan, because you're manually bypassing the dependency resolution process.

Cons

This is dangerous, and can produce subtly broken builds. When you override the dependency resolver like this, the onus is on you to make sure that the components in the resulting install plan are actually compatible! In the best case, you might find that your install plan cannot compile (e.g. if the version you're overriding exports functions with incompatible type signatures). In the worst case, you might find subtle logic bugs resulting from code that compiles but expects different semantics.

Iterated Upgrades

If your build tool can't find an install plan without changing your other existing dependency constraints, and you're not willing to take on the significant risk of overriding a dependency version, then you'll just have to do it the hard way.

Doing it the hard way means updating your existing dependency constraints until you have a working build that also excludes the vulnerable version of the component. For an example, check out our earlier discussion around dependency compatibility in the “Upgrading Components” section.

Your upgrade will look something like this:

{
  "dependencies": {
    "A": "^1.0",
    "B": "~1.1"
  }
}

{
  "dependencies": {
    "A": "^1.4",
    "B": "~1.3"
  }
}

Why do we call this tactic "iterated upgrades"? Because it's difficult to predict exactly what changes you need to make in order to get a working install plan (remember, NP-complete). Instead, you have to fiddle with your dependency constraints, re-run your build, and then repeat until you find a working build.

Pros

• This option is safe. You're not overriding anything, so the build tool will make sure that the dependency requirements of all of your components are satisfied.
• It usually doesn't take that long in practice. You can often heuristically find a working set of dependency constraints quickly by adding your transitive dependency constraint, trying the build, upgrading the first constraint that breaks the build, trying the build again, and then repeating until you find a working install plan. For serious vulnerabilities, it often also helps to examine the CHANGELOGs of components, since they might have a specific version whose dependencies are chosen to avoid the vulnerability.

Cons

• In the very worst case, this approach could take a long time. The space of possible installation plans is very large.
• The more realistic concern is that you might discover that you need a high-effort upgrade. For example, you might need to bump a major version of a widely used library (e.g. your framework or your ORM) in your codebase. There's really no silver bullet in these scenarios: you'll need to either patch the component yourself, spend the effort to do dependency upgrades, or accept the risk of having a vulnerability.

Related Reading

Hopefully, this post is useful to you as you consider the best approach to fix vulnerabilities in your third-party components. If it is, consider also reading some related posts from our team here at FOSSA:

• Three Pillars of Reproducible Builds about designing builds that are reproducible and therefore less vulnerable to compromise.
• Understanding and Preventing Dependency Confusion Attacks about mitigating software supply chain attacks.