Third party dependencies — worth the risk?

Most of the industry is liberally using third party dependencies at this point, and I generally agree with this. People might complain about npm dependency hell, and some few companies are still writing everything from scratch, but using (and hopefully contributing back to) open-source software seems to have rightfully won. However, while finding and pulling in a dependency to solve your problem seems easy, it can create a land mine of problems down the road.

This article covers some of those problems that you will run into and approaches you can use to analyze and manage the associated risk as a developer. However, it can also serve as loose guidance for someone in security trying to review dependencies, or someone trying to implement broader improvements around handling dependencies.

Open source is great

Note: For the rest of this article, I'll use open source and dependencies roughly interchangeably, since OSS is by far the most common type of dependency, and many of the discussed issues come from the lack of explicit contract or support around dependencies.

Before we go too far, let's get this out of the way: OSS is pretty cool. Someone else has solved a problem you have, and you can just reuse their solution, while also being able to read, understand, and modify it! I publish almost all of my personal code, and if I had my way, companies would too.

Pulling in dependencies gives you access to huge swathes of talent, including people who are experts in areas you've never heard of. It allows you to focus on core business logic, rather than implementing something unrelated but necessary. Imagine how different the world would look if no one had invented web browsers, and everyone needed to independently implement rendering/networking/etc for their Windows apps! While pulling in a regex crate may not feel like a monumental achievement, it's thousands of hours of expertise that you're able to gloss over and benefit from.

Similarly, dependencies can be used very broadly, putting many eyes on them and giving you confidence that they're correct. The aforementioned regex crate is downloaded more than 200,000 times a day, and more popular tools like React are downloaded almost 20M times per day! Even beyond correctness, the many other people using your dependency tend to have have features they want and are contributing back. Unless you have a huge team of engineers, you probably won't be able to match the development velocity of a large open source project while also meeting your other responsibilities and deadlines.

But it's not perfect

Using open source software isn't all sunshine and roses. The many eyes theory isn't a silver bullet — just because many people are using something doesn't mean many people are looking closely at it. Using more software means more security vulnerabilities (and possibly new types of them) and moving parts that can cause failures, and using a piece of OSS doesn't absolve you of responsibility to due diligence around it.

Sometimes, the many other users can even work against you, such as if you end up with a dependency that's updating and making potentially risky changes faster than you can even keep up with! When you run into this, your code will slowly fall further and further behind upstream, until you're using an old enough dependency that people are unwilling to update out of fear of potential breakages. Note that even if the dependencies update at a "reasonable" pace, you can still end up in this situation if your code enters maintenance mode for a period of time.

Finally, remember that dependencies have licenses, and those licenses may not be palatable to you or your legal department. If you embrace a culture of easily pulling in dependencies, you may also accidentally embrace a culture of violating licenses unless you have a system to avoid this.

Remember your responsibilities

As a developer, it can be tempting to do a quick Google search for your problem, find a dependency that can solve it, and slap that into your project. Because it's so easy to npm install or cargo add, it can be easy to forget that you're taking on a permanent responsibility for that dependency in your project. If you ever need to extend the dependency, you may need to fork it or work with the upstream maintainer to get your new feature added. You may end up in a situation where you need to replace the entire dependency with your own implementation (while not breaking existing code), or you might be responsible for understanding its codebase to extend it. At the end of the day, you're responsible for building systems that do something — using a dependency doesn't change that, you're just hoping to avoid some of the responsibilities that includes, and gambling on whether that'll come back to bite you.

When you're solving a problem, stop to think before pulling in a third party dependency — remember that there may be more ramifications than you'd initially expect, and (as always) consider whether the benefits outweigh the costs. As attractive as it is to delegate responsibilities to a third party, it's important to remember that you're still responsible for the system you're building, which includes all of the decisions of said third party once you pull in their code.

Choosing dependencies

So you, intrepid coder, have decided to ignore my warnings and pull in a dependency anyways? Figuring out whether a dependency will be useful or not is left as an exercise for the reader, but the next step is to estimate how much risk you're taking on from it. Here's a short (and non-quantitative) checklist for evaluating the dependency you've picked:

Alternatives: What are the alternatives to using this dependency? Do you have the skill and time to build it yourself, or are there multiple competing projects trying to accomplish the same thing? You probably won't find a feasible alternative to Linux for a server operating system, but you should consider implementing is-even yourself.

Legality: Is the dependency licensed in a way that you want to comply with? When in doubt, you'll want to check with a lawyer — adhering to the license is the one hard requirement on this checklist.

Sane versioning story: Is it clear from the version scheme whether an update will be safe or not? I'm a particular fan of semantic versioning, but the important thing is that you (or the wretched soul who encounters the depedency) can feel confident updating it. If it's not obvious when breaking changes have been introduced, you'll always be walking on eggshells and wondering if your tests are sufficient to validate a version bump (they aren't). If it doesn't feel safe to update, your dependency will eventually rot.
Note that promises around backwards compatibility are a sufficient alternative to sane versioning. For example, Linux can't be accused of having a sane versioning system, but you can certainly trust that they won't break userspace.

Other users: If only a few other people are using a dependency you rely on, you might as well consider yourself the owner of it. You ideally want a decent-sized community to run into and fix bugs, iron out API awkwardness, and check for obvious security holes. Some problem spaces are less common than others, and you may find yourself in a situation where your options are to built it yourself or to use a specific commit from a repository with one contributor and no other users. You should recognize that in those situations you'll have no real alternative to owning and understanding the dependency if anything goes wrong, and I'd recommend avoiding them if you don't have the time budget to solving most of the problem yourself.

Open issues and PRs: The dependency you're looking at is probably hosted on Github, Gitlab, or some similar alternative. Can you piece together a story around how actively it's maintained from the set of open issues and PRs? If you need to make any changes, you'll want a team of maintainers that are active enough to at least merge your changes upstream, otherwise you've accidentally purchased a one-way ticket to Forkville. Aside from how maintainable it is, does it look like people are having big issues with it? If you see issues saying that it sometimes deletes everything on your machine, you should probably be a lot more concerned than if it's mostly a list of feature requests.

Feature set: Does the dependency try to do one thing and do it well? Open source projects with a targeted goal are much easier to predict and trust than projects that have a grab bag of functionality. Does it have (or look like it might add) crazy features that introduce unnecessary bloat or risk to your system?
Aside: I appreciate how cargo natively supports optional features, making it common to pull in only desired code in Rust projects.

Reported CVEs: Does the dependency have reported CVEs? If so, how many are there, and do they look like fundamental issues or rare mistakes? For example, Imagemagick is probably better at delivering exploits than parsing images, and will probably always be risky since it's parsing complex formats in C. In contrast, I'd probably use doas even with a few CVEs, since they seem to be smaller mistakes in a generally well-architected program.

Skim the code: When you're pulling in a dependency, you're hopefully planning to save a bunch of time you'd otherwise spend implementing it yourself. Spend some of that time skimming through the code to get a bit of a sense for it! I like to spend ~15mins looking through code that does parsing/networking/other scary things to get a sense for how much the existing contributors have thought about security. Doing the same quick passes around more central pieces of the dependency can give you some signal on the code quality, as well as how much work it might be if you end up needing to maintain a fork.

Consider where you're using it: Does this dependency handle a core competency that you shouldn't be outsourcing to an external dependency, or will it be used in a system that needs to be secure or reliable? There are situations where you may have higher standards than an external dependency can be expected to uphold, and explicitly checking for those can be useful.
That being said, take this criteria with a grain of salt. It's often even easier to reuse a dependency elsewhere in a system than it is to introduce it into the system in the first place, and the reasoning you initially used will rarely carry over. If you're feeling hesitant even though the current use case checks out, you probably have a good reason.

There are lots of other things you can look at — you could even do a thorough code review! However, I think this set of criteria provides a reasonably comprehensive assessment for the time spent. After working through these questions, you should have a better sense of how risky it might be to use the dependency, and can weigh that against the benefits using it would provide.

Read label before use

Now that you've evaluated a dependency and decided to use it, there are still a few few other things worth thinking about before calling it a day:

Document your analysis: You did some work to choose the dependency, including considering the benefits, alternatives, and risks. Documenting that will allow someone else to follow in your footsteps, as well as make it easier for others to check your work.

Have a concept of ownership: Someone should be responsible for the dependency, whether that's a person or a team. When breaking changes are released or a vulnerability is discovered, it will be key to have someone who feels confident coordinating the next steps. If nothing else, having an owner gives other people a point of contact for questions about it.

Have an update strategy: Dependencies that aren't maintained aren't an asset — they're a liability. You need some system to respond to known security patches at a minimum, and should probably have much more. Ideally you can automate this process and tie it into tests, allowing your dependency to always be up-to-date. If that's not feasible, determine whether you can commit to keeping updated to the latest minor or major version.

...and know when to update: It's not enough to have a good strategy around updating if you don't know when to update. At a minimum, you should have some way to know when your dependency is vulnerable to a publicly disclosed exploit. You also may want to know when significant new versions have been released, either to manually update or to give people a heads up that your automatic updates will be picking up the new version.

Consider compartmentalizing it: Can you sandbox or otherwise compartmentalize this dependency? There are a lot of approaches here with varying costs and benefits. Wrapping it with your own API may make it easier to replace if it becomes needed, while putting it into an isolated process or on separate hardware may significantly reduce your exposure to security vulnerabilities. Taking any steps here can reduce the risk you take on by adding the dependency.

These tasks should probably be handled in a standardized fashion — having a single process that can cover all dependencies makes a lot of sense because of scaling. For example, if there's a single location tracking all dependencies, you might get an SBOM for free. Similarly, it's a lot more efficient to have a single automated updating system than to have each dependency owner figuring something out on their own.

No forking way

There may come a time when someone suggests forking a dependency in order to add features or fix bugs. They'll probably say this is a temporary workaround, and that the changes will either be upstreamed or that the new release will obviate the need for them. If you believe this, you may want to consider investing with Bernard L. Madoff Investment Securities. You'll probably end up maintaining the fork forever, or at least until it becomes a big enough problem that a major project is staffed to rip it out.

Unless you're prepared to independently maintain a dependency (including its docs and tests) for all eternity, don't agree to fork it. It will inevitably diverge from upstream, require urgent fixes and features, and delay projects that need to deal with it. Anyone who needs to interact with it in the future will probably hate you, including your friendly neighborhood security team whenever they need security patches applied.

Think before you leap

It may not seem like it, but I'm actually pro-dependency. For better or worse, you probably can't imagine a world where your code doesn't run on a Linux-based server, and there are thousands of other examples of how the entire software industry sits on top of open source software.

With that in mind, be careful when using dependencies — they always pull in some risk, and the benefits they provide don't always out-weigh those risks. A modicum of consideration around the associated risks and ways to manage them may save you substantial time and effort in the future.