The Challenge
Verizon develops and delivers dozens of mobile applications across an ever-expanding and complex global marketplace. These include popular consumer products such as Yahoo Finance, Yahoo Fantasy Sports, and Yahoo Play, as well as B2B products for advertisers and app developers. Mobile apps present a challenge when it comes to maintaining open source license compliance. They are composed of many libraries, which, in turn, are composed of many, many more libraries.
Much of the underlying ecosystem used to develop mobile apps is open source. Thus, when using open source components in an app, an app publisher is obligated to disclose the licenses and copyright information associated with those components. This can be challenging with one or two apps and daunting with dozens (no less hundreds) of apps to manage.
The Solution
To ensure Verizon’s apps are high quality, a dedicated team known as the Mobile Excellence Team measures how fast the apps load, how much space, battery, and network bandwidth they use, how they look on dozens of devices, and, of course, what libraries they use. Measuring is essential to improving things.
The team selected FOSSA to help manage open source license compliance across multiple products and multiple business units, including Verizon Media and Verizon Wireless. FOSSA provides analysis for a portfolio of mobile apps, all compiled during the build. By integrating deeply into the CI/CD process, FOSSA helps identify what’s really in the code, not just what should have been in the code.
But implementing a new tool across dozens of engineering teams could have been a huge challenge. Different teams operate slightly differently, and that makes integration challenging. To address this, the Verizon's mobile excellence team standardized its builds process by using Screwdriver, an open source CI/CD platform built in-house by Yahoo. The standardized tooling improves overall build quality and consistency. Moreover, FOSSA made it easy to leverage Screwdriver’s simple configuration files and implement across all the mobile engineering teams at once, despite Screwdriver's customizations.
"Integrating FOSSA within the build process for Verizon mobile apps not only ensures apps are compliant with open source licenses but also generates automated credit reports that we include in each app."
Gil Yehuda, Sr. Director of Open Source
The Results
Implementing FOSSA across Verizon was straightforward. Nearly all Verizon's mobile apps are built using Screwdriver, which treats build-instructions like code. After testing and a few minor configuration changes to the existing iOS and Android build scripts, FOSSA was able to start running during all builds across many apps, all within days — as opposed to the weeks or even months typically required for scanning at Verizon's scale. The team set up two scripts: one that allows teams to run FOSSA on demand and another that sets FOSSA to run by default on test and production builds. This way, new apps can work out any issues iteratively before setting FOSSA to run by default.
"By using FOSSA, Verizon can deploy software at scale with confidence. Continuous integration, continuous delivery, and continuous compliance are required for any product to provide value."
Mani Subramaniam, Software Development Engineer
After implementing FOSSA, Verizon scanned hundreds of projects, as well as thousands of builds, dependencies, and unique dependencies. This proved the solution could scale. Automating open source license compliance with FOSSA saved both the legal and engineering teams massive amounts of time, resulting in at least hundreds of thousands of dollars in indirect costs, in addition to the direct savings as a result of lower risk. For example, instead of manually auditing open source packages and associated licenses project by project, FOSSA automatically scans and validates the associated licenses during each build.
"Integrating FOSSA as part of the CI/CD process, Verizon can detect licenses for open source dependencies in real time."
Balaji Som Singh, Director of Systems Engineering
