When managing Go projects, you need to track dependencies, check for vulnerabilities, and ensure license compliance. Instead of relying on external tools, let’s build our own dependency analyzer using Go’s standard library.
The Core Structure
We’ll work with Go modules, so we need structures to represent them:
package main
import (
"bufio"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"regexp"
"sort"
"strings"
"time"
)
type Module struct {
Path string
Version string
Indirect bool
}
type GoMod struct {
Module Module
Requires []Module
}
Our tool will handle three operations: listing dependencies, vulnerability scanning, and license checking.
Parsing go.mod Files
Understanding Module File Structure
The go.mod file uses a specific format that we need to parse correctly. Module declarations start with the module keyword followed by the module path. Dependencies are listed in require statements, which can be single-line or grouped in multi-line blocks.
The parsing logic handles both formats by tracking whether we’re inside a multi-line require block. We use regular expressions to extract the module path and version from each line, and detect indirect dependencies by looking for the // indirect comment. This approach gives us the same information that go list would provide, but without spawning external processes.
Rather than shelling out to go list, we can parse the go.mod file directly:
func parseGoMod() (*GoMod, error) {
file, err := os.Open("go.mod")
if err != nil {
return nil, fmt.Errorf("go.mod not found: %v", err)
}
defer file.Close()
goMod := &GoMod{
Requires: []Module{},
}
scanner := bufio.NewScanner(file)
inRequire := false
requireRegex := regexp.MustCompile(`^s*([^s]+)s+([^s]+)(?:s+//s*indirect)?`)
moduleRegex := regexp.MustCompile(`^modules+(.+)`)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if strings.HasPrefix(line, "module ") {
if matches := moduleRegex.FindStringSubmatch(line); len(matches) > 1 {
goMod.Module = Module{Path: matches[1]}
}
}
if strings.HasPrefix(line, "require (") {
inRequire = true
continue
}
if inRequire && line == ")" {
inRequire = false
continue
}
if inRequire || strings.HasPrefix(line, "require ") {
cleanLine := strings.TrimPrefix(line, "require ")
if matches := requireRegex.FindStringSubmatch(cleanLine); len(matches) >= 3 {
module := Module{
Path: matches[1],
Version: matches[2],
Indirect: strings.Contains(line, "indirect"),
}
goMod.Requires = append(goMod.Requires, module)
}
}
}
return goMod, scanner.Err()
}
The parser handles both single-line requires and multi-line require blocks. It extracts module paths, versions, and identifies indirect dependencies.
Vulnerability Database Queries
How Vulnerability Checking Works
Vulnerability databases maintain records of known security issues in software packages. Each vulnerability gets assigned identifiers like CVE numbers and includes details about affected versions. The process works like this: we send the package name and version to the database API, it checks if that specific version has any known vulnerabilities, then returns a list of issues if found.
The OSV database is particularly useful because it aggregates vulnerability data from multiple sources and provides a unified API. When we query it, we’re essentially asking “does this exact version of this package have any reported security problems?” The database performs version matching and returns structured data about any findings.
We can check the OSV (Open Source Vulnerabilities) database for known issues:
func checkOSVDatabase(modulePath, version string) []string {
url := "https://api.osv.dev/v1/query"
payload := map[string]interface{}{
"package": map[string]string{
"name": modulePath,
"ecosystem": "Go",
},
"version": version,
}
jsonData, err := json.Marshal(payload)
if err != nil {
return []string{}
}
client := &http.Client{Timeout: 10 * time.Second}
resp, err := client.Post(url, "application/json", strings.NewReader(string(jsonData)))
if err != nil {
return []string{}
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return []string{}
}
var result struct {
Vulns []struct {
ID string `json:"id"`
Summary string `json:"summary"`
} `json:"vulns"`
}
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return []string{}
}
var vulnerabilities []string
for _, vuln := range result.Vulns {
vulnStr := fmt.Sprintf("%s: %s", vuln.ID, vuln.Summary)
vulnerabilities = append(vulnerabilities, vulnStr)
}
return vulnerabilities
}
func checkVulnerabilities() {
goMod, err := parseGoMod()
if err != nil {
fmt.Printf("Error: %vn", err)
return
}
vulnerableModules := 0
for i, mod := range goMod.Requires {
fmt.Printf("rScanning %d/%d: %s", i+1, len(goMod.Requires), mod.Path)
vulns := checkOSVDatabase(mod.Path, mod.Version)
if len(vulns) > 0 {
vulnerableModules++
fmt.Printf("n🚨 %s@%s:n", mod.Path, mod.Version)
for _, vuln := range vulns {
fmt.Printf(" - %sn", vuln)
}
}
}
if vulnerableModules == 0 {
fmt.Println("n✅ No known vulnerabilities found")
} else {
fmt.Printf("n⚠️ Found %d vulnerable modulesn", vulnerableModules)
}
}
The vulnerability checker sends a JSON payload with the module name and version, then parses the response for any reported vulnerabilities.
License Information Fetching
How License Detection Works
License compliance checking involves identifying what legal terms govern each dependency in your project. Most open source projects include license files in their repositories, and platforms like GitHub parse these files to identify the license type using SPDX identifiers.
Our approach leverages GitHub’s license detection API, which analyzes repository contents and returns standardized license identifiers. For modules hosted on GitHub, we extract the owner and repository name from the module path, then query GitHub’s API endpoint that specifically provides license information. This gives us machine-readable license data without having to download and parse license files ourselves.
Different licenses have different requirements , some like MIT are very permissive, while others like GPL have copyleft requirements that might affect how you can distribute your software. Understanding these differences is crucial for legal compliance.
For GitHub-hosted modules, we can get license data from their API:
func fetchGitHubLicense(owner, repo string) string {
url := fmt.Sprintf("https://api.github.com/repos/%s/%s/license", owner, repo)
client := &http.Client{Timeout: 10 * time.Second}
resp, err := client.Get(url)
if err != nil {
return "Unknown"
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return "Unknown"
}
var result struct {
License struct {
SPDXID string `json:"spdx_id"`
} `json:"license"`
}
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return "Unknown"
}
if result.License.SPDXID != "" && result.License.SPDXID != "NOASSERTION" {
return result.License.SPDXID
}
return "Unknown"
}
func fetchLicenseFromRepo(modulePath string) string {
if strings.HasPrefix(modulePath, "golang.org/x/") {
return "BSD-3-Clause"
}
if !strings.HasPrefix(modulePath, "github.com/") {
return "Unknown"
}
parts := strings.Split(modulePath, "/")
if len(parts) < 3 {
return "Unknown"
}
return fetchGitHubLicense(parts[1], parts[2])
}
func checkLicenses() {
goMod, err := parseGoMod()
if err != nil {
fmt.Printf("Error: %vn", err)
return
}
licenseCount := make(map[string]int)
for i, mod := range goMod.Requires {
fmt.Printf("rProcessing %d/%d...", i+1, len(goMod.Requires))
license := fetchLicenseFromRepo(mod.Path)
licenseCount[license]++
fmt.Printf("r %s: %sn", mod.Path, license)
}
fmt.Println("nLicense Distribution:")
for license, count := range licenseCount {
percentage := float64(count) / float64(len(goMod.Requires)) * 100
fmt.Printf(" %s: %d modules (%.1f%%)n", license, count, percentage)
}
}
The license checker recognizes that golang.org/x/ packages use BSD-3-Clause, then queries GitHub’s API for other repositories.
Dependency Analysis with Checksum Verification
The dependency analyzer lists modules and verifies their integrity using go.sum:
func parseGoSum() map[string]string {
checksums := make(map[string]string)
file, err := os.Open("go.sum")
if err != nil {
return checksums
}
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
parts := strings.Fields(scanner.Text())
if len(parts) >= 3 {
module := parts[0] + "@" + parts[1]
checksums[module] = parts[2]
}
}
return checksums
}
func analyzeDependencies() {
goMod, err := parseGoMod()
if err != nil {
fmt.Printf("Error: %vn", err)
return
}
checksums := parseGoSum()
fmt.Printf("Module: %sn", goMod.Module.Path)
fmt.Printf("Found %d dependencies:nn", len(goMod.Requires))
direct, indirect := 0, 0
sort.Slice(goMod.Requires, func(i, j int) bool {
return goMod.Requires[i].Path < goMod.Requires[j].Path
})
for _, mod := range goMod.Requires {
status := "direct"
if mod.Indirect {
status = "indirect"
indirect++
} else {
direct++
}
checksumKey := mod.Path + "@" + mod.Version
hasChecksum := "❌"
if _, exists := checksums[checksumKey]; exists {
hasChecksum = "✅"
}
fmt.Printf(" %s %s@%s (%s)n", hasChecksum, mod.Path, mod.Version, status)
}
fmt.Printf("nSummary: %d direct, %d indirect dependenciesn", direct, indirect)
}
Command Interface
The main function routes commands to the appropriate handlers:
func main() {
if len(os.Args) < 2 {
fmt.Println("Usage: go run main.go <command>")
fmt.Println("Commands:")
fmt.Println(" deps List all dependencies")
fmt.Println(" vulns Check for vulnerabilities")
fmt.Println(" licenses Check license compliance")
os.Exit(1)
}
switch os.Args[1] {
case "deps":
analyzeDependencies()
case "vulns":
checkVulnerabilities()
case "licenses":
checkLicenses()
default:
fmt.Println("Unknown command")
os.Exit(1)
}
}
Save the code as main.go and run it in any Go project:
# List dependencies with checksum verification
go run main.go deps
# Scan for vulnerabilities
go run main.go vulns
# Analyze licenses
go run main.go licenses
The output shows dependency information, vulnerability reports, and license distribution across your project’s dependencies. The tool demonstrates how dependency analysis works behind the scenes, parsing module files, querying public APIs, and cross-referencing data sources.
This implementation covers the basic concepts but is just a starting point. Real vulnerability scanning requires comprehensive databases, sophisticated version range matching, false positive filtering, and robust error handling. License compliance tools need legal policy engines, compatibility matrices, and custom license detection beyond what GitHub’s API provides. For production use, you’d want multiple data sources, caching, rate limiting, and much more thorough validation logic.
Happy coding 😉
You can find source code here https://github.com/rezmoss/go-dependency-scanner
