For a new Linux Foundation project, define its Jenkins jobs as YAML in ci-management or releng/builder, select reusable Global-JJB templates, test the generated jobs in Jenkins Sandbox, and submit the change to Gerrit. The same workflow covers build-node selection, managed files, log retrieval, and custom builder images without editing production jobs directly.
How the LF Jenkins setup is organized
Linux Foundation Release Engineering consolidates jobs that once ran on project-specific virtual machines onto a shared Jenkins service. Each Git repository receives a Jenkins view, while Jenkins Job Builder (JJB) turns YAML definitions into Jenkins job configuration.
As an Amazon Associate I earn from qualifying purchases.
Projects normally contribute configuration in one of two repositories:
ci-managementreleng/builder
The repository and directory used for a particular LF project depend on that project’s existing release-engineering setup. Follow the pattern already used by neighboring projects rather than creating a new layout.
#1 Best Overall
Adding Jenkins jobs for a new project
- Create the project directory. Add a
jjb/<new-project>directory in the selected repository. - Add the project YAML file. Create
<project>.yamlin that directory. The file declares the project and references the jobs or job groups supplied by Global-JJB. - Select templates for the project’s technology. Use the recommended Global-JJB groups where they match the repository, such as CI, Python, Node, or ReadTheDocs groups.
- Set a build node. Choose a
build-nodevalue that exactly matches an available node-template label. - Install and validate JJB locally. Use a Python virtual environment, install the repository’s requirements (or install JJB with
pip), and verify the executable withjenkins-jobs --version. - Render and upload in Sandbox. Run the repository’s documented
jenkins-jobsworkflow so the YAML is translated to XML and uploaded to Jenkins Sandbox. - Exercise the jobs. Check configuration, triggers, node assignment, and the expected build behavior in Sandbox before proposing the change.
- Submit the change to Gerrit. Have the YAML and any related configuration reviewed through the normal
ci-managementorrelengGerrit process.
Sandbox testing is the safe default: do not modify production jobs directly. The exact command-line options for rendering and uploading can vary with the repository’s wrapper and credentials, so use the checked-in requirements and instructions for that repository.
Minimal Maven job set
For a Maven project, the guide identifies these five jobs as the documented minimum:
| Job | Purpose in the Maven set |
|---|---|
gerrit-maven-clm |
Runs the project’s component/license-management check. |
gerrit-maven-merge |
Builds the merge path after a Gerrit change is accepted. |
gerrit-maven-release |
Provides the Maven release job. |
gerrit-maven-verify |
Verifies a proposed Gerrit change. |
gerrit-maven-sonar |
Runs Sonar analysis. |
gerrit-maven-verify-dependencies is optional. Add it when dependency verification is relevant to the project rather than treating it as part of the minimum set.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Choosing a build node
LF Jenkins creates build agents on demand and removes them after the job terminates. Node templates are administered through Jenkins’ OpenStack Cloud plugin. A job’s build-node must match a label exposed by one of those templates.
When an existing label is enough
- Use an existing label when its operating system, tools, language runtime, and resource size satisfy the job.
- Prefer the shared template used by comparable projects so the configuration remains reviewable and easier to maintain.
When to request a new configuration
If the project needs a runtime or toolchain not provided by an available template, submit the change to ci-management or releng. Do not silently invent a label in the project YAML: a label without a matching cloud template leaves the job unable to acquire an agent.
Operational trade-off
| Capacity choice | What it means | Trade-off |
|---|---|---|
| On-demand agents | Agents appear only while a job runs and are deleted afterward. | Reduces idle infrastructure, but a job depends on a correctly configured template and may wait for provisioning. |
| Project-specific image/template | A dedicated environment supplies the project’s exact tools. | Improves reproducibility for unusual builds, but adds image maintenance and review overhead. |
Testing in Jenkins Sandbox versus production
Sandbox resembles production but is intentionally isolated. It does not publish artifacts to Nexus or Nexus3 and does not vote in Gerrit. Sandbox may contain dummy configuration files and credentials, and it has fewer virtual-machine nodes than production.
| Capability | Sandbox | Production |
|---|---|---|
| Configuration and trigger checks | Use Sandbox first. | Runs the live configuration. |
| Merge, push, CLM, Docker, and Sonar jobs | Can be tested to a useful extent. | Required for live integrations. |
| Artifact publication | No Nexus/Nexus3 publication. | Real artifact repositories are used. |
| Gerrit voting | No Gerrit votes. | Real Gerrit communication and voting. |
| Nexus IQ, Sonar, Gerrit, or Nexus communication | Not a fidelity check for the live services. | Use production to confirm these communications. |
| Available VM nodes | Fewer nodes than production. | Full production capacity and labels. |
A safe test sequence
- Render the YAML with the repository’s JJB setup and inspect the generated job configuration.
- Upload the result to Sandbox and confirm that the job resolves the intended
build-nodelabel. - Run representative merge, verify, CLM, Docker, or Sonar jobs where Sandbox supports them.
- Check console output for configuration and toolchain errors, remembering that dummy credentials and the absence of artifact publication are expected.
- Use the production environment only when the test requires real Nexus-IQ, Sonar, Gerrit, or Nexus communication.
Jenkins Job Builder and Global-JJB
What JJB does
Jenkins Job Builder translates declarative YAML into Jenkins XML configuration and manages the resulting jobs. A Python virtual environment keeps the JJB installation separate from system packages. Projects can install from the repository’s requirements.txt or with pip; jenkins-jobs --version confirms that the executable is available.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →What Global-JJB provides
Global-JJB is the LF CI library of reusable Jenkins Job Builder templates. It exists so projects can select standardized jobs instead of defining the same templates locally. Its documented groups cover common CI patterns as well as technologies such as Maven, Python, Node, and ReadTheDocs.
Global-JJB compared with project-local templates
| Approach | Best fit | Main implication |
|---|---|---|
| Global-JJB template or group | A project follows a supported, common build pattern. | Less duplicated YAML and a shared maintenance path. |
| Project-local template | The build has requirements that the shared library does not express. | More control, but the project owns additional configuration and future maintenance. |
Global-JJB versus the LF Pipelines Library
The LF Pipelines Library is a library of Jenkins pipeline functions intended to replicate Global-JJB functionality while standardizing pipeline creation. It is a different implementation style, not a second name for the JJB templates.
| Decision axis | YAML with Global-JJB | Pipeline functions with the LF Pipelines Library |
|---|---|---|
| Definition style | Jenkins Job Builder YAML rendered to XML. | Jenkins Pipeline code calling shared functions. |
| Reuse model | Shared JJB job templates and groups. | Shared pipeline functions. |
| Reviewability | Declarative job settings are easy to compare in YAML, with generated XML as the Jenkins output. | Control flow is visible in pipeline code, but reviewers must understand pipeline behavior. |
| Typical migration choice | Good for an established JJB project or a job that maps directly to an existing template. | Good when the project is standardizing around pipelines and needs function-level orchestration. |
| Documented functions | Not applicable. | lfCommon, lfDefaults, lfInfraShipLogs, lfJava, lfNode, and lfParallelCostCapture. |
Choose the approach already used by the project unless there is a clear migration requirement. Mixing styles without an ownership plan increases the number of places where build behavior can diverge.
Managed configuration files
Managed Config Files are kept in the ci-management/jenkins-config/managed-config-files tree. Store shared configuration there rather than embedding a copy in every job. Treat changes to these files as centrally managed configuration: review them with the same care as job definitions because multiple jobs may consume one file.
Recommended Free Tools
Finding logs and understanding retention
Use the LF log server rather than relying on Jenkins console pages for long-term access. The guide says logs are compressed and stored in a Nexus repository.
| Environment or policy | Documented behavior |
|---|---|
| Log archive | Log-server archives are stored for six months. |
| Production cleanup | Logs older than 180 days are deleted every day at 08:00 UTC. |
| Sandbox cleanup | Sandbox logs and jobs are deleted every Saturday at 08:00 UTC. |
The six-month archive statement and the 180-day production cleanup threshold are operational policy values. Retrieve important diagnostics before the relevant cleanup window, especially for a Sandbox job whose records may disappear on the next Saturday cleanup.
Building a custom Jenkins builder image
Use the ci-management/packer directory when an existing build image cannot provide the required toolchain. A new builder image needs both of these files:
packer/templates/BUILDER.jsonpacker/provision/BUILDER.yaml
The documented recommendation is to provision the image with Ansible. For Sandbox testing and deployment, use the Global-JJB gerrit-packer-merge job. Keep the image definition, provisioning steps, and the consuming build-node label aligned; an image that is built successfully but never exposed through a matching node template cannot serve the project job.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesPractical troubleshooting checklist
The job cannot acquire an agent
- Compare the YAML
build-nodevalue with the exact label on an OpenStack node template. - Confirm that the required template exists in the environment where the job is running; Sandbox has fewer nodes than production.
- If the label or image is new, submit the corresponding infrastructure change to
ci-managementorreleng.
The generated job differs from the YAML
- Verify the JJB version with
jenkins-jobs --version. - Check that the intended Global-JJB group or template is included and that the project directory is under
jjb/<new-project>. - Render and inspect the generated XML in Sandbox before requesting production review.
A Sandbox test appears not to publish or vote
This is expected isolation behavior: Sandbox does not publish to Nexus/Nexus3 or vote in Gerrit. Validate job logic there, then reserve live integration checks for production.
Historical logs are missing
Check the log server and its compressed Nexus archives first. Records may have passed the six-month archive policy, the 180-day production cleanup threshold, or the weekly Sandbox cleanup.
Quick Recap
Choosing an implementation path
| Situation | Recommended path |
|---|---|
| New project with a conventional Maven, Python, Node, or documentation build | Use the matching Global-JJB group, set an existing node label, test in Sandbox, and submit the YAML through Gerrit. |
| Existing project already managed with JJB | Extend the current YAML and shared templates before introducing pipeline code. |
| Project adopting standardized Jenkins pipelines | Evaluate the LF Pipelines Library functions and define ownership for the pipeline code. |
| Build requires tools absent from all node templates | Add a reviewed Packer/Ansible builder image, expose it through an OpenStack template label, and test with gerrit-packer-merge in Sandbox. |
| Test requires real repository or Gerrit side effects | Use Sandbox for configuration validation, then coordinate a controlled production test because Sandbox intentionally omits those communications. |
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
