Halloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsSlow PC?RecommendedPC slow today? Run a repair scan before it gets worseResolve common Windows issues and optimize system performance.Scan NowHalloween DealsAmazon USOctober picks are moving - compare before checkoutAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk12 min

How to Build a CI Testing Environment With Docker and Docker Compose

A practical guide to running integration tests against real databases and services with Docker Compose in CI, including profiles, override files, readiness checks, unique project names, secrets, caching, diagnostics, and cleanup.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The reliable pattern is to let Docker Compose define the application, test runner, and every dependency your tests need, then give each CI job its own Compose project. Start the stack, wait for dependencies to pass health checks, run the test command, collect logs and reports, and tear the project down in cleanup. Use profiles or a CI override file to change only what differs from development, and use Docker secrets rather than ordinary environment variables for passwords.

What a Compose-based CI environment does

Docker Compose describes a multi-container application in YAML and manages the containers as one project. Docker documents Compose for development, staging, production, testing, and CI workflows. In a test job, the file can define the application under test, a disposable test runner, a database, a queue, a cache, and any other service the test suite calls.

As an Amazon Associate I earn from qualifying purchases.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The important distinction is between defining the environment and executing the tests. Compose creates the environment; your test command supplies the pass or fail result. A successful job therefore has four properties:

  • Complete dependencies: every network service required by the suite is declared, rather than relying on software preinstalled on the CI worker.
  • Isolation: each build uses a unique Compose project name and removes its containers, networks, and volumes when it finishes.
  • Readiness: stateful services report that they can accept requests before tests begin.
  • Diagnostics: logs and test reports are retained even when a test command fails.

Typical service layout

Service Purpose CI guidance
app The image or source tree being tested. Build it in the job when the commit changes the image.
test A one-off runner that executes unit, integration, or end-to-end tests. Use docker compose run --rm test for an ephemeral command.
db A real database instance for integration tests. Add a health check and a disposable volume.
queue or cache Message brokers, caches, or other external dependencies. Give each dependency its own health check where possible.

Keep service names stable. Compose places services on a project network, so the test runner can connect to db or queue by service name instead of a changing container IP. Do not hard-code a CI host address for a dependency that lives in the same Compose project.

#1 Best Overall
Sale
TOZO A1 Wireless Earbuds Bluetooth 5.3 Light Weight in Ear Headphones | IPX5 Waterproof,2 Mics for AI Calls,Immersive Premium Sound Bass Headset with Charging Case,32 EQ via App Black
  • [Ultra-Lightweight Ear Buds Designed for Small Ears] Each earbud weighs only 3.7g and features a compact, ergonomic in-ear design made especially for small ears. Secure, low-profile, and comfortable for workouts, all-day wear or overnight sleeping.The ultra-slim profile sits flush in your ear with no hard edges poking or rubbing, so you can even sleep on your side without discomfort.
  • [Immersive Stereo Sound with TOZO OrigX Technology] TOZO OrigX tuning delivers clear vocals, balanced mids, and natural stereo sound for music, podcasts, and videos.
  • [Long Battery Life for Daily Use] Get up to 7 hours of playtime on a single charge, with up to 32 hours total using the charging case—ideal for workdays, commuting, and extended listening sessions.
  • [Bluetooth 5.3 & Stable Connection] Bluetooth 5.3 provides fast pairing, stable wireless performance, and reduced dropouts as you move around home or office.
  • [Deep Bass with Clear Vocals] High-performance drivers produce punchy bass while keeping vocals clean and detailed for everyday listening.

Prerequisites and repository layout

Install Docker Engine with the Compose plugin on the CI worker. The commands in this guide use the current v2 syntax, docker compose, rather than the older standalone docker-compose command. Confirm the installation before the job starts:

docker version
docker compose version

Keep the Compose definition and the files it references in the repository. A practical layout is:

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
.
├── compose.yaml
├── compose.ci.yaml
├── Dockerfile
├── run_tests
└── .dockerignore

The CI file should not contain production credentials. Non-secret switches such as a log level, test marker, or database name can come from an environment file or explicit variables. Passwords, tokens, certificates, and signing keys belong in a CI secret store and should be mounted through a secret mechanism.

Write the base Compose definition

A minimal application, test runner, and database

The following example uses PostgreSQL to show the important relationships. Replace the image, build command, and test command with those for your project. The database password is read from a Compose secret file; in CI, create that file from the provider’s protected secret store and delete it during cleanup.

name: ci-example

services:
  app:
    build:
      context: .
    profiles: [test]
    environment:
      DATABASE_HOST: db
      DATABASE_PORT: 5432
      DATABASE_NAME: app_test
      DATABASE_USER: app
      DATABASE_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password

  test:
    build:
      context: .
    profiles: [test]
    depends_on:
      db:
        condition: service_healthy
    environment:
      DATABASE_HOST: db
      DATABASE_PORT: 5432
      DATABASE_NAME: app_test
      DATABASE_USER: app
      DATABASE_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password
    command: ./run_tests

  db:
    image: postgres:16
    profiles: [test]
    environment:
      POSTGRES_DB: app_test
      POSTGRES_USER: app
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password
    healthcheck:
      test: [CMD-SHELL, 'pg_isready -U app -d app_test']
      interval: 5s
      timeout: 5s
      retries: 12
    volumes:
      - db_data:/var/lib/postgresql/data

secrets:
  db_password:
    file: ./secrets/db_password.txt

volumes:
  db_data:

The test container is intentionally separate from app. It can run migrations, seed data, and the complete suite without turning the application process into the job controller. If your tests exercise a running HTTP server, keep app enabled and have the test runner call it at its Compose service name.

Rank #2
Sale
Smart Watch for Men Women Compatible with iPhone Samsung Android Phone 1.69 inch Full Touch Screen IP68 Waterproof Bluetooh Fitness Tracker Heart Rate/Sleep Monitor (Black)
  • 【IP68 Waterproof Multifunctinal Smart Watch】Torjalph fitness activity tracker smart watch is IP68 waterproof technology ,you can wear it when swimming,handwashing,it's rain-proof which means that you perform multiple exercise modes outside without worrying about sweat or rain corroding your fitness smart watch,and this watch equipped with a 230 mAh large capacity battery magnetic charging ,and this sports activity tracker watch can be used 5-7 days If fully charged.
  • 【Larger 1.83'' Full Screen Watch & Multi-dial Selection】1.83 inch full HD color touch screen smart watch for men / women provides excellent 240*284 resolution image quality ,This Bluetooth smart watch offers a better touch experience and remarkable immersion and ,more than 200 watch faces for you to choose from App"Da Fit ", you can change the dial style according to your mood to wear keeping the world on your wrist
  • 【Accurate Activity Tracker Smart Watch 】sports watch activity tracker uses sweat and dust resistant breathable silicone strap, our Fitness Tracker Smart Watch for Android iOS Phones supports 12 sports modes,walking,running,bicycle,swimming,badminton,basketball,football,rope skipping. This bluetooth calls smart watch for men women is lightweight and stylish, bringing more convenient and relaxing sports experience while also providing more protection.
  • 【Your 24 Hour Personal Daily Health Monitor】Fitness watch for women men full touch screen IP67 waterproof bluetooth smartwatch for Android iOS Phones with heart rate blood pressure monitor multifunction Andriod smart watch with steps counter sleep monitor sports fitness watch, monitoring blood oxygen saturation allows you to understand your physical condition in real time to track fitness progress ,to know your health status in time.
  • 【More Functions Bluetooth Call Smart Watch】This fitness activity smart watch full screen for iPhone and Android phones includes a bunch of new features including:The bluetooth smart watch can hang up the phone, receive and display WhatsAPP /Text /Facebook /Twitter /Messenger reminder.Support sedentary reminder, alarm clock, stopwatch, timer,weather information,music control, remote camera control compatible with Android 5.0 and above or iOS 9.0 and above smartphones.

Validate the rendered configuration

Before creating containers, render the complete configuration:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
docker compose --profile test config

config parses the files, resolves variables and merges, and prints canonical YAML. Review the output in CI logs when debugging an unexpected image, volume, command, or environment value. Run the same command locally with the same profile and override files so a developer can see exactly what the job will use.

Choose profiles or a CI override file

You do not have to maintain a completely separate Compose definition for development, testing, and staging. Profiles let one file contain optional services and settings. In the example, the test profile is selected with --profile test. Development-only services can use a dev profile, while staging services can use staging.

Use a second file when CI needs structural changes such as different mounts, ports, environment values, or restart behavior. Compose merges files from left to right, so the later file supplies additions or overrides:

docker compose -f compose.yaml -f compose.ci.yaml --profile test config
docker compose -f compose.yaml -f compose.ci.yaml --profile test up --build -d

A small compose.ci.yaml might change the test command and disable a development restart policy:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #3
New Bee Guitar Cable 10ft Electric Instrument Bass AMP Cord
  • [Low Noise High Fidelity] New bee guitar cord with 22AWG oxygen-free-copper center conductors provides enhanced signal transfer, the PVC insulation material and the high-density stranded copper double shield provide noise rejection for quiet operation
  • [Ideal for Choice] New bee 1/4 straight and right angle metal connectors, ensures a secure connection in any instrument, pedal, or amp jack. L shape great fro on stage guitar stands
  • [Wide Compatible] Suitable for mixer, power amplifier, microphone, effector, decoder, equalizer, electoronic organ, electric guitar, electric bass and other electronic equipments. Widely used in the installation of smart home, karaoke speaker, professional stage audio, etc
  • [Easy Disassemble] Making incidental repairs quick and easy! Very nice, flexible cable
  • [Dont Hesite to Order] 24 hours customer services and professional technology team are standing by
services:
  app:
    restart: no
    environment:
      APP_ENV: ci
  test:
    restart: no
    environment:
      APP_ENV: ci
      TEST_REPORT_DIR: /artifacts
    volumes:
      - ./artifacts:/artifacts

Profiles are preferable when the differences are mostly optional services. An override file is clearer when CI needs a distinct set of mounts or environment values. In both cases, render the merged result with docker compose config before execution.

Make dependency readiness explicit

depends_on controls startup order; it does not, by itself, prove that a database has completed initialization and is accepting connections. A container can be running while its process is still replaying logs, creating schemas, or refusing connections. Add a health check and use the service_healthy condition when tests must wait.

Design a useful health check

  • Check the service’s real protocol, not merely whether a process exists. pg_isready is more useful for PostgreSQL than a generic process check.
  • Set an interval and timeout long enough for the CI worker, then allow enough retries for first-time initialization.
  • Use credentials and database names that match the service environment.
  • Keep the check cheap; it runs repeatedly while the dependency is starting.

For services without a native readiness command, use a small client request or a project-provided probe. If the application performs migrations, run them after the database health check and before tests. A healthy database does not imply that your application’s schema is current.

When Compose still reports an unhealthy service

  1. Inspect the dependency log with docker compose logs db.
  2. Inspect the health status and failing output with docker inspect on the container ID shown by docker compose ps.
  3. Check that the health command uses the same user, database, and port configured for the service.
  4. Increase retries only after confirming that initialization is legitimately slow; a permanently failing check indicates a configuration or image problem.

Isolate concurrent CI jobs

Two builds running the same Compose file can otherwise collide over project names, container names, networks, and volumes. Give every job a unique project name based on the CI system’s build identifier:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
docker compose -p ci-${BUILD_ID} -f compose.yaml -f compose.ci.yaml --profile test up --build -d

You can set the same value for the whole job:

export COMPOSE_PROJECT_NAME=ci-${BUILD_ID}

Use an identifier that is unique for the lifetime of the job and valid for Compose project naming. Do not set fixed container_name values in the file; they defeat project isolation and make parallel builds compete for the same name. Use named volumes only within the project, and remove them when the job is finished if the data must never survive a build.

Rank #4
BSIMB 15.6" Digital Family Calendar & Chore Chart, No Subscription Fees
  • Digital Family Calendar: BSIMB digital calendar wall touch screen puts an end to your chaotic life. Designed for busy families, the digital calendar keeps you organised and in step with your family. Assign different colours and tasks to each family member to make family scheduling easier and more intuitive.
  • Easy Setup and Auto-Sync: Plug in, connect to WLAN and auto-syn your electric calendar in minutes. Digital wall calendar compatible with Google calendar, iCloud calendar, Outlook calendar, Cozi calendar and Yahoo calendar. Add events directly on the device or through the ecalendar app. Electronic calendar stays current whether you're at work or traveling—no manual updates needed.
  • All-In-One Home Assistant: Smart calendar builds habits and encourage independence with personalised chores. Display daily meal plan to reduce the daily ‘What's for dinner?’ stress. To-do lists keep track of everything you want. Turning chores into valuable achievements with stars and rewards, making life easier and more enjoyable.
  • Manage Anytime Anywhere: BSIMB electronic calendar for family stay connected with the powerful ecalendar app. Access and manage your digital wall calendar anytime, anywhere with free features like stars & rewards, photo screensaver, chore charts, meals plans, to-do lists etc. Electronic planner paid features like Magic Import tailor advanced tools for users with special needs.
  • Stay in Touch with Loved Ones: Share photos or videos from your phone to electronic wall calendar with app, instantly update your life moment, and keep in touch with your family, especially those who are far away; you can also surprise your smart calendar digital wall by sending him a wish card; it's also great for keeping tech-illiterate elders in touch with the youngsters.

Handle secrets without leaking them

Ordinary environment variables are convenient for non-sensitive configuration, but they can appear in process listings, diagnostic output, or CI metadata. Docker advises using a secret mechanism for passwords and other sensitive values.

Use mounted secret files

Compose makes a declared secret available inside the container under /run/secrets/<name>. Images that support the _FILE convention can read the value without placing it directly in the environment. For an image that does not support that convention, have the application read the file itself or use a short entrypoint that exports the value only inside the process.

Protect the secret source

  • Create secrets/db_password.txt in the workspace only for the duration of the job, with restrictive file permissions.
  • Never commit that path or echo its contents in a diagnostic command.
  • Delete the file in the cleanup step, even if the test command fails.
  • Use separate credentials and databases for CI; never point an integration test at a shared production or staging database.

Run the isolated test lifecycle

Docker describes the canonical isolated loop as create the environment, run the suite, and destroy the environment. A CI job should add image building, log collection, and guaranteed cleanup:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
set -eu

export COMPOSE_PROJECT_NAME=ci-${BUILD_ID}
COMPOSE='docker compose -f compose.yaml -f compose.ci.yaml --profile test'

cleanup() {
  ${COMPOSE} logs --no-color > compose.log || true
  ${COMPOSE} down --volumes --remove-orphans || true
  rm -f secrets/db_password.txt
}
trap cleanup EXIT

${COMPOSE} config
${COMPOSE} up --build -d
${COMPOSE} run --rm test

The exact shell error-handling syntax can be adapted to your CI runner. The essential behavior is that cleanup runs on success, failure, cancellation, and test timeout. If you need to execute a command in a service that is already running, use docker compose exec instead of run. The latter creates a one-off container; exec enters an existing container.

Collect evidence before teardown

Always capture logs before down, because removing the project removes the easiest path to container output. Preserve:

  • Compose logs with timestamps and without color codes.
  • JUnit, TAP, coverage, or other test reports written to a mounted artifacts directory.
  • The rendered docker compose config output, after redacting secret values.
  • Container status and exit codes when a service fails to start.

Uploading these files as CI artifacts turns an otherwise opaque failure into a debuggable one. Do not upload mounted secret files or unredacted environment dumps.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Integrate Compose into a CI provider

A provider-neutral job has this order:

  1. Check out the commit and install or select the required Docker version.
  2. Restore Docker or language-package caches when available.
  3. Materialize protected secrets and set a unique project name.
  4. Run docker compose config to validate the merged definition.
  5. Build and start the stack with up --build -d.
  6. Run migrations or other initialization, then execute the test service.
  7. Collect logs and reports.
  8. Run down --volumes --remove-orphans in an unconditional cleanup step.

Docker’s official GitHub Actions example follows a build, export, test, then multi-platform build-and-push flow and reuses the initial build’s internal cache for the later build. A Compose-based project can keep the same idea: perform the Compose build and test in the job, then use the resulting image or cache in a later publishing job. Pin action versions according to your repository policy and keep publishing credentials out of the test job when the job does not need them.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Example GitHub Actions job shape

jobs:
  integration:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Prepare secret file
        run: |
          mkdir -p secrets
          printf %s '${{ secrets.CI_DB_PASSWORD }}' > secrets/db_password.txt
      - name: Validate Compose
        env:
          COMPOSE_PROJECT_NAME: ci-${{ github.run_id }}
        run: docker compose -f compose.yaml -f compose.ci.yaml --profile test config
      - name: Build and test
        env:
          COMPOSE_PROJECT_NAME: ci-${{ github.run_id }}
        run: |
          docker compose -f compose.yaml -f compose.ci.yaml --profile test up --build -d
          docker compose -f compose.yaml -f compose.ci.yaml --profile test run --rm test
      - name: Collect logs
        if: always()
        env:
          COMPOSE_PROJECT_NAME: ci-${{ github.run_id }}
        run: docker compose -f compose.yaml -f compose.ci.yaml --profile test logs --no-color > compose.log || true
      - name: Upload diagnostics
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: compose-diagnostics
          path: |
            compose.log
            artifacts/
      - name: Tear down
        if: always()
        env:
          COMPOSE_PROJECT_NAME: ci-${{ github.run_id }}
        run: docker compose -f compose.yaml -f compose.ci.yaml --profile test down --volumes --remove-orphans

In a real workflow, place secret-file deletion in the final cleanup block as well. The example shows the control flow; adjust action versions, secret syntax, artifact paths, and runner labels to your provider and repository policies.

Improve reproducibility and speed

Pin what the test depends on

Use explicit image tags or digests instead of a moving latest tag when a dependency’s behavior affects test results. Pin language package lockfiles and the Dockerfile base image. Record the rendered Compose configuration as an artifact so a failed build can be tied to the exact image and variable values used, with secrets removed.

Cache safely

Build-layer caches can make repeated CI runs much faster. Cache keys should include the files that invalidate dependencies, such as a lockfile and Dockerfile. Do not let one branch’s mutable cache silently replace another branch’s tested image. A cache accelerates a build; it must not be treated as the source of correctness.

Keep test data disposable

Use a project-scoped volume for database files during a job, then remove it with down --volumes. If a test needs a known starting state, run migrations and seed data in the job rather than relying on a volume left by a previous run. Parallel jobs should never share that volume.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Diagnose common failures

Symptom Likely cause Action
The test runner cannot resolve db. The services are not on the same Compose project network, or the test uses a host name from another environment. Use the Compose service name, verify both services are in the selected profile, and inspect docker compose ps.
Connection refused immediately after startup. Startup order was mistaken for readiness. Add a protocol-level health check and condition: service_healthy; run migrations after readiness.
Two builds remove each other’s containers. Both jobs use the default or a fixed project name. Set -p ci-${BUILD_ID} or COMPOSE_PROJECT_NAME uniquely per job and remove fixed container_name values.
Tests pass locally but fail in CI because a port is busy. The CI file publishes a host port that is unnecessary or already allocated. Prefer internal service-to-service networking; publish a port only when a host process must connect, and override it for CI.
A service exits before logs are collected. The job tears down immediately after the failing command. Use an unconditional cleanup block that saves docker compose logs before down.
Configuration differs from what the file appears to say. Variable interpolation or multi-file merging changed the effective values. Run docker compose config and inspect the rendered result with secrets redacted.
A password appears in build or test output. A sensitive value was passed as a normal environment variable or echoed by a script. Move it to a secret mount, remove shell tracing around secret commands, and rotate any value that was exposed.

Security and maintenance checklist

  • Use a dedicated CI project name for every build.
  • Keep service names stable and avoid fixed container names.
  • Use health checks for databases, queues, and APIs whose process can be running before they are ready.
  • Validate the fully merged configuration before starting containers.
  • Use profiles or an additive CI file rather than copying the entire development definition.
  • Mount passwords and tokens as secrets; do not commit secret files or print them in logs.
  • Build the test image from the commit under test, and pin dependency images and lockfiles.
  • Collect logs and reports before teardown, including on failure and cancellation.
  • Remove containers, networks, orphaned services, and disposable volumes in cleanup.
  • Review cache scope so an untrusted change cannot inject artifacts into a trusted publishing job.

Putting the pattern into practice

Start with one Compose file that can bring up the database and test runner on a laptop. Add a real readiness check, then select it through a test profile. Introduce a CI override only when the job needs different mounts, ports, or environment values. In the pipeline, render the merged configuration, assign a unique project name, build and start the stack, run the test service, preserve diagnostics, and tear everything down.

This arrangement gives every commit the same dependency topology while allowing CI-specific isolation and security controls. It also makes failures actionable: the rendered configuration shows what was requested, health status shows whether dependencies were ready, logs show what happened, and the unique project name ensures one build cannot damage another.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Crashes, No Sound, or Screen Glitches?Free driver scan
PC Slower Than It Used to Be?Free scan - under a minute

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.