FREEDOM251

Freedom251 › Code Clone Detection Tools

semdup

Paid LinuxMacSelf-hostedWindows

Visit semdup Alternatives

semdup's own site

What semdup is

semdup is an open-source command-line tool for finding near-duplicate code in a repository by comparing semantic meaning rather than token sequences. It is aimed at catching duplication in AI-generated code and helping agents avoid reimplementing existing code. Its default embedding model, nomic-ai/CodeRankEmbed, runs locally on CPU or CUDA, and the maker says everything stays local unless an external backend is selected. Supported languages are Rust, TypeScript, Python, Go, Java, C#, PHP, Ruby, C, and C++; heavily macro-based C and C++ may perform less well.

Users can adjust similarity thresholds, set distinct warning and error thresholds, and choose how many times a match must appear. A diff mode compares changed code with a Git base reference, and a supplied GitHub Actions workflow and action can run checks on pull requests and pushes. The optional Amazon Bedrock backend uses Titan Text Embeddings V2 through the AWS SDK credential chain and needs bedrock:InvokeModel permission. The initial setup, refresh, or scan downloads the model into a user cache and verifies files against compiled BLAKE3 hashes. Index data and cached embeddings are stored in SQLite, keyed by content hash and model ID. Sparse search is approximate and may miss matches that exact search finds; automatic indexing switches to sparse search above 20,000 scannable units. The project is licensed under MIT or Apache-2.0, at the user's choice.

Who it suits

semdup is suited to developers who want repository-level semantic duplicate checks, including teams reviewing code generated by AI tools. Its GitHub Actions workflow and diff mode also suit projects that want checks on code changes.

What it does well

  • Compares semantic meaning rather than token sequences.
  • Default model runs locally on CPU or CUDA.
  • Supports ten listed programming languages.
  • Offers configurable warning and error thresholds.
  • GitHub Actions can run checks on pull requests and pushes.
  • Available under MIT or Apache-2.0.

What to know first

  • Sparse search can miss matches found by exact search.
  • Automatic indexing switches to sparse search above 20,000 scannable units.
  • Heavily macro-based C and C++ may perform less well.
  • The maker warns it may contain bugs and produce false positives.

Getting started

  1. Visit https://github.com/niklebedenko/semdup.
  2. Choose Linux, macOS, or Windows, or use the self-hosted option.
  3. Initialize semdup, refresh it, or run a scan; the model downloads to a user cache and is verified before loading.
  4. Configure similarity thresholds and the minimum number of duplicates for a cluster.
  5. Optionally add the provided GitHub Actions workflow or action for pull requests and pushes.
  6. Optionally select the Amazon Bedrock backend and provide access through the AWS SDK credential chain with bedrock:InvokeModel permission.

Verdict

Choose semdup if you want a free, open-source repository scanner for semantically similar code, with local inference and CI workflow support. Consider another approach if you need sparse search to find every pair: the project says it is approximate, and the tool may produce false positives.

Questions

How much does semdup cost?

semdup is free and open source, available under either the MIT or Apache-2.0 license.

Which platforms does it support?

The listed platforms are Linux, macOS, Windows, and self-hosted use.

Which languages can it scan?

The project lists Rust, TypeScript, Python, Go, Java, C#, PHP, Ruby, C, and C++. Heavily macro-based C and C++ may perform less well.

Does semdup send code to an external service?

The default model runs locally on CPU or CUDA, and the maker says everything stays local unless an external backend is selected. An optional Amazon Bedrock backend is available.

What are the search limitations?

Sparse search is approximate and may miss pairs found by exact search. Automatic indexing switches to sparse search above 20,000 scannable units.

Can it run in code review workflows?

Yes. The project provides a GitHub Actions workflow and action for checks on pull requests and pushes, and its CLI includes a diff mode that compares changed code against a Git base reference.

Facts from semdup's own pages

Bedrock integration
An optional Amazon Bedrock backend uses Titan Text Embeddings V2 through the AWS SDK credential chain and requires bedrock:InvokeModel permission. · github.com, 4 Oct 2026
Caveat
The project says it is still young, may have bugs, and can produce false positives because it uses fuzzy search. · github.com, 9 Oct 2026
Code review
The CLI includes a diff mode that compares changed code against a Git base reference, and the project provides a GitHub Actions workflow and action. · github.com, 9 Oct 2026
Configuration
Users can configure similarity thresholds, warning and error thresholds, and the minimum number of duplicates in a cluster. · github.com, 9 Oct 2026
Download integrity
The first initialization, refresh, or scan downloads the model into a user cache and verifies files against compiled BLAKE3 hashes before loading them. · github.com, 4 Oct 2026
Index limitation
Sparse search is approximate and can miss pairs found by exact search, while exact search compares every pair and avoids dropping matches. · github.com, 4 Oct 2026
Known limitation
The maker says the tool is still young, may contain bugs, and can produce false positives because it uses fuzzy search. · github.com, 4 Oct 2026
Languages
It supports Rust, TypeScript, Python, Go, Java, C#, PHP, Ruby, and C/C++, with a note that heavily macro-based C/C++ may perform less well. · github.com, 9 Oct 2026
License
The project is available under either the MIT or Apache-2.0 license. · github.com, 9 Oct 2026
Local inference
The default model runs on the user’s machine using CPU or CUDA. · github.com, 9 Oct 2026
Maintainer support
The maker welcomes contributions to the project. · github.com, 4 Oct 2026
Matching controls
Users can tune similarity thresholds, configure separate warning and error thresholds, and require matches to appear a chosen number of times. · github.com, 4 Oct 2026
Model choice
The default embedding model is nomic-ai/CodeRankEmbed, and users can configure a compatible model or use a Python sidecar for other backends. · github.com, 4 Oct 2026
Optional backend
An optional Amazon Bedrock backend uses Titan embeddings and the AWS SDK credential chain. · github.com, 9 Oct 2026
Purpose
Semdup detects near-duplicate code by comparing semantic meanings rather than token sequences. · github.com, 9 Oct 2026
Search limits
Sparse search is approximate and can miss matches that exact search would find; automatic indexing switches to sparse search above 20,000 scannable units. · github.com, 9 Oct 2026
Storage
Semdup stores its index and cached embeddings in SQLite, keyed by content hash and model ID. · github.com, 9 Oct 2026
Use case
The maker describes it as useful for catching AI-generated code duplication and preventing agents from reimplementing existing code. · github.com, 4 Oct 2026
Workflow integration
The project provides a GitHub Actions workflow and action that can run semdup checks on pull requests and pushes. · github.com, 4 Oct 2026

Where semdup stands on Freedom251

Look at these too