Recommended Free Tools
To block a named bot or crawler in an Apache .htaccess file, match the request’s User-Agent header and return a 403 response before your normal rewrite rules process the request:
Antihost offers Linux VPS plans with full root access, unlimited traffic on a 1 Gbps port and a 14-day free trial.
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (BadBot|AnotherBot) [NC]
RewriteRule ^ - [F,L]Replace the names with a distinctive fragment seen in your access logs. This filter is useful for discouraging cooperative or low-effort unwanted clients, but it is not authentication: a client can change its User-Agent string trivially.
As an Amazon Associate I earn from qualifying purchases.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →What the blocking rule does
The three directives work together:
RewriteEngine Onenables URL rewriting in this directory.RewriteCond %{HTTP_USER_AGENT} (BadBot|AnotherBot) [NC]tests the incomingUser-Agentrequest header against a regular expression. The vertical bar means “or,” so either name matches. The[NC]flag makes the comparison case-insensitive.RewriteRule ^ - [F,L]applies when the condition is true. The^pattern matches the beginning of the per-directory path, and therefore matches every path handled by a document-root.htaccess. The-substitution leaves the URL unchanged.[F]returns HTTP 403 Forbidden, while[L]stops the current rewrite round.
Put the condition and rule in the .htaccess file that controls the paths you want to protect. A rule in the document root can cover the site; a rule in a lower directory can limit the block to that directory.
Check these prerequisites first
An identical snippet can behave differently on different Apache installations. Confirm the following before changing production configuration:
#1 Best Overall
- [Ultra-Lightweight Ear Buds Designed for Small Ears] Each earbud weighs only 3.7g and features a compact, ergonomic in-ear design made especially for small ears. Secure, low-profile, and comfortable for workouts, all-day wear or overnight sleeping.The ultra-slim profile sits flush in your ear with no hard edges poking or rubbing, so you can even sleep on your side without discomfort.
- [Immersive Stereo Sound with TOZO OrigX Technology] TOZO OrigX tuning delivers clear vocals, balanced mids, and natural stereo sound for music, podcasts, and videos.
- [Long Battery Life for Daily Use] Get up to 7 hours of playtime on a single charge, with up to 32 hours total using the charging case—ideal for workdays, commuting, and extended listening sessions.
- [Bluetooth 5.3 & Stable Connection] Bluetooth 5.3 provides fast pairing, stable wireless performance, and reduced dropouts as you move around home or office.
- [Deep Bass with Clear Vocals] High-performance drivers produce punchy bass while keeping vocals clean and detailed for everyday listening.
- Apache is serving the request. These directives apply to Apache HTTP Server contexts, not every web server that happens to accept an
.htaccessfilename. - Overrides are permitted. The server administrator must allow the relevant directives for the directory through the site’s
AllowOverridepolicy. If overrides are disabled, Apache will ignore the file or return a configuration error. - The needed rewrite support is loaded.
RewriteCondandRewriteRulerequire the Apache rewrite module to be available. - You know which directory the file controls. Per-directory rewrite patterns are evaluated after Apache removes that directory’s prefix. A pattern that is correct in the document root may be wrong in a nested directory.
- You have a rollback copy. Save the current file before editing. Keep existing front-controller rules used by a content-management system or framework.
If you do not administer the main Apache configuration, .htaccess is commonly the available place to apply a directory-level rule. Your host still has to permit overrides.
Method 1: block User-Agents with mod_rewrite
Block one or several names site-wide
In the document-root .htaccess, use a grouped expression for the names you want to reject:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (BadBot|AnotherBot) [NC]
RewriteRule ^ - [F,L]The expression is intentionally a fragment rather than a complete header. It matches a header containing either string, regardless of case. Choose a distinctive fragment from your access logs; a short term such as bot can accidentally match legitimate tools and browsers.
Match an exact prefix when appropriate
If the unwanted client identifies itself with a predictable prefix, anchor the expression at the beginning of the header:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^NameOfBadRobot [NC]
RewriteRule ^ - [F,L]The ^ inside the condition is a regular-expression anchor. It means the header must start with that text. This is safer than an unanchored fragment when the name might appear in unrelated software strings. If the client appends a version after its name, the prefix still matches.
Limit the rule to a subtree
To deny the bot only under a path such as private/, change the rule pattern:
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (BadBot|AnotherBot) [NC]
RewriteRule ^private/ - [F,L]In a document-root file, this targets requests whose per-directory path begins with private/. If the file itself is inside the private directory, Apache has already removed that directory prefix; in that case, ^ can match everything in the directory. This distinction is a frequent cause of rules that appear not to work.
Understand [L] and [END]
[L] means “last rule for this rewrite round.” In a complex .htaccess, Apache can begin another per-directory rewrite pass after an internal rewrite. Apache 2.4 also provides [END], which stops all subsequent per-directory rewrite processing for the request. Use it when you specifically need to prevent re-entry and your existing rewrite flow is compatible with Apache 2.4:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (BadBot|AnotherBot) [NC]
RewriteRule ^ - [F,END]Do not replace every existing [L] blindly. Review the site’s current rules first; a front controller, authentication rule, or subdirectory configuration may rely on another rewrite pass.
Rank #2
- 【IP68 Waterproof Multifunctinal Smart Watch】Torjalph fitness activity tracker smart watch is IP68 waterproof technology ,you can wear it when swimming,handwashing,it's rain-proof which means that you perform multiple exercise modes outside without worrying about sweat or rain corroding your fitness smart watch,and this watch equipped with a 230 mAh large capacity battery magnetic charging ,and this sports activity tracker watch can be used 5-7 days If fully charged.
- 【Larger 1.83'' Full Screen Watch & Multi-dial Selection】1.83 inch full HD color touch screen smart watch for men / women provides excellent 240*284 resolution image quality ,This Bluetooth smart watch offers a better touch experience and remarkable immersion and ,more than 200 watch faces for you to choose from App"Da Fit ", you can change the dial style according to your mood to wear keeping the world on your wrist
- 【Accurate Activity Tracker Smart Watch 】sports watch activity tracker uses sweat and dust resistant breathable silicone strap, our Fitness Tracker Smart Watch for Android iOS Phones supports 12 sports modes,walking,running,bicycle,swimming,badminton,basketball,football,rope skipping. This bluetooth calls smart watch for men women is lightweight and stylish, bringing more convenient and relaxing sports experience while also providing more protection.
- 【Your 24 Hour Personal Daily Health Monitor】Fitness watch for women men full touch screen IP67 waterproof bluetooth smartwatch for Android iOS Phones with heart rate blood pressure monitor multifunction Andriod smart watch with steps counter sleep monitor sports fitness watch, monitoring blood oxygen saturation allows you to understand your physical condition in real time to track fitness progress ,to know your health status in time.
- 【More Functions Bluetooth Call Smart Watch】This fitness activity smart watch full screen for iPhone and Android phones includes a bunch of new features including:The bluetooth smart watch can hang up the phone, receive and display WhatsAPP /Text /Facebook /Twitter /Messenger reminder.Support sedentary reminder, alarm clock, stopwatch, timer,weather information,music control, remote camera control compatible with Android 5.0 and above or iOS 9.0 and above smartphones.
Place the block before broad rewrites
Put the User-Agent condition before catch-all rules that route every request to an application entry point. Otherwise, the request may be internally rewritten before the denial rule gets the opportunity to run, or the application may generate a response first. This ordering is an operational safeguard; verify it against the actual rules in your file.
Method 2: use SetEnvIfNoCase and Require
Apache’s robot-blocking guidance also supports an authorization approach that does not use mod_rewrite:
SetEnvIfNoCase User-Agent ^NameOfBadRobot goaway
<RequireAll>
Require all granted
Require not env goaway
</RequireAll>SetEnvIfNoCase examines the User-Agent header without regard to case. When the anchored expression matches, it sets an environment marker named goaway. The Require block grants access generally, then excludes requests carrying that marker.
Use this in a directory or other context where both directives are allowed. The exact placement matters: a rule in a parent directory can affect descendants, while a rule in a nested directory has narrower scope. If Apache reports that a directive is not allowed, the server’s override policy or loaded modules must be adjusted by an administrator.
Rewrite and authorization methods compared
| Decision point | mod_rewrite |
SetEnvIfNoCase plus Require |
|---|---|---|
| Primary input | %{HTTP_USER_AGENT} in a rewrite condition |
User-Agent matched by SetEnvIfNoCase |
| Typical response | 403 from the [F] flag |
Authorization denial for requests marked with the environment variable |
| Termination control | [L] for the current round or [END] on Apache 2.4 |
Handled by the authorization phase rather than rewrite termination flags |
| Path control | Directly expressed in the RewriteRule pattern, such as ^private/ |
Controlled by the directory or protected context containing the directives |
| Best fit | Sites already using rewrite rules or needing precise path and ordering control | Sites that prefer an authorization rule and have the required directives permitted |
| Resistance to spoofing | Weak; the header is client-controlled | Weak; the header is client-controlled |
Neither method authenticates a crawler or proves its identity. Choose based on your existing Apache configuration and the scope you need, not on an expectation that one User-Agent check is harder to evade.
Understand the response and rewrite flow
The [F] flag is deliberately a denial, not a redirect. A matching request receives HTTP 403 Forbidden and the requested URL is not substituted. This makes the result easy to recognize in logs and command-line tests. If your site has a custom error document, Apache may render that document while retaining the forbidden status.
[L] ends the current rewrite round, so later rules in that round are skipped. It does not promise that no later per-directory pass can occur after an internal rewrite. That is why Apache 2.4’s [END] flag can matter in a complicated file: it prevents subsequent per-directory rewrite processing for the request. Use the strongest termination behavior only when it fits the rest of the configuration.
A catch-all front-controller rule can obscure the intended result if it runs first. Keep the User-Agent condition and denial rule above broad application rewrites, then test a real URL that exercises the same path. Do not assume that a visible 200 response from an application means the condition was evaluated; inspect the status and the error log.
Scope, matching, and maintenance details
Decide whether the policy is about an identity token or a location. The condition chooses the client string; the RewriteRule chooses the path. In a document-root file, ^ covers every path handled there, while ^private/ covers only that subtree. In a nested file, Apache strips the directory prefix before matching, so the same text has a different meaning. Write down the file’s directory and the intended URL examples before editing.
Free tools Windows power users keep installed
One-click scans. No signup required.
Rank #3
- [Low Noise High Fidelity] New bee guitar cord with 22AWG oxygen-free-copper center conductors provides enhanced signal transfer, the PVC insulation material and the high-density stranded copper double shield provide noise rejection for quiet operation
- [Ideal for Choice] New bee 1/4 straight and right angle metal connectors, ensures a secure connection in any instrument, pedal, or amp jack. L shape great fro on stage guitar stands
- [Wide Compatible] Suitable for mixer, power amplifier, microphone, effector, decoder, equalizer, electoronic organ, electric guitar, electric bass and other electronic equipments. Widely used in the installation of smart home, karaoke speaker, professional stage audio, etc
- [Easy Disassemble] Making incidental repairs quick and easy! Very nice, flexible cable
- [Dont Hesite to Order] 24 hours customer services and professional technology team are standing by
Use an anchored expression when the token is consistently first, as in ^NameOfBadRobot. Use an unanchored but distinctive fragment when the client adds another product token before its name. Avoid very short words and generic terms. Keep alternatives separated with |, but remove entries that no longer appear in logs; a long list is harder to audit and more likely to catch an unrelated client.
Version strings are poor long-term match targets because clients change them. Match the stable product token instead, and anchor only when the client’s format is reliable. If punctuation is part of the token, copy the exact stable portion from a log and test it with a harmless request before deploying.
Hosting and deployment considerations
Apache-compatible hosting is relevant because .htaccess works only when the server permits directory overrides. Shared hosting may expose no main configuration access, making this file the practical place for a local rule, but the provider still controls whether the needed directives are enabled. A hosting control panel that accepts a file named .htaccess does not by itself prove that mod_rewrite, SetEnvIfNoCase, or authorization directives are available.
Ask the administrator which method is permitted if Apache reports a directive error. Do not keep retrying syntax changes when the limitation is policy. If you can edit the main virtual-host configuration, that context may offer a more centralized control; if you cannot, preserve the smallest working directory rule and document its intended scope.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteImplement the block safely
- Identify the client. Inspect access logs and select a distinctive User-Agent fragment. Prefer a full product token or anchored prefix over a generic word.
- Define the scope. Decide whether every URL should be denied or only a subtree. In a document-root file, use
^for all paths or a path prefix such as^private/. - Back up the file. Copy the current
.htaccesswith its permissions intact. Keep a separate copy of any CMS or framework rewrite section. - Insert the smallest rule. Place the block before broad catch-all rewrites. Start with
[F,L]; consider[F,END]only when Apache 2.4 behavior and the rest of the file justify it. - Save and make a controlled request. Test a blocked User-Agent and a normal one before announcing the change or relying on it for traffic control.
- Review the error log. A syntax or directive error can make the entire site return 500. Restore the backup immediately if that happens, then correct the rule in a staging or maintenance context.
Test the result with curl
Use a request that sends the exact header you intend to match. Replace the URL with a real path on your site:
curl -I -A "BadBot" https://www.example.com/
curl -I -A "Mozilla/5.0" https://www.example.com/The first request should receive HTTP/1.1 403 Forbidden (or the equivalent HTTP/2 status display) when the rule matches. The second should follow the site’s normal response path. Test a URL inside and outside the targeted subtree if you used a path restriction.
To test case-insensitivity, vary the capitalization:
curl -I -A "badbot" https://www.example.com/
curl -I -A "BADBOT" https://www.example.com/Both should be denied when [NC] is present. If your site redirects HTTP to HTTPS, test the final HTTPS URL and inspect each response with -I or -v; a redirect from another layer can otherwise be mistaken for a successful pass.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Also test a header that should not match, a versioned form of the target token, and a request to a neighboring directory. These checks reveal accidental substring matches and directory-scope mistakes without requiring live crawler traffic.
Rank #4
- Digital Family Calendar: BSIMB digital calendar wall touch screen puts an end to your chaotic life. Designed for busy families, the digital calendar keeps you organised and in step with your family. Assign different colours and tasks to each family member to make family scheduling easier and more intuitive.
- Easy Setup and Auto-Sync: Plug in, connect to WLAN and auto-syn your electric calendar in minutes. Digital wall calendar compatible with Google calendar, iCloud calendar, Outlook calendar, Cozi calendar and Yahoo calendar. Add events directly on the device or through the ecalendar app. Electronic calendar stays current whether you're at work or traveling—no manual updates needed.
- All-In-One Home Assistant: Smart calendar builds habits and encourage independence with personalised chores. Display daily meal plan to reduce the daily ‘What's for dinner?’ stress. To-do lists keep track of everything you want. Turning chores into valuable achievements with stars and rewards, making life easier and more enjoyable.
- Manage Anytime Anywhere: BSIMB electronic calendar for family stay connected with the powerful ecalendar app. Access and manage your digital wall calendar anytime, anywhere with free features like stars & rewards, photo screensaver, chore charts, meals plans, to-do lists etc. Electronic planner paid features like Magic Import tailor advanced tools for users with special needs.
- Stay in Touch with Loved Ones: Share photos or videos from your phone to electronic wall calendar with app, instantly update your life moment, and keep in touch with your family, especially those who are far away; you can also surprise your smart calendar digital wall by sending him a wish card; it's also great for keeping tech-illiterate elders in touch with the youngsters.
Troubleshoot common failures
The bot still receives a normal page
- Confirm that the request reached Apache and that the edited file is in the directory serving that URL.
- Check that rewrite support is loaded and that
AllowOverridepermits the directives. - Verify the actual header in the access log. Crawlers often change capitalization, add version text, or use a different token on different requests.
- Look for a broad rule above the block that redirects or rewrites the request. Move the User-Agent block earlier, preserving required framework rules.
- Check the regular expression for punctuation, spaces, or an anchor that is too strict. A prefix expression such as
^NameOfBadRobotwill not match a header that begins with another product token.
Legitimate visitors are blocked
Review the fragment for short or common words. Replace it with a distinctive token and test several ordinary browser headers. If multiple names are grouped with |, test each alternative separately and remove any that produce false positives.
Every request returns 403
Inspect the condition and path pattern. A condition that is too broad, or a rule copied into a directory where ^ covers all paths, can deny more traffic than intended. Temporarily reduce the expression to one known token, test, then add alternatives one at a time.
The site returns 500 after saving
Read the Apache error log first. Typical causes include a misspelled directive, a flag unsupported by the installed Apache version, or a directive forbidden by the override policy. Restore the backup to recover service, then validate a minimal rule before adding complexity.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsThe SetEnvIf version is rejected
The authorization example depends on the directives being available in that context. A host may permit rewrite rules but not environment or authorization directives. Ask the administrator which directives and override classes are enabled, or use the permitted method.
Useful expression patterns
Several known clients
RewriteCond %{HTTP_USER_AGENT} (BadBot|AnotherBot|CrawlerX) [NC]
RewriteRule ^ - [F,L]Keep the alternatives readable and periodically remove names that no longer appear in logs. Grouping everything under a generic term increases accidental matches.
A client with punctuation or a version token
Match a stable, distinctive portion instead of copying an entire version string. Versions change frequently; a stable product token is easier to maintain. Anchor it with ^ only when the client consistently places that token first.
A sensitive directory only
RewriteCond %{HTTP_USER_AGENT} ^NameOfBadRobot [NC]
RewriteRule ^private/ - [F,L]This leaves public URLs available while denying the selected client under private/. Confirm that the file is in the document root; directory placement changes what the rule pattern sees.
What User-Agent blocking cannot do
Apache’s documentation gives the central limitation plainly: “Any technique that relies on the USER_AGENT string can be trivially circumvented, since that string can be changed by the client.” A determined scraper can send a normal browser string, rotate identities, or omit useful identifying information. The rule therefore filters self-identifying or low-effort unwanted clients; it does not establish who is connecting.
Do not use a User-Agent rule as the sole protection for credentials, private data, administrative functions, or an API that requires reliable client identity. For sustained attacks, Apache recommends considering a higher-level control such as a firewall. Depending on the threat, you may also need IP reputation controls, rate limiting, authentication, application-level bot controls, or a CDN/WAF. Those controls address different signals and should be selected according to your traffic pattern and operational authority.
Maintain the rule over time
- Watch logs after deployment. Confirm that the intended token appears and that legitimate clients are not being denied.
- Keep the expression narrow. Every additional alternative increases maintenance and false-positive risk.
- Document the reason. Add a short comment with the date and the observed token so another administrator knows why it exists.
- Review application rewrites after upgrades. CMS, framework, or Apache changes can alter rule order and per-directory behavior.
- Retest rollback. Keep the known-good file available and know how to restore it through your host’s file manager, deployment system, or shell access.
A User-Agent denial is appropriate when you need a quick, transparent filter in a directory you control. Apply it narrowly, place it before competing rewrites, verify the response with a controlled request, and treat the header as an easily forged hint rather than a security credential.
Frequently Asked Questions
Can I block a bot without mod_rewrite?
Yes. Apache supports SetEnvIfNoCase User-Agent together with a <RequireAll> block that grants access generally and excludes requests carrying the marker variable. Both directives must be permitted in the directory context.
Why does RewriteRule ^ match every URL?
In a document-root .htaccess, Apache removes the directory prefix before evaluating the pattern, so every per-directory path begins at the start position. A nested file sees only the path below that directory.
Should I use [L] or [END]?
Use [L] to stop the current rewrite round. On Apache 2.4, [END] also prevents later per-directory rewrite processing. Choose [END] only after checking that your existing rewrite flow does not depend on another pass.
What status should a blocked User-Agent receive?
The [F] flag returns HTTP 403 Forbidden and leaves the URL unchanged. Confirm the status with curl -I or a verbose request rather than relying only on the response body.
How do I avoid blocking legitimate browsers?
Use a distinctive product token from your logs, avoid short generic fragments, and test each alternative separately. Review logs after deployment and remove patterns that create false positives.
Will this stop a determined scraper?
No. User-Agent is a client-supplied header and can be changed or omitted. Treat the rule as a filter for self-identifying or low-effort clients, not as authentication or proof of identity.
Why did adding the rule cause a 500 error?
Check the Apache error log for a misspelled directive, an unsupported flag, or an override policy that forbids the directive. Restore the known-good backup, then test a minimal rule in a controlled environment.
Can my hosting provider disable this method?
Yes. The host controls AllowOverride, loaded modules, and which directive classes are available. Ask which method is permitted when a valid-looking rule is ignored or rejected.
The Bottom Line
Use RewriteCond on %{HTTP_USER_AGENT} and RewriteRule ^ - [F,L] for a straightforward 403 block, or use SetEnvIfNoCase with Require when authorization directives are available. Scope the pattern to the correct directory, place it before catch-all rewrites, back up the file, and verify with controlled requests. Because User-Agent is client-controlled, stronger threats require controls beyond this header check.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
