You can collect product prices, availability, and GTINs only after checking the particular retailer’s terms and access rules, and you need to preserve enough context to identify exactly what each observation describes. Robots.txt is a crawler instruction file—not legal clearance. Record the displayed offer, product and variant, timestamp, and a distinct availability or request-failure state so you can validate comparisons instead of treating missing data as a price.
Check the retailer’s terms before collecting data
There is no single permission answer that applies to every store, jurisdiction, collection method, or use. Review the retailer’s current terms for the specific site section and fields you intend to collect; also check its robots.txt directives. The UNECE International Working Group on Price Indices advises practitioners to check both when assessing whether a website is eligible and what information can be collected (UNECE, International Working Group on Price Indices).
As an Amazon Associate I earn from qualifying purchases.
For each source, keep a record of when you checked, which terms applied, the relevant language, the site section and fields in scope, and any conditions such as login or region. Recheck when the terms or collection scope changes. A retailer’s published restrictions are specific to that retailer: for example, Target’s Terms & Conditions restrict certain automated extraction and commercial uses involving product listings and prices. That example does not establish another store’s rules.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minuteUse robots.txt as a technical signal, not permission
Robots.txt communicates crawling instructions to crawlers; it does not settle whether collection is allowed under a site’s terms or applicable law. Nor is it a reliable way to keep a URL out of search results: Google explains that a blocked URL may still appear in results if it is discovered elsewhere (Google Search Central’s robots.txt guide).
#1 Best Overall
- Larger battery enables longer continuous usage and twice the stand-by time. With the unique battery indicator light showing the remaining battery level, no more Low Battery Anxiety.
- The curved handle is extended and widened. With specially designed smooth and flat trigger for a better grip.
- The orange anti shock silicone protective cover can prevent scratches and friction even when dropped from up to 6.56 feet. IP54 technology protects the wireless barcode scanner from dust.
- Plug and play with the USB receiver or the USB cable, no driver installation needed. Easy and quick to set up. Wireless transmission distance reaches up to 328 ft. in barrier free environment.
- Supports almost all 1D Barcodes: Febraban Bank Code, Codabar, Code 11, Code93, MSI, Code 128, EAN-128, Code 39, EAN-8, EAN-13, UPC-A, ISBN, Industrial 25, Interleaved 25, Standard 25, Matrix. Reads damaged, fuzzy, reflective and smudged barcodes.
If your software identifies itself as a crawler, interpret the directives relevant to that user agent and the particular site. Crawler-specific details should not be generalized: Amazon’s documentation, for instance, says AmazonProductDiscoverybot respects user-agent and disallow directives but does not support crawl-delay, nofollow, or noindex (Amazon Product Bot). Those statements describe that crawler, not a universal robots.txt standard or an authorization to collect data.
Keep each observation interpretable
A price record is useful only if an analyst can tell what was observed, where, and when. UNECE describes a typical scraped-data structure that includes date, retailer, website category, item name, price, and item count. Eurostat’s HICP guidance also identifies a unique item URL or code, product type, name, description, and offer price as useful parameters (Eurostat, Practical guidelines on web scraping for the HICP).
Rank #2
- Plug and play, This laser handheld barcode scanner has simple installation with any USB port and Ideal for businesses, shops and warehouse operations. Its function is unbeatable and easy to use, design is stylish
- Compatible with Windows, Mac, and Linux; works with Word, Excel, Novell, and all common software
- Scanning Speed: 200 scans per second. Scanning angle: Inclination angle 55°, Elevation angle 65°. Operational Light Source:Visible Laser 650-670nm.
- Decode Capability: Code11, Code39, Code93, Code32, Code128, Coda Bar, UPC-A, UPC-E, EAN-8, EAN-13, ISBN/ISSN, JAN.EAN/UPC Add-on2/5 MSI/Plessey, Telepen and China Postal Code,Interleaved 2 of 5, Industrial 2 of 5, Matrix 2 of 5, etc ; 300 configurable options for prefix, suffix and termination strings, support turn on/off the beep.
- Color: Black. Dimensions: 3.6 x 2.6 x 6.1 inches. Type of Cable: 2M or 6ft straight cable. Shock: 1.5m drop on concrete surface. Regulatory Approvals: FCC CE.
| Record field | What to preserve | Why it matters |
|---|---|---|
| Observation time | Date and time, timezone, and collection or run identifier | Lets you distinguish one observation from another and trace updates or errors. |
| Source and item reference | Retailer, page URL or item code, and website category | Identifies where the record came from and helps find the same listing again. |
| Product identity | Name, description, brand, variant attributes shown, and GTIN if published and verified | Helps distinguish products and variants whose titles may look alike. |
| Offer | Displayed price and currency as shown on the page | Preserves what was observed without implying it is a guaranteed checkout price. |
| Availability and request state | Observed availability, plus separate unknown or request-failure states | Prevents unavailable items and interrupted requests from being recorded as zero-priced products. |
| Collection metadata | Relevant run details and notes needed to trace changes or failures | Supports investigation when records or site behavior change. |
Keep “unavailable,” “unknown,” and “request failed” distinct from one another and from a numeric price. Eurostat specifically calls for handling non-availability and server problems; separating these states in a timestamped dataset is a practical way to avoid silently converting missing observations into misleading values.
Match the exact product before comparing prices
A product title is not necessarily unique or stable: retailers may abbreviate it, revise it, or use similar names for different sizes, colors, or configurations. Prefer an exact GTIN when the value is supplied and can be matched to the product and variant. Otherwise, retain the retailer’s stable item code and the page’s supporting attributes. Eurostat recommends a stable identifier over time and notes that a GTIN or shop item code can be used to track the same products.
Rank #3
- Continuous Usage All Day: The EY-H2 USB barcode scanner is designed to always be ready for the next scan, which significantly reduces downtime and repair costs; it shortens checkout lines, improves customer service, and boosts business productivity
- Plug and Play: Eyoyo wired barcode scanner is connected via a USB cable, with no need to install any driver or software; It offers effortless connection and is compatible with Windows, Mac, Android, and Linux; Seamlessly works with Quickbook, Word, Excel, Novell, and all common software
- Supports Multiple 1D/2D Barcodes: Eyoyo QR code scanner scan with most 1D 2D barcodes with ease; 1D Barcodes: EAN, UPC, Code 39, Code 93, Code 128, UCC/EAN 128, Codabar, Interleaved 2 of 5, ITF-6, ITF-14, ISBN, ISSN, MSI-Plessey, GS1 Databar, Code 11, Industrial 25, Matrix 2 of 5, etc. 2D Barcodes: QR, DataMatrix, PDF417, and so on
- Supports Screen Scanning: The Eyoyo 2D scanner is capable of reading barcodes from smartphone screens, such as mobile coupons, digital wallets, and digital loyalty cards; Before scanning, simply turn your screen brightness to the maximum
- Sturdy Anti-Shock and Durable Design: The Eyoyo 2D barcode scanner features an ergonomic design made of high-quality ABS, enabling it to withstand repeated drops from 5 ft/1.5 m high onto the concrete ground; The durable plastic material ensures a long service life
Use GTINs as assigned, not inferred
GTINs are product identifiers assigned by manufacturers. Google’s product-data guidance identifies GTIN, MPN, and brand as common identifiers; it describes UPC as GTIN-12 and EAN as typically GTIN-13. Its instruction is explicit: “Don’t make up, guess, or include values from similar products.” See Google Merchant Center Help’s guidance on unique product identifiers and Google Search Central’s product-information guidance.
- Preserve the GTIN exactly as published, along with where it appeared.
- Check that it belongs to the exact product and variant being recorded; do not substitute a similar item’s identifier or a retailer’s internal code as a GTIN.
- If no GTIN is assigned or you cannot verify the value, leave the GTIN field blank or mark it unverified according to your schema. Do not manufacture a value from a title, barcode format, or another product’s listing.
Validate collection runs and comparisons
Site layouts and product offers can change, and automated requests can fail. After a run, check the data for missing or newly appearing products, duplicate identifiers, abrupt changes in record counts, suspicious or zero prices, outliers, and request interruptions. Eurostat recommends monitoring validity and usefulness, preparing for non-availability and server problems, and checking for duplicates, suspicious values, and outliers.
Rank #4
- Widely Compatible: Bluetooth Barcode Scanner for iPhone iPad Android Tablet PC, Support HID / SPP / BLE mode via bluetooth, Work with Windows XP/7/8/10, Mac OS, Windows Mobile, Android OS, iOS, Linux.
- Strong Recognition Ability: With the 2500 pixels high-resolution CCD sensor Engine, Rapidly decodes all 1D and stacked barcodes (including ISBN book), even worn, damaged or tightly spaced codes. Scan 1D codes directly from paper or screen, such as a computer monitor, smartphone, or tablet, or scan through glass surfaces, plastic shrink wrap, a CCD scanner is likely the best way to go.
- Automatic Scanning: NT-1228bc barcode scanner have three scanning modes: manual trigger mode, continuous scanning mode and auto-sensing scanning mode. In addition, there is a storage mode. Storage mode can be used when you are out of range of Bluetooth and wireless connectivity. Supports storage of up to 100,000 barcodes. Note: Before use, you need to scan the corresponding setting barcode on the manual.
- 2600mAh Battery Upgraded: Continuous scanning up to 200,000 times on a full charge. After a full charge the scanner can be used for one month at least, even in warehouses and at pos checkout counters where scanners are frequently used. In libraries and hospitals it can be used even longer.
- Programmable Configuration: Add custom prefixes/ suffixes, delete characters, Add keyboard keys/ combinations (terminator TAB, CR&LF, Home etc.), Enable or disable the barcode type as you want. Buzzer can be set to mute to allow for a quiet operation.(Note: It does not work with square POS / Divalto / DoorDash / Lightspeed POS system)
- Investigate duplicate IDs associated with different products or variants before merging their histories.
- Compare record counts and field completeness between runs to spot collection failures or site changes.
- Review anomalous values against the source page and preserve uncertainty rather than silently correcting or discarding them.
- When evaluating sources or collection approaches, compare the same factors: terms and scope, product coverage and fields, freshness and timestamp quality, exact identity matching, availability and error handling, and validation and change detection.
A displayed offer price is a time-specific observation from a particular source, not a promise of the price at checkout. Keep its timestamp and source attached; offers and site content can change.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →What this process does—and does not—establish
Checking a retailer’s terms and technical access signals, recording observations carefully, and validating the data are responsible collection practices. They do not establish a universal legal rule for scraping prices or availability. The applicable answer can depend on the retailer’s current terms, the collection details, geography, and intended use; when those stakes matter, seek advice appropriate to the situation.
Quick Recap
Best Value
- CCD Image Scanning Technology - NetumScan 1D barcode reader is equiped with advanced CCD sensor, which can quick capture 1D codes from paper and screen, including CODE128, UPC/EAN Add on 2 or 5, that can read even deformed barcodes, i.e. smudged, damaged, fuzzy, reflective barcodes, etc. Reading faster and more accurate than laser scanner.
- Sturdy Anti-shock and Durable Design - Ergonomic design with high-quality ABS making it can support withstand repeated drops from 2m high to the concrete ground, durable to use. Durable plastic material guarantees long service life.
- Three scanning mode - Key trigger mode + Auto-induction mode + Continuous Mode. There is no need to pull the trigger in auto-sensing mode and continuous scanning. Sometimes the self-sensing scanning function is in the inactive stage, please contact us and be at your service at any time.
- Supported 1D Bar Code - 1D Decode Capability: UPC-A, UPC-E, EAN-8, EAN-13, ISSN, ISBN, Code 128, GS1-128, Code39, Code93,Code32, Code11, UCC/EAN128, Interleaved 2 of 5, Industrial 2 of 5, Codabar(NW-7), MSI, Plessey, RSS, China Post, etc.
- Widely Use Range - This NetumScan Handheld USB barcode scanner can be used in supermarkets, convenience stores, warehouse, library, bookstore, drugstore, retail shop for file management, inventory tracking and POS(point of sale), etc.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
