DEV Community author Fortaki says they assembled a multilingual and programming-code dataset for Central Asian AI that occupied nearly 150 GB uncompressed and about 27.7 GB as a compressed archive. The account describes ten hours of maximum-profile .7z compression, repeated out-of-memory errors and operating-system freezes—but it does not provide enough hardware or settings detail to reproduce the process or promise a way to avoid those failures.
What the dataset is—and what the size figures mean
Fortaki describes the collection as technical text and source code in six human languages—Russian, Kyrgyz, Kazakh, Uzbek, Tajik and English—and four programming languages: Python, C++, Rust and Go. The author reports nearly 150 GB uncompressed and approximately 27.7 GB compressed. The linked Hugging Face card lists 150 GB total size and 27.7 GB total file size, broadly matching those headline figures. Fortaki’s DEV Community account and the Hugging Face dataset card are the sources for these claims; the measurements have not been independently audited here.
As an Amazon Associate I earn from qualifying purchases.
The article’s itemized figures do not account for the full headline total:
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstall| Category | Size reported by Fortaki |
|---|---|
| Russian | 60 GB |
| Kyrgyz | 23 GB |
| Kazakh | 22 GB |
| Uzbek | 7 GB |
| Tajik | 7 GB |
| English | 1.5 GB |
| Programming code | 20 GB |
| Sum of listed categories | Approximately 120.5 GB |
The category values total approximately 120.5 GB, around 29.5 GB short of 150 GB. Neither the article nor the card explains the difference, so the breakdown should not be treated as a complete accounting of the reported total.
#1 Best Overall
Why compression became a memory problem
Fortaki says they used maximum-profile .7z compression in an attempt to shrink the collection. In the author’s words, “The process took 10 agonizing hours.” The account also reports repeated out-of-memory errors and system freezes during that effort.
That is a description of one person’s run, not a reproducible benchmark. The post does not specify the computer’s memory, processor, operating system, compressor version, exact settings or the configuration that ultimately produced the archive. It therefore cannot establish which resource constraint caused each failure or provide a tested setting that readers can copy. The reported 27.7 GB archive is the author’s result, not a guarantee that maximum compression will achieve the same size or finish successfully on another machine.
What the files and repository are reported to contain
The article says the extracted files are .txt and .jsonl, with language material and code in the languages listed above. The Hugging Face card describes directories named /python, /rust, /cpp, /go and /docs, and says the layout supports training and validation splits. Those are descriptions from the author and card; the card’s viewer reported that it could not detect supported data files. The directory layout, splits and current archive contents therefore cannot be confirmed from the reviewed pages alone.
What the dataset might be useful for
Fortaki proposes using the collection for code-model fine-tuning involving comments and technical documentation in Central Asian languages, technical translation, and continued pretraining or domain adaptation. The Hugging Face card also names cross-lingual code understanding, technical documentation, NLP and cross-lingual information retrieval.
Rank #3
These are intended applications, not demonstrated results. Neither page provides evaluation scores, a benchmark comparison or evidence that training on the dataset improves a model. Teams considering it would need to inspect the files and run task-specific quality and performance checks before relying on it.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.License claims and provenance questions
Fortaki states that the project is distributed under CC BY 4.0, and the Hugging Face card also names that license. The card characterizes the materials as open-source components, public-domain texts and synthetic benchmarks. These descriptions do not amount to an item-by-item provenance inventory or establish that every upstream source permits redistribution under the stated arrangement.
Rank #4
The reviewed pages do not document the complete source list, filtering and deduplication procedures, or a license review for each component. Anyone redistributing the collection or using it in a commercial project should verify the relevant files’ origins and applicable terms rather than relying solely on the overall license label.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Quick Recap
What to verify before building on it
- Confirm that the archive can currently be downloaded and unpacked; the dataset card viewer’s inability to detect supported files leaves current usability unconfirmed.
- Check that the actual language and code contents, file formats and directory structure match the descriptions.
- Reconcile the roughly 120.5 GB itemized breakdown with the nearly 150 GB headline figure.
- Review sources, filtering, deduplication and upstream license terms for the data you plan to use.
- Evaluate coverage and quality for your intended task instead of treating proposed applications as validated outcomes.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
