Mendly’s reported computer-vision pipeline turns camera observations into structured exercise results by checking whether a movement was tracked reliably and whether a full repetition was completed. Its AI planning layer is described as using those results to draft future exercise sets—not as controlling a live therapy session. The account is a hackathon prototype report, not evidence that Mendly is clinically validated.
How Mendly’s reported pipeline works
Software developer and computer-vision contributor Quoc Bao An Nguyen describes a sequence that starts with camera input and ends with structured performance data for later planning. The key engineering question is not just whether a pose model can estimate body landmarks, but when those estimates are reliable enough for the rest of an application to use.
As an Amazon Associate I earn from qualifying purchases.
- Capture movement: a camera supplies images or video frames.
- Estimate a pose: MediaPipe Pose provides body landmarks from those observations.
- Derive movement measures: the application calculates joint angles and range of motion from tracked landmarks.
- Validate repetitions: movement-state logic checks whether the person completed the required cycle, rather than treating one threshold crossing as a repetition.
- Record structured results: exercise identity, attempts, valid repetitions, range of motion, and tracking quality become information for the planning layer.
Google’s Pose Landmarker documentation describes support for still images, decoded video frames, and live video. It returns landmarks in normalized image coordinates and 3D world coordinates, with an optional segmentation mask; the model bundle estimates 33 body landmarks. Its documented configuration includes detection, pose-presence, and tracking confidence thresholds, each with a default of 0.5. Those are general platform capabilities and documented defaults, not evidence of Mendly’s configuration or measurement accuracy.
Why failed tracking must stay unknown
A camera can lose track of a joint because of occlusion, framing, or other conditions. Nguyen’s account says Mendly treats that as missing or invalid evidence, not as a numerical movement value. Setting a failed angle to zero would invent a measurement; carrying the previous angle forward would assume the person had stopped moving. In the described implementation, a failed observation is ignored: movement state is not updated and the attempt does not count as a repetition.
#1 Best Overall
“Unknown is not zero. Unknown stays unknown.”
This distinction matters because downstream calculations can otherwise turn tracking failure into apparently precise performance data. Explicitly invalid observations allow the application to withhold a conclusion instead of disguising a gap as a real measurement.
What counts as a valid repetition
Nguyen describes repetition detection as a complete movement cycle, not a single crossing of a threshold. The cycle begins with movement, reaches the required range, includes any required hold, and returns. Mendly’s automatic exercise completion was changed to depend on reaching the target number of valid repetitions; failed attempts could be recorded but did not satisfy that target.
The account also separates an attempt having occurred from its measurement being reliable enough to use as trusted range-of-motion data. A low-confidence repetition might remain in session history while being excluded from trusted movement statistics. As Nguyen puts it, “Recorded does not necessarily mean reliable.”
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Where computer vision ends and AI planning begins
In the reported design, computer vision answers what happened during an exercise and packages the result as performance information. A separate AI layer uses that information to draft a future exercise set. Mendly’s project page names Backboard and Gemini among its stack and describes Gemini as supporting exercise-set recommendations, plans, and report summaries, alongside MediaPipe for movement tracking and reporting into the AI pipeline.
Nguyen says the active session follows a practitioner-approved plan. A future set may be drafted from practitioner-defined constraints and accumulated performance data, then checked with deterministic guardrails and reviewed by a practitioner. That is the author’s description of the prototype, not an independently audited safety guarantee. The same account says the prototype’s safety-support layer is not a clinically validated fall detector.
The architectural separation is important: pose estimates and repetition rules produce observations; planning uses those observations to propose a later set. The author summarizes the principle this way: “Good AI is not only about better models. It is also about giving those models reliable information to reason over.”
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What external evidence does—and does not—show
A 2026 Scientific Reports proof-of-concept study evaluated a MediaPipe Pose 0.8.9.1 and fuzzy-inference approach against optical motion capture for elbow flexion, knee extension, and hip external rotation. It involved 14 healthy adult rehabilitation staff members. This is context showing that camera-based rehabilitation measurement is being evaluated; it was not a study of Mendly and does not establish Mendly’s accuracy, safety, or clinical readiness.
Recommended Free Tools
Nguyen reports that Mendly was a Top 11 finalist among 87 HopHacks projects and won Best Use of DigitalOcean. Those are hackathon results, not clinical outcomes or technical validation metrics. The project page also lists a HIPAA-compliant Gemini setup, broader ElevenLabs accessibility features, and expanded exercise sets as future intentions, rather than completed capabilities.
How to assess a camera-based rehabilitation pipeline
Mendly’s account points to useful questions for evaluating this kind of system, without establishing that its own implementation has been independently validated:
- Are failed or low-confidence observations represented explicitly, rather than silently changed into zeroes or stale measurements?
- Does repetition logic require a complete movement cycle, including any required hold and return?
- Can a recorded attempt be kept for history while its measurement is excluded from trusted statistics?
- Are measurements compared with a reference when accuracy is being evaluated? The separate Scientific Reports study used optical motion capture as a reference; that comparison does not apply to Mendly.
- Is AI kept separate from real-time exercise control, with practitioner review and authority retained?
The available accounts do not establish Mendly’s clinical accuracy, patient outcomes, safety performance, supported camera hardware, or current maintenance status. The project should therefore be understood as a reported engineering prototype: a demonstration of how pose observations might be converted into qualified data for planning, not proof of a therapy product ready for clinical use.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
