3 AI Coding Models, 1 Android Game: Who Actually Wins?

Learn more：https://treerouter.com

Abstract

Simple algorithm exercises and todo‑list applications cannot fully expose the real‑world engineering limits of large coding models. Complex 3‑D interactive projects represent rigorous end‑to‑end testing, because they combine game‑loop logic, three‑dimensional mathematics, touch‑event handling, state management, and mobile‑device adaptation. This article describes a controlled practical benchmark where Step 5 Preview, DeepSeek V4 Pro and GLM‑5.3 were tasked with building a Three‑js‑based 3‑D parkour game packaged for Android WebView. Test metrics include first‑attempt runnable success rate, number of dialogue rounds required for complete playable status, volume of manual code repair, code‑standard compliance, and mobile‑platform awareness. This report also documents real‑world deployment pitfalls encountered when running GLM‑5.3 locally with vLLM Docker images, and outlines distinct suitability profiles for each model. For teams evaluating multiple coding‑specialized models, this test workflow can be reproduced to compare practical output quality rather than relying purely on public leaderboard figures.

1. Why Android‑Targeted 3‑D Parkour Game Works as a Rigorous Benchmark

Short algorithmic tasks frequently appear in model training corpora. Models can produce plausible‑looking outputs through pattern‑matching without demonstrating genuine engineering judgement. In contrast, building a functional mobile‑oriented Three.js parkour game brings together multiple orthogonal engineering requirements:

1. Game‑loop timing: Correct implementation of requestAnimationFrame with delta‑time calculations to decouple object movement speed from frame‑rate fluctuations.
2. 3‑D mathematical logic: Scene assembly, camera tracking, vector transformations and collision‑detection spatial judgements.
3. Touch‑input handling: Differentiating swipe gestures versus single taps through touchstart, touchmove and touchend events for mobile‑device interaction.
4. State and resource management: Organizing geometries, materials and audio assets; implementing clean state transitions for start‑game, runtime, game‑over and restart workflows.
5. Mobile‑specific adaptation: Viewport configuration, high‑DPI screen adjustments, frame‑rate stability and proper memory cleanup.

Merely rendering a rotating 3‑D scene constitutes only a baseline pass. Real gaps between models emerge from handling implicit requirements: error‑resilient touch logic, performance throttling, and proper cleanup on game restarts — items not explicitly stated within the user prompt.

1.1 Unified Prompt and Evaluation Metrics

All three models received exactly the same prompt:
> Build a 3‑D parkour mini‑game for Android using Three.js and Android WebView packaging. The character automatically advances along the Z‑axis. Players control horizontal X‑axis movement via left‑right swipes. Random‑height, randomly‑spaced cuboid obstacles are generated. Collision triggers game‑over, showing total score alongside a restart button. Include a start‑screen, sound‑effects, scoring logic and progressive speed increase. Deliver self‑contained HTML that runs directly in browsers and provide step‑by‑step instructions for Android packaging.

Output token ceiling was uniformly set to 4096 to prevent premature response truncation. Five objective evaluation dimensions were defined:

1. First‑attempt runnable rate: whether generated HTML executes without runtime errors inside browsers and physical Android hardware.
2. Required conversation rounds to reach fully playable status (including start‑UI, gameplay, scoring, game‑over and restart).
3. Manual repair volume: approximate number of code lines manually modified by human evaluator.
4. Code specification quality: module separation, variable‑naming discipline, comments and constant‑definition practice.
5. Mobile‑platform awareness: proactive mitigation for touch conflicts, frame‑rate drops and viewport compatibility.

1.2 Rationale for Three.js plus Android WebView

Unity project generation is unsuitable for LLM testing. Models can output C# scripts, yet cannot create binary scene assets, prefabs and editor configurations, leaving most project‑setup labour to humans. Raw Kotlin plus OpenGL‑ES imposes steep debugging barriers for matrix‑transform and shader code, substantially lowering first‑pass success rates across all candidates.

Three‑js encapsulated within an Android WebView offers major practical advantages for benchmarking. The complete game can live inside a single HTML file. A minimal native Android wrapper can embed the web content into an APK package with only a small number of Kotlin lines. This keeps feedback cycles short and improves experiment reproducibility. Testing covers both code‑generation competence and the model’s comprehension of real‑world mobile‑software release workflows.

2. Test Environment Setup and GLM‑5.3 vLLM Deployment Challenges

Each model operated under different runtime conditions, a factor that must be considered when interpreting final results:

Step 5 Preview functions as a complete agent product rather than a bare model‑weight API; some of its observed strengths originate from surrounding agent‑loop infrastructure, not purely underlying model capabilities. GLM‑5.3 was deployed locally to eliminate remote‑API variance and investigate widely‑reported compatibility problems between GLM‑5.3 and older vLLM container images.

2.1 Critical vLLM Compatibility Pitfall for GLM‑5.3

Using a stable older vLLM Docker image produces fatal loading failures:

AssertionError: Unsupported architecture 'GLMForCausalLM'
KeyError: 'glm_arch'

The root cause is straightforward: legacy vLLM releases lack registry entries recognizing the new GLM‑5.3 model architecture. Investigators should inspect the architectures field inside config.json from model weights and cross‑reference against vLLM changelogs.

Recommended container‑image selection workflow:

1. For generally stable new‑model support: pull vllm/vllm‑openai:latest.
2. For bleeding‑edge models not yet included in formal releases: use vllm/vllm‑openai:nightly.

Additional caveat applies when using AWQ‑quantized model variants: mismatched versions between vLLM kernel and autoawq library will allow weight loading but trigger tensor‑parallel‑related inference failures, which are easily misdiagnosed as GPU‑memory exhaustion. Production deployments must pin exact image‑identifier strings rather than relying on floating latest tags.

2.2 Unified Inference and Android Runtime Settings

GLM‑5.3 and DeepSeek V4 Pro shared identical hyper‑parameters: temperature=0.3, top_p=0.95, max_tokens=4096. Since Step 5 Preview runs inside a managed agent sandbox, low‑level sampling parameters cannot be adjusted; constraints are enforced through prompt wording only.

All HTML outputs were loaded within one shared Android WebView configuration:

webView.settings.javaScriptEnabled = true
webView.settings.domStorageEnabled = true
webView.settings.mediaPlaybackRequiresUserGesture = false
webView.setLayerType(View.LAYER_TYPE_HARDWARE, null)

This guarantees that performance or behavioural discrepancies originate from generated source‑code rather than differences in embedding environments.

3. Per‑Model Practical Test Outcomes

3.1 Step 5 Preview: Strong end‑to‑end engineering‑cycle capability

Step 5 Preview delivered a self‑contained HTML file on its very first generation pass. Output included start‑screen UI, background music, collision sound‑effects, real‑time score display and progressive velocity increase. Velocity parameters were exposed as configurable constants instead of hard‑coded magic numbers. The initial build executed successfully both within desktop browsers and physical Android hardware.

Two mobile‑device defects appeared after real‑device validation: overly sensitive swipe handling and frame‑rate degradation when shadow‑mapping was active on low‑end hardware. After feeding these observations back as feedback, the second iteration introduced configurable touch‑sensitivity constants plus conditional shadow activation logic that adapts according to screen‑pixel density.

Several noteworthy engineering choices stood out in generated source‑code:

• Game‑loop calculations strictly employed delta‑time normalization via requestAnimationFrame, preventing speed distortion under variable frame‑rates.
• Collision detection applied simplified AABB bounding‑box approximations appropriate for game‑play feel rather than computationally expensive high‑fidelity mesh‑testing.
• Mobile‑adaptation measures included viewport‑fit=cover for notch‑screen compatibility, touch‑action: none gesture suppression and overscroll‑behavior settings eliminating unwanted page scrolling artefacts.

Restart logic correctly reset scores, obstacle collections and audio states. The primary downside was relatively verbose code (~600 lines), occasionally adding decorative features beyond minimal prompt requirements. Overall, Step 5 Preview demonstrated end‑to‑end engineering‑cycle strengths: it delivers complete playable products without extensive human stitching‑together of isolated fragments.

3.2 DeepSeek V4 Pro: Reliable execution given well‑defined requirements

DeepSeek V4 Pro also produced browser‑runnable HTML on the first pass, though it omitted start‑screen interfaces, audio feedback and varied obstacle spacing. It faithfully implemented items explicitly written in prompts, yet rarely inferred implicit product‑experience improvements.

After two rounds of targeted human feedback, developers added start‑UI, game‑over summary screens and sound‑effects. A further revision introduced 1‑second invincibility time‑stamp checks to eliminate duplicate‑score counting caused by repeated collision‑frame triggers.

Code organisation represented its greatest strength. Functions such as createScene, createPlayer, spawnObstacle, checkCollision possessed clean boundaries. Tunable constants were grouped together at file top. This structure simplifies subsequent human‑driven modification and debugging.

DeepSeek V4 Pro exhibits the profile of a dependable executor: it precisely carries out clearly‑specified demands, yet seldom volunteers supplementary design decisions. This behaviour works well for scenarios where engineers maintain detailed acceptance checklists.

3.3 GLM‑5.3 (local‑deployed): Good local‑control value with higher manual‑repair overhead

After switching to an updated vLLM nightly image, GLM‑5.3 completed local inference successfully. However its first‑pass output contained runtime defects: an undefined shadow‑map variable triggering ReferenceError, plus failure to clear obstacle collections on game restart, producing ghost‑obstacle artefacts. Human repair required approximately 30 changed source‑code lines.

Three dialogue rounds were needed for full playable functionality: repairing runtime exceptions, implementing offline sound‑generation using Web‑Audio‑API (avoiding external asset dependencies), and building segmented progressive‑speed logic. While multi‑round edits preserved existing long‑file context well, the model required extremely concrete phrasing for feature requests; vague wording like “add sound‑effects” risked generating file‑path‑dependent audio tags that malfunction offline.

For comparison purposes, the evaluator also ran limited parallel tests against doubao‑seed‑2.0‑code. GLM‑5.3 demonstrated superior retention of long‑file state through multiple editing iterations, making it better suited for larger continuous‑project work. doubao‑seed‑2.0 showed advantages for isolated local function‑level completion tasks.

Local‑hosted GLM‑5.3 grants data‑privacy advantages for sensitive proprietary‑code workloads. Nevertheless, operators must budget for extra operational overhead associated with container‑image version management, quantisation‑library compatibility and ongoing runtime troubleshooting.

4. Comparative Summary Table and Model‑Positioning

From this practical benchmark, three distinct usage profiles emerge:

1. Step 5 Preview (engineering‑cycle‑oriented): Best suited for rapid prototyping and MVP construction. It autonomously fills many implicit product‑quality gaps, at the cost of occasionally generating surplus code.
2. DeepSeek V4 Pro (stable‑execution‑oriented): Ideal when precise requirement‑checklists already exist. Produces well‑structured, human‑maintainable code and strictly follows explicit instructions without excessive speculative additions.
3. GLM‑5.3 (local‑control‑oriented): The primary benefit lies in private‑network inference without sending proprietary source‑code to external cloud APIs. Accept higher manual‑fix overhead and ongoing DevOps work for vLLM‑stack maintenance.

When organisations run comparative evaluations across multiple model endpoints, centralised access management lowers operational friction. A unified API gateway such as Treerouter consolidates multi‑model routing, credential handling and consumption logging, simplifying repeat benchmark workflows.

5. Conclusion

Benchmarks using trivial algorithm tasks can mask real‑world weaknesses that only surface under integrated mobile‑software‑engineering constraints. This Three‑js Android parkour‑game test exposed meaningful behavioural differences among Step 5 Preview, DeepSeek V4 Pro and GLM‑5.3 despite all three being capable of producing syntactically valid code.

Step 5 Preview demonstrates strong implicit product‑awareness and delivers near‑complete playable outputs with minimal human intervention. DeepSeek V4 Pro provides solid, maintainable implementation given thorough, well‑structured requirements. Self‑hosted GLM‑5.3 delivers acceptable results after manual patching, with the major trade‑off of complex local‑infrastructure maintenance including strict vLLM‑image‑version discipline.

No single model universally dominates every scenario. Engineering teams can adopt a mixed workflow: leverage agent‑oriented models for fast prototype iteration, use instruction‑faithful models for production‑grade implementation once requirements stabilise, and deploy local‑open‑weight models for privacy‑sensitive proprietary‑code work. Real‑world multi‑feature testing on target hardware remains indispensable, as static benchmark scores cannot fully predict runtime mobile‑device behaviour.