TheModelDesk

Ploy has transitioned its AI website builder from Claude Opus 4.8 to GPT-5.6 Sol, the latest flagship model from OpenAI. The migration follows extensive head-to-head testing, where GPT-5.6 Sol outperformed the previous default in both speed and quality. The AI agent handles complex tasks including page planning, codebase analysis, component writing, and visual verification.

Performance metrics show significant gains: build times decreased by 2.2x (from 8 minutes to 3 minutes 42 seconds), and operational costs dropped by 27% (from $3.06 to $2.22 per build). Additionally, GPT-5.6 Sol proved more efficient with token usage, producing leaner code and significantly fewer CSS variables compared to Claude Opus while maintaining or improving visual scores.

The team noted that the migration revealed hidden provider-specific biases in their evaluation harness. Initial failures were often caused by the harness being tuned to Claude's sequential tool-calling style, whereas GPT-5.6 Sol utilizes parallel calls and batched file reads. This discovery emphasized the need to triage traces and adjust evaluation frameworks to avoid grading new models based on how well they imitate the previous ones.