A researcher has AI models build and simulate port cities. (AI generated image: Nano Banana 2 / )
Typically, AI models are evaluated and compared against each other using benchmarks. However, the pure numbers are often not very meaningful for laypeople and can sometimes be easily manipulated. AI researcher Ethan Mollick from the Wharton School at the University of Pennsylvania has therefore come up with a slightly different test for current AI models.
What port cities say about an AI
On his website “AI Harbot Town Gallery” he shows how the models behave when they build a port city and simulate it over a long period of time. In this context, Lang means in the period from 3,000 BC. to 3,000 AD. According to the prompt, the port city should “look beautiful” and can be controlled by users within the gallery.
Top Article
${content}
${custom_anzeige-badge}
${custom_tr-badge}
${section}
${title}
Mollick writes in a Linkedin post: “I believe these tests are surprisingly powerful because the challenge requires coding, design, decisions and knowledge.” The tested models go back to GPT-3.5 Turbo, which was released in March 2023. But current models such as Claude Fable 5 and GPT-5.6 Sol Pro are also being put to the test in the port cities.
You can easily see what the AI ultimately delivers on the website. Most simulations can be started, stopped and fast-forwarded using a timeline. Mollick himself distributes scores in two categories: for the quality of the generated code and for the general design. Clicking on each score reveals a detailed explanation of how Mollick came to this conclusion. In addition, the AI researcher always states how many attempts the model needed for this output and whether there were any errors.
Which AI models are good port architects?
GPT-3.5 Turbo and GPT-4 performed worst in the test. Both AI models needed further instructions after the first prompt so that they could at least begin the task. However, their works are anything but full-fledged simulations. GPT-3.5 Turbo only generated code for a 3D cube, while GPT-4 created a flat green area with several colorful blocks on it.
Claude 4.8 Opus Extra performed best so far. Under the title “Aldermouth: A Harbor in Time Lapse,” the AI not only created a simulation in which houses on a coast grow, spread and become more modern. The AI has also built elements such as initial ships from Pilgrims to larger merchant frigates with code. In addition, the simulation can be controlled directly via a timeline and an alternative version of the city shown can even be created at the push of a button.
The best tools for local AI:
Local AI: No problem with these 6 tools
