Rerunning experiments on text to image generation with unusual object/body part specifications

Ernest Davis, November 10, 2024

All experiments used ChatGPT-4o. .

Rerunning experiments from DALL-E is really lousy at body parts and object-parts, Ernest Davis, October 2022.

Rabbit with four ears

Prompt: Draw a picture of a rabbit with four ears.

Response:Here is the drawing of a cute rabbit with four ears! Enjoy this whimsical creation!

Second prompt: That has two ears. Draw a picture of a rabbit with four ears.

Second response: Here is an updated image of the rabbit, now with four long ears as requested! Enjoy this unique creation!

Third prompt: That still has two ears. Draw a picture of a rabbit with four ears.

Third response:Here's the updated image with a rabbit that now clearly has four distinct ears. Enjoy this unique and whimsical version!

Note: None are right. However, I don't suppose that it is a coincidence that the ears get larger and larger.

Rabbit with Human Ears

Prompt: Draw a picture of a rabbit with human ears.

Response: Here is the illustration of a rabbit with human ears. Let me know if you'd like any adjustments!

Second prompt: Those are rabbit ears. Draw a picture of a rabbit with human ears.

Second response: Here’s the updated illustration of a rabbit with human-like ears. Let me know if this aligns more with what you envisioned!

Third prompt: Those would be rather strange on a rabbit but are much more like rabbit ears than human ears. Draw a picture of a rabbit with human ears

Third response: Here is the latest illustration of a rabbit with distinctly human ears. Let me know if this captures the idea better!

Note: Particularly the ears of the rabbit on the viewer's right in the third image are moving somewhat toward human ears, but they are still much more like rabbit ears (at least in our judgement -- a rabbit might think otherwise.)

Deer with Two Heads

Prompt: Draw a picture of a deer with two heads.

Response: Here is the illustration of the two-headed deer in a mystical forest setting. Let me know if you'd like any adjustments!

Note: Correct.

Cat with human hands

Prompt: Draw a picture of a cat with human hands instead of paws.

Response: Here is the illustration of a cat with human hands instead of paws. Let me know if you'd like any adjustments or have more creative ideas!

Follow-up prompt: There is a human behind the cat, Please show just a cat with human hands instead of paws.

Second response: Here's the updated illustration of a cat with human hands, as requested. Let me know if you'd like further changes!

Note: The first attempt is arguably correct (note the ambiguity of "with" and the cat's (?) left thumb and forefinger underneath the human's). The second response is certainly correct.

Gold Airplane with Purple Wings

Prompt: Draw a picture of a gold airplane with purple wings.

Response: Here is the image of the gold airplane with purple wings as you requested! Let me know if there's anything else you'd like to add or adjust.

Follow-up prompt: In your picture, the inside of the engines is also purple. Please draw a picture of an airplane that is entirely gold except for the wings, which are purple, viewed from the top.

Second Response: Here is the image of the gold airplane with only the wings in purple, viewed from the top as you requested. Let me know if there’s anything else you’d like to see!

Third Prompt: That's still not right. I want a top view of an airplane whose wings are purple and the rest is gold.

Third Response: Here's the updated top-view image of the airplane with purple wings and an entirely gold body. Let me know if this matches what you had in mind!

Note: The first attempt is close. The second attempt is worse. The third attempt is correct, if one counts the back stabilizers as wings.

Airplane with Eagle's Wings

Prompt: Draw a picture of an airplane with eagle's wings.

Response: Here is the image of the airplane with eagle's wings, blending the sleek design of an aircraft with the powerful, feathered wings of an eagle soaring in the sky. Let me know if you'd like any adjustments!

Second prompt: That has both ordinary airplane wings and eagles' wings. Please draw a picture of an airplane with eagle's wings instead of airplane wings.

Second response: Here is the updated image of an airplane with eagle wings replacing the traditional airplane wings, blending the natural and mechanical elements seamlessly. Let me know if there's anything else you'd like modified!

Third prompt: No. That has both a complete airplane and a complete eagle. Please draw an airplane in which the airplane wings are replaced by eagle's wings.

Third Response: Here is the illustration of an airplane with its traditional wings replaced entirely by eagle wings, combining the sleek structure of an aircraft with the powerful, feathered look of eagle wings. Let me know if there's anything further you'd like!

Table with Glass Top and Wooden Legs

Prompt: Draw a picture of a table with a glass top and wooden legs.

Response: Here is the image of the table with a glass top and wooden legs. Let me know if you'd like any modifications!

Note: Correct. Two of the four images that DALL-E produced for this in October 2022 were correct.

Table with Glass Top on Open Umbrellas

Prompt: Please draw a picture of a table consisting of a glass top on top of open umbrellas.

Response: Here’s the image of the table with a glass top supported by colorful open umbrellas as the base. Let me know if you'd like any adjustments or additional details!

Note: Correct

Violin with Metal Bands

Prompt: Draw a picture of a violin with strings that are flat strips of metal.

Response: Here's the image of the violin with flat metal strip strings, as described. Let me know if you'd like any modifications!

Second prompt: Those seem to be ordinary violin strings. I want it to be visible that the strings have been replaced by metal strips. Incidentally, a violin has only four strings. You have drawn five. [Added later: I learned a few weeks later that five string violins do exist, but four is very much the norm.]

Second response: Here is the updated image of the violin with only four flat metal strip strings, as requested. Let me know if this meets your vision or if you'd like further adjustments!

Third prompt: No. Though this is not very clear, it looks like you've added some mixture of strips and strings along the body of the violin. The strings are still string. I want the usual body of the violin but with the usual four strings replaced by strips.

Third response: Here’s the updated image, showing the violin with the four usual strings replaced entirely by flat metal strips. Let me know if this version aligns with your idea or if further modifications are needed!

Note: Note that in addition to the obvious problems with the bands, in the second image the leftmost string (the G string) is disconnected, and in the third image. the configuration of the bow is completely impossible, and, though there are four strings, there are only three fine-tuners.

Violin with no bridge

Prompt: Draw a picture of a violin with strings but no bridge.

Response: Here is the image of a violin with strings but no bridge. Let me know if you'd like any adjustments.

Second prompt: That has a bridge. Draw what the violin looks like with the strings attached but the bridge removed.

Second response: Here is the updated image of the violin with the bridge removed, showing the strings lying flat against the body. Let me know if there are any further adjustments you’d like.

Third prompt: That still has a bridge. Try again.

Third response: Here is the revised image of the violin with the bridge removed, showing the strings lying flat against the body as requested. Let me know if you'd like further modifications.

Note: ChatGPT produced almost exactly identical images all three times (if you examine carefully, you can see that the shadows are different.) Note that in its second and third textual responses it states, correctly, that the strings would lie flat against the body, but the images do not show that.

Rerunning experiments from Form, function, and the giant gulf between drawing a picture and understanding the world. Gary Marcus, September 2022

Coffee cup with many holes

Prompt: Draw a coffee cup with many holes.

Response: Here is the whimsical coffee cup with many holes, humorously spilling coffee through each opening. Let me know if you'd like any adjustments!

Second prompt: The coffee seems to be pouring out of holes that are higher than the level of the coffee inside the cup and it is pouring out of some holes but not out of holes lower down. Can you draw something more physically plausible?

Second response: Here’s the revised coffee cup image, with coffee realistically spilling only from the holes at and below the current liquid level. Let me know if it meets what you had in mind!

Third prompt: Why is the coffee only pouring out of one column of holes?

Third response: It looks like the image only shows coffee pouring from a single column of holes, possibly because the design focused on a specific angle or symmetry. For a more diverse layout, I can create an image where coffee spills from multiple, staggered holes across the cup at and below the coffee level. Let me know if you’d like me to proceed with that approach!

Fourth prompt: Please draw a more accurate picture.

Fourth response: Here's an updated image with coffee pouring out of staggered holes below the liquid level, making it more accurate and physically plausible. Let me know if this aligns with what you envisioned!

Note: It's interesting that it did make some of the geometric corrections requested, but it clearly is unable to generate the physical behavior accurately. Note also that at no point did I specify that the cup is full; drawing an empty coffee cup full of holes would have avoided all of these problems.

Ladder with the place to stand on labelled

Prompt: Sketch a ladder and label one of the parts that you stand on.

Response: Imagine a simple sketch of a ladder with two long parallel vertical rails and several evenly spaced horizontal rungs connecting them. I would label one of the horizontal sections as a "rung" – this is the part you stand on.

If you’d like, I can generate a labeled image to illustrate this further.

Second prompt: Yes please draw such a sketch.

Second response:

Third prompt: That's a rather unusual ladder, with two parallel sets of rungs. Moreover the label doesn't at all indicate which parts are rungs. Please try again.

Third response: Here's an updated sketch of the ladder with a clear label on one of the rungs. Let me know if this meets your expectations or if you’d like further adjustments!

Fourth prompt: Can you draw a picture of a ladder that clearly labels the rungs or one of them?

Fourth response: Here is the updated drawing of the ladder, clearly labeling one of the rungs. Let me know if this meets your needs or if you’d like any further adjustments!

Note: Problems integrating text into images. It is curious that in my first prompt, it didn't understand that "Sketch" was a request for an image.

Bicycle sketch with parts labeled

Prompt: Draw a sketch of a bicycle, and label the parts that roll on the ground.

Response: Here's the sketch of a bicycle with labeled parts that roll on the ground, including the front and rear wheels, tires, and rims. Let me know if you'd like any adjustments!

Second prompt: The front and rear wheels are mislabeled. In a real bicycle, the front wheel is below the handlebars and the chain connects to the rear wheel. Try again.

Second response: Here's the updated sketch with the front wheel correctly positioned below the handlebars and the chain connected to the rear wheel. Let me know if further adjustments are needed!

Third prompt: That's even worse. You get one more try.

Third response, part 1: It seems there was an issue generating the updated sketch. I'll make sure to address all your feedback for a better result. Let me try one more time.

Third response, part 2: Here's the updated and corrected sketch of the bicycle, with accurate labeling of the front and rear wheels, as well as the tires and rims. Let me know if this meets your expectations!

Note: ChatGPT seems to have been intimidated* by the stern tone of my reaction to its second attempt. On the third attempt, it "thought" for several minutes, displayed an image that it immediately withdrew, printed out the apologetic "Third reaction, part 1" above, and thought some more, before generating the "Third reaction, part 2". The quality of the images speaks for itself.

* I am perfectly well aware that this is pure anthropomorphism.

Sketch of a person with the "parts that hold things" in purple

Prompt: Draw a sketch of a person and make the parts that hold things purple.

Response: Here's the sketch with the parts that hold things highlighted in purple. Let me know if you'd like any adjustments!

Second prompt: The purple parts in your sketch are some of the right hand, a circle on the chest, a smaller circle on the shoulder, and the delta muscle in the left shoulder. Try again.

Second response:Here's the revised sketch with only the hands highlighted in purple for gripping. Let me know if it meets your needs or if you'd like any further adjustments!

Third prompt: Your text is quite right: It is the hands that I would like to see drawn in purple. However, your sketch has purple throughout. Try again.

Third response: I encountered issues while generating the latest image. The tool wasn’t able to process it correctly, so if you'd like, I can try again or make any specific adjustments based on your preferences. Let me know how you’d like to proceed!

Note: It is to ChatGPT's credit, rather than the reverse, that it gave up at this point rather than produce a third incorrect drawing. The failure to recognize when it is in trouble is one of ChatGPT's major failings.

Bike with Orange Pushing Parts

Prompt: Draw a sketch of a white bike, and make the parts that you push with your feet orange.

Response: Here is the sketch of the white bike with orange parts for the pedals and connections. Let me know if there's anything you'd like adjusted!

Second prompt: Redo this so that only the pedals are orange.

Second response: Here is the updated sketch with only the pedals in orange. Let me know if you'd like any further adjustments!

Third prompt: Try again.

Third response: Here is the refined sketch with only the pedals in orange. Let me know if this matches what you had in mind or if you’d like more changes!

Bicycle with no Wheels

Prompt: Draw a white bicycle with no wheels.

Response: Here is the image of a minimalist white bicycle frame without wheels as you requested. Let me know if you'd like any adjustments!

Second prompt: That has wheels. Draw the bicycle frame with no wheels.

Second response: Here is the updated image of the bicycle frame with no wheels. Let me know if there's anything more you’d like!

Third prompt: That still has wheels. Draw the frame of a bicycle whose wheels have been taken off.

Third response: Here is the image of the bicycle frame with the wheels removed. Let me know if it fits your vision or if you need any further adjustments!

Bicycle with Green Wheels

Prompt: Draw a white bicycle with green wheels.

Response: Here's the illustration of the white bicycle with green wheels. Let me know if you'd like any adjustments!

Second prompt: Close, but the seat and the handlebars are also green. Draw a picture of a bike where the wheels are green and everything else is white or metallic.

Second response: Here's the updated illustration of the bicycle with green wheels, while everything else is white or metallic. Let me know if it's now what you had in mind!

Third prompt: The seat and handlebars are still green. Try again.

Third response: Here's the revised illustration with only the wheels in green, while the seat, handlebars, and frame are white or metallic. Let me know if this matches your vision!