April 22, 2023
Welcome to the machine.
“Knock - Knock”
“Who’s there?”
“Me, a human”
“Welcome human. What’s your prompt? Think carefully - the few words you say will determine your fate”.
I stumbled the other day on promptperfect, a SaaS website that offers to optimize one’s prompt, taking into account the specifics of the various models that could be used. This works, allegedly, for chat-like LLMs, as well as for prompt to image applications. Prompt engineering has been a thing for a while, but I have never really dived into it - yet. Intrigued, I decided to give it a try - on the image side - to create a couple of pieced for my Insta profile, keeping up with an exercise initiated last year for #CreativeAI_Sydney, emulating famous rock album covers.
I settled on trying to do a version/tribute of “Welcome to the machine” logo - an artwork designed by Aubrey Powell and associated officially with the album “Wish you were here” although fans have associated it with this specific song.
I wrote my initial prompt as:
"A circle in front of a black background. The circle is divided in four equal parts. In the bottom left, a desert with a thin white sky. In the bottom right, the sea with a thin white sky. On the top right, an indigo sunset. On the top left, a pink sunrise with the sun in the corner. In front of the circle there is an orthographic perspective view of a robotic handshake. Digital art."
Orthographic is probably not the best choice of word - my bad. Isometric or axonometric would have been more accurate/specific here.
The tool is easy to use, and give you the ability to optimize your initial prompt for different models. It also shows a preview of “what to expect” - i.e. it does not actually run the models.
That’s what it came up with from my prompt:
Well, if anything, the previews are not really convincing.
So I went in the actual models, and, rather than trying only the prompts for the models they were intended to, I tried out all the combos.
I’ll let you be the judges of the results obtained.
Like with previous experiments, the pieces produced are often sexy and stunning indeed. Somehow, I cannot see any evidence that the prompts “fine tuned” for a given model worked better in this very model and in the others in terms of aesthetics. The most jaw dropping for me (2 of them figured below) were more often than not generated from a mismatch between the model used and the model for which the prompt was ‘optimized’.
Prompt optimized for Dalle-2, executed by Kadinsky
Prompt optimized for Dalle-2, executed by Kadinsky
Something that is front and center in all discussions about AI these days, is the fact that all these prompts and all these models failed to actually address my expectations, i.e. actually ‘understanding’ my prompt, and produce something ticking my 2 main check-boxes at the same time:
a/ A circle divided in four.
b/ A robotic handshake in front of it.
Some got one of these points pretty well (images below), principally with my original prompt, but none got them both. Someone is up for a challenge?
Original prompt, executed by Midjourney
Prompt optimized for Kadinsky, run by Midjourney
Prompt optimized for MidJourney, run by Dalle-2
Original prompt, run by Stable Diffusion
Original prompt, run by Stable Diffusion
Conclusion of this exercise? The “prompt optimizing service” sounds a lot like digital snake oil, at least when applied to image generation. Will try for text!
All images here.
Texts of the 4 optimized prompts:
A futuristic digital artwork featuring a circular composition with four distinct landscapes: a serene desert, an expansive ocean, a vibrant indigo sunset, and a peaceful pink sunrise. In the center of the circle, there is an orthographic perspective view of two robotic hands shaking in agreement. The image is set against a black background for maximum contrast and impact.
Orthographic view of a robotic handshake in front of a circle divided into four parts, each depicting a different landscape: desert with thin white sky, sea with thin white sky, indigo sunset, and pink sunrise with sun in corner. Circle is highly detailed and realistic, featuring intricate textures and shading. Handshake is rendered in hyperrealistic style with metallic sheen and precise details. Digital art inspired by Alex Ross and Simon Stålenhag.
A dynamic and visually stunning digital artwork featuring a futuristic robotic handshake in an orthographic perspective view. The background showcases a circular split design with four equal parts, including a desert landscape with a thin white sky, the sea with a thin white sky, an indigo sunset, and a pink sunrise with the sun in the corner. The overall style is inspired by cyberpunk and science fiction art trends on DeviantArt, using vibrant neon colors and intricate details to create a breathtaking image. Rendered in Maya or Blender with advanced lighting techniques for maximum realism and impact –upbeta
A stunning digital artwork featuring an orthographic perspective view of a futuristic robotic handshake in front of a mesmerizing circular backdrop. The circle is divided into four equal parts, showcasing a breathtaking indigo sunset on the top right and a stunning pink sunrise with the sun in the corner on the top left. On the bottom left, there's a serene desert landscape with thin white sky while on the bottom right, there's a vast sea with thin white sky. The artwork is rendered using advanced techniques inspired by trending digital artists like Beeple and Ash Thorp, incorporating vibrant colors, intricate details, and photorealistic textures to create an immersive and engaging visual experience -upbeta