Can AI Tell a Story?

Understanding What Text-to-Image AI Actually Does With Your Words

Time to Complete: 30 minutes

PDF 5-Minute Warm-Up Activity can be downloaded above. 

Who This Is For: This lesson is for anyone whose work involves creating, commissioning or evaluating visual content and who has experimented with AI image tools without fully understanding what drives their outputs or their failures. That includes graphic designers and art directors in publishing, advertising and digital media who are being asked to integrate tools like DALL-E-2, Midjourney or Stable Diffusion into production workflows; educators and instructional designers who want to use AI-generated visuals in learning materials but are unsure how to assess quality beyond surface appeal; marketing managers and brand strategists in creative agencies who commission AI-generated content and need a framework to evaluate whether outputs actually serve narrative and communication goals; and writers, authors and game developers who want AI to illustrate their stories but keep running into inconsistencies across image sequences. The shared frustration across all these roles is the same: AI tools produce impressive individual images but fall apart when asked to maintain a coherent visual story, and most users do not know why or how to fix it. This lesson explains the mechanics behind that failure and gives you practical tools to work around it. 

Real-World Applications

Children's book publishers and educational content studios are already deploying text-to-image models to reduce illustration costs and accelerate production timelines. The core challenge they face maps directly to what this lesson covers: DALL-E-2 and Midjourney can generate a single striking scene, but maintaining a protagonist's face, clothing and emotional arc across twelve sequential pages requires prompt engineering skill that goes far beyond typing a description. Understanding the mechanics of semantic alignment, attention mechanisms and evaluation metrics is no longer optional for creative professionals in these industries. It is the difference between using AI as a genuine production tool and producing a set of beautiful but disconnected images that an art director has to rebuild from scratch. 

Lesson Goal

You will develop practical skills in AI-driven visual storytelling by learning how text-to-image models translate written descriptions into images. By the end of this lesson you will be able to construct effective prompts for sequential visual outputs, identify the technical reasons why AI tools struggle with narrative coherence and design evaluation criteria that measure storytelling success rather than technical image quality alone. The lesson draws directly on research comparing DALL-E-2, Midjourney, Stable Diffusion and Craiyon across realism, semantic alignment and contextual consistency benchmarks. 

The Problem and Its Relevance

Text-to-image AI tools are frequently described as creative partners, but they are better understood as very sophisticated single-frame generators. They do not read your story. They respond to the words you give them right now, with no memory of the image they produced a moment ago, which means every scene in a narrative sequence is generated from scratch regardless of what came before. This limitation is not a bug waiting to be patched. It reflects how these systems are architecturally designed: models like DALL-E-2 use CLIP-based alignment to match a single text input to a visual output, and that alignment resets with every new prompt. Asking such a system to maintain a character across twelve scenes is like asking someone with no short-term memory to illustrate a novel, one chapter at a time, after erasing everything they saw before. The evaluation metrics used to benchmark these models make the problem worse, not better. Inception Score and Frechet Inception Distance measure statistical image quality relative to a reference dataset. They say nothing about whether the image serves the story, advances a character arc or maintains the visual logic of a scene that already happened. A technically perfect image that contradicts the narrative is indistinguishable from a good one under these metrics. Prompt engineering is widely promoted as the solution to all AI image failures, but this framing shifts the burden of the system's structural limitations onto the user. When a model fails to keep a character consistent across scenes, that is not a prompting error. It is an architectural constraint, and treating it as a user skill problem prevents the field from developing the tools that would actually address it. 

Why Does This Matter?

Understanding how text-to-image AI works and fails in storytelling contexts matters because:

1.    Prompt quality determines output quality: The specificity and structure of a written description directly shape whether an AI generates an image aligned with the intended scene or produces something plausible but narratively wrong.

2.    Single-image success does not equal narrative coherence: Models that produce visually striking individual images frequently lose character appearance, setting logic and emotional continuity when generating a sequence, because they have no memory of prior outputs.

3.    Standard metrics measure the wrong things: Inception Score and Frechet Inception Distance assess technical image quality but cannot evaluate whether a visual actually supports narrative meaning, emotional tone or thematic development.

4.    Artistic consistency remains unsolved: Even DALL-E-2 struggles to apply compositional principles like the rule of thirds consistently, and no current model reliably maintains a single art style across a sequence of outputs without manual intervention.

5.    Creative control requires technical understanding: Knowing how attention mechanisms and diffusion processes function allows a practitioner to diagnose failures and refine prompts strategically rather than regenerating images repeatedly and hoping for better results.

6.    Ethical stakes extend beyond copyright: AI image tools raise questions about the displacement of human illustrators, the homogenization of visual culture and the reduction of interpretive space that traditional illustration has always left open for readers.

7.    Commercial goals shape tool design: The models most widely used in creative industries were built for general visual appeal, not narrative fidelity, and understanding that distinction helps practitioners select tools that match actual production needs. 

Three Critical Questions to Ask Yourself

•      Do I understand the difference between generating a visually appealing image and creating a visual that advances a specific narrative moment?

•      Can I explain why maintaining character consistency across an AI-generated image sequence is structurally difficult, not just a matter of writing better prompts?

•      Am I able to design an evaluation framework for AI-generated visuals that measures storytelling success rather than technical image quality alone? 

Activity Steps

Review the research comparing DALL-E-2, Midjourney, Stable Diffusion and Craiyon on realism, semantic alignment and contextual consistency benchmarks. Pay attention to where each model succeeds on individual image tasks and where each breaks down when applied to sequential visual outputs. Keep this comparative picture in mind as you move through the steps below.

Working in groups, your task is to: 

(i) Select a short narrative requiring four to six sequential images to tell a complete story. Options include a folktale, a historical moment, an educational process or an original fiction fragment with a clear beginning, middle and end.

Guidance: Choose narratives with a recurring character and at least two distinct settings to make coherence challenges visible rather than easy to avoid. 

(ii) Write a detailed text prompt for each image in the sequence. For each prompt specify

◦       The core visual elements that must appear, including characters, objects and settings

◦       Compositional guidance such as perspective, framing and focal point

◦       Stylistic direction covering artistic medium, color palette and mood

◦       An explicit link to the previous image that names what must carry over, such as the character's appearance or the time of day 

(iii) Assess your prompts against three criteria

◦       Semantic alignment: How precisely does each prompt communicate the narrative moment it is meant to depict?

◦       Sequential coherence: What specific language in each prompt works to maintain visual continuity with the image before it?

◦       Creative intent: Where does each prompt deliberately leave room for AI interpretation and where does it demand a specific outcome? 

(iv) Anticipate failure modes specific to your narrative. For each of the following risks, write a concrete example of how it might appear in your sequence and propose a prompt revision that would reduce it

◦       Character appearance shifts between images

◦       Setting or spatial relationships become inconsistent

◦       Emotional tone drifts away from the narrative's intended register

◦       Art style changes noticeably from one image to the next 

(v) Build an evaluation framework that measures success beyond technical image quality. Define at least four criteria and create a simple scoring rubric for each. Suggested criteria include

◦       Character consistency across the full sequence

◦       Clarity of narrative progression from image to image

◦       Emotional alignment with the intended tone at each story moment

◦       Fidelity between the written prompt and the visual output

◦       Stylistic coherence across all images in the sequence 

(vi) Compare your prompt-only approach with two alternatives. Alternative A uses reference images alongside text prompts. Alternative B uses iterative refinement, where you generate initial images and then write revised prompts based on what the AI produced. For each approach, evaluate narrative control, production efficiency and creative flexibility as they apply to your specific scenario.

Guidance: Accept that perfect sequence coherence is unlikely with current tools. Identify which narrative elements are non-negotiable for your story and which can tolerate variation without breaking the reader's understanding. 

Individual Reflection

After completing the activity, consider and share with your group

•      How this exercise changed your understanding of what makes a prompt effective for narrative purposes versus aesthetic purposes

•      Whether you now see visual storytelling differently, knowing that AI tools generate each frame without memory of what they produced before

•      What the gap between single-image quality and sequence coherence reveals about the current state of generative AI

•      How you would approach working with AI image tools in a real creative project, given what you now know about their structural constraints

•      Whether understanding these limitations changes how you evaluate AI-generated visuals in media you encounter outside this lesson 

The Bottom Line

Effective use of text-to-image AI in storytelling requires understanding that these tools are not visual narrators. They are pattern-matching systems that respond to individual prompts in isolation, and that distinction matters enormously when the goal is a sequence of images that tells a story rather than a single image that looks impressive on its own. The creative industries promoting AI as a storytelling tool are, in many cases, describing a capability that does not yet exist at the architectural level. Selling prompt engineering courses as the solution to sequence incoherence is the equivalent of selling better handwriting lessons to fix a broken printing press. The four model families reviewed in the research, GANs, diffusion models, transformers and hybrid approaches, each offer different strengths in image quality, stylistic range and prompt responsiveness. None of them has solved narrative coherence across sequences without sustained human intervention. Knowing which model to use for which task, and where each will require manual correction, is the practical AI literacy this field needs most. When you can articulate why a specific narrative moment demands a particular visual treatment, what trade-offs exist between automation and creative control, and which evaluation criteria matter most for your storytelling goal, you have developed the judgment needed to use these tools as genuine production assets rather than expensive sources of unpredictable outputs. That judgment applies whether you are producing children's books, designing educational content, building games or evaluating AI-generated imagery in the visual culture around you. 

#TextToImageAI  #AIVisualStorytelling  #PromptEngineering  #GenerativeAIDesign  #AIImageGeneration