Assignment 1 · 15% of the course grade

From Images to Film

A Generative AI Workflow

Make a 30–60-second video with sound. Use generative AI to create images, turn them into video clips, and create at least one audio element. Edit these into a finished piece that tells a story, expresses an idea, or creates a mood.

Due

Plan Generate images Animate Add sound Edit

Need a starting point? Explore workflow tools and creation websites ↓ · Workflow figures · Technical resources

Follow these five steps

  1. Plan your idea

    Choose a simple idea, such as a short story, fictional advertisement, or visual experiment. Describe who it is for and what you want viewers to feel. Sketch or describe 4–6 shots in order.

  2. Generate your images

    Create a starting image for each shot. Refine the images so that the style, characters, and setting fit together. Save your key prompts and selected images.

  3. Turn images into video

    Use image-to-video generation to animate your images. Decide what should move in each shot, including the subject or camera. Select the clips that best support your idea.

  4. Add sound

    Generate at least one audio element: music, sound effects, background sound, or narration. Match the sound to the action or mood. Dialogue is optional.

  5. Edit and export

    Arrange your clips, trim the timing, and balance the sound. Add titles or captions if helpful. Export a 30–60-second MP4 with audio and watch it once from beginning to end.

Save your progress: Keep two before-and-after examples as you work. These can show a change to an image, motion, sound, or edit. For each, explain the problem, your change, and the result.

Submit two files

  • 1. Final video (MP4)
    30–60 seconds, with audio.
  • 2. Process report (PDF, 6–10 slides)
    Include all the items below in this one report.
  • Idea and storyboard: Your concept, intended audience, and plan for 4–6 shots.
  • Workflow: A simple diagram showing how you went from images to video and audio. Include tool names, key images, and selected prompts.
  • Two improvements: Before-and-after examples with a short explanation of what changed and why. Use screenshots or still frames; describe sound changes in words.
  • Reflection (200–300 words): What decisions did you make? What was difficult? What did you learn?
  • Credits: State what AI generated, what you edited yourself, and where any external images, music, or other assets came from.
Grading rubric
What we look forWeight
Idea: A clear story, message, or mood.25%
Visuals: Consistent images, purposeful motion, and clear editing.25%
Sound: Audio supports the video and is balanced.15%
Process: Explain your workflow, improvements, and learning.25%
Documentation: Complete report and clear credits.10%

Suggested workflow tools and websites

Choose one workspace to organize your images, video, and sound. The options below link to actual creation tools, with a suggested route for this assignment. A node is simply one step in a visual workflow, such as generating an image or animating it.

Where to start: For Chinese-language tools, try RunningHub for ComfyUI workflows, TapNow for a creative canvas, Jimeng for a simple image-to-video route, or LiblibAI for image workflows. International options are listed below too. You only need one route.

Chinese-language platforms

RunningHubRun and adapt ComfyUI workflows online

Open RunningHub ↗ · rhTV canvas and templates ↗

  1. Browse ComfyUI workflows or AI applications. Start with an image-generation template and make your shot images.
  2. Choose an image-to-video workflow, upload one selected image, and describe the movement. Run a short test before repeating it for the other shots.
  3. Use a supported audio workflow or generate audio separately. Download the clips and sound, then assemble your final edit.

Search terms to try: 文生图, 图生视频, 分镜, 音效. Save a screenshot of the workflow and the settings you changed.

LiblibAI / 哩布哩布Find image models and reusable community workflows

Open LiblibAI ↗ · Image-editing workflow tutorial (中文) ↗

  1. Find an image-generation or editing workflow. Choose one that explicitly supports online running.
  2. Open “在ComfyUI中运行” where available, upload your reference image, and edit the prompt. Create a consistent set of shot images.
  3. Save the images and animate them in Jimeng, Vidu, RunningHub, or another accessible image-to-video tool. Add sound and finish the edit.

Some community workflows require local installation or additional models. Read the workflow description before choosing it.

Jimeng / 即梦Generate images, refine them on a canvas, and animate them

Open Jimeng ↗ · Official reference and audio-video examples (中文) ↗

  1. Use image generation to create your shot images. Use the smart canvas to refine composition or edit selected areas.
  2. Open video generation, upload a starting image, and describe the subject and camera movement. Use an ending frame if that mode is available and useful.
  3. Review each clip. Keep generated sound if suitable, or add generated audio separately. Assemble the shots in your video editor.

Keep the character description, visual style, and aspect ratio consistent across shots.

TapNowConnect text, images, video, and audio on one canvas

Open TapNow ↗ · Chinese user guide ↗ · Nodes and connections ↗

  1. Create a canvas or copy a template. Add your idea and references as nodes.
  2. Generate a shot image, then connect it as a reference for a video node. Repeat for your other shots.
  3. Generate audio using an available audio node. Download the selected outputs, assemble the final sequence, and save a screenshot of your canvas.

A connection passes a reference to the next step. Check the selected model’s supported inputs before running it.

More workflow platforms

Freepik SpacesConnect image, video, and audio steps in your browser

Open Spaces ↗ · Official getting-started guide ↗

  1. Create a Space or start from a template. Add your idea and reference images.
  2. Connect a prompt to an image-generation node. Review the image, then feed your chosen result into a video-generation node.
  3. Repeat for your planned shots. Add an audio step, such as voiceover, where available.
  4. Export your clips and audio, then assemble the final 30–60-second video in your editor. Save a screenshot of the connected steps.
LTX Studio / FlowsDevelop a storyboard, generate shots, and edit a sequence

Open LTX Studio ↗ · Build your first Flow ↗

  1. Start with your concept or script and develop a 4–6-shot storyboard.
  2. Generate and refine the shot images, then animate the selected images. Reuse character and setting references across shots.
  3. Arrange the clips in the timeline and work on pacing and sound. If a feature is unavailable on your plan, finish that step in your usual editor.
  4. For a visible workflow diagram, explore Flows: connect prompts, image generation, and video generation, and save a screenshot for your report.
ComfyUI / Comfy CloudBuild or adapt a reusable node workflow

Open Comfy Cloud ↗ · ComfyUI and workflow gallery ↗ · Beginner tutorial ↗

  1. Open Comfy Cloud, or install ComfyUI on a suitable computer. Start with an image-generation template.
  2. Generate and save your shot images. Load an image-to-video template and use your chosen image as its input.
  3. Run one shot, inspect the result, and adjust the prompt or settings. Generate sound with a supported workflow or add it separately.
  4. Save the workflow and selected outputs. Assemble the clips and sound in your video editor.

Choose templates supported by your setup. Local workflows need suitable hardware and model files; Cloud and hosted Partner Nodes may require credits. Partner Node requirements ↗

Figma Weave (formerly Weavy)Combine generation and visual editing on a node canvas

Explore Weave and workflow examples ↗ · Open the workspace ↗

  1. Start from a workflow example and add your reference image or prompt.
  2. Generate image variations and refine the selected image with editing steps such as cropping, masking, or relighting.
  3. Use an available video-generation step to animate it, or export the image to one of the video websites below.
  4. Repeat for your shots, then add generated audio and finish the sequence in a video editor. Capture the workflow for your report.
Other creation websitesUseful when you prefer a guided interface for individual steps
  • Lovart ↗ — Develop visual ideas, edit images, and generate video in a design workspace. Suggested route: brief → shot images → revisions → animation → final edit.
  • Hailuo AI / 海螺 ↗ — MiniMax’s video creation website. Upload your generated image, describe the motion, and review the clip. Explore its Video Agent workflow if available in your account.
  • Vidu ↗ — Use Image to Video or Reference to Video to animate your shot images. Download your selected clips and assemble them with sound.

For every route: Keep your generated images, save two before-and-after examples, and show the tools and steps in your report. Audio may be generated with the video or separately.

Check access before starting. Features, supported models, export options, and credits vary by account and region. These tools are optional; use an accessible alternative if needed. A more expensive platform does not earn higher marks.

See how the pieces fit together

Figure 1 · A complete workflowUse one platform or combine tools. Both audio routes below are valid.
01 · PLANIdea + shot listWhat happens in 4–6 shots?
02 · IMAGEGenerate + refineSave one image per shot
03 · MOTIONImage → videoAdd action + camera direction
Route A · Sound with video

Use the generated clip and its audio.

Route B · Separate sound

Generate music, effects, or narration and align it in the edit.

EDIT → EXPORTArrange shots · trim timing · balance sound · export a 30–60-second MP4
Figure 2 · One idea, four connected shotsIllustrative storyboard: “A paper boat finds its way home.” These are planning sketches, not generated outputs.
Establish · 0–10 s

A boat rests in a rainy alley.

Sound: Rain + distant street ambience
Follow · 10–20 s

It drifts forward; the camera follows.

Sound: Water + light rain
Discover · 20–30 s

A warm doorway appears ahead.

Sound: Water + a soft musical cue
Arrive · 30–40 s

The boat settles beside the doorstep.

Sound: Rain fades; music resolves

This example totals 40 seconds. Keep the same boat, palette, and lighting; change the framing and action. You may use any subject or style.

Technical resources · use what you need

These are optional references. Start with the guide for your chosen tool; you do not need to read everything.

Quick tips for your first experimentSmall changes make results easier to understand
  • Image prompt: Describe the subject, setting, composition, lighting, and style.
  • Motion prompt: Describe what moves and how the camera moves. Start with one clear action.
  • Consistency: Reuse references and keep the aspect ratio the same across shots.
  • Comparison: Change one thing at a time. Keep the model and available settings fixed; if seeds cannot be fixed, note that randomness can affect the result.
  • Sound: Listen to the full edit. Keep narration clear and avoid abrupt changes in volume.

Keep it simple. Use any accessible tools. Coding and paid subscriptions are not required. We value thoughtful choices and what you learn; expensive tools or high resolution do not automatically earn higher marks.