Choose the system that produces an asset you can publish with documented rights, repeat across the scenes you need, and finish within a realistic correction budget. One perfect render says little about the next attempt. These tests compare observed results and workflow constraints, without declaring universal winners.
Can it do the job again?
These findings come from Maker Signal’s local and hosted test record dated September 20–21, 2026. The same coffee-cup scenario probes instruction following, readable text, exact object count, subject consistency, camera motion and physical contact. Repeat runs exposed differences that a single successful render would hide.
- Text success was specific. Ideogram placed a legible standalone OPEN sign in 4/4 images, then 8/8; local FLUX.1 [schnell] did so in 5/5. Neither met the three-book instruction in those batches. These sample sizes do not establish an overall winner.
- Consistency needs its own test. Recraft’s reference feature carried recognizable subject characteristics into one new pose. A locked SDXL seed did not. Similar faces within a batch are a different finding from identity preserved across new scenes.
- Quality and predictability diverged. Kling’s action and camera result repeated while its wait fell from about 19–20 minutes to under three. Luma’s two repeat clips both measured 720p and 5.041 seconds, but cost 178 and 559 credits.
Method and limits
Local tests used Windows, an NVIDIA GeForce RTX 5060 Laptop GPU with 8 GB VRAM and 32 GB system RAM, with ComfyUI and models already installed. Template defaults were retained unless a variation is explicitly named below. Hosted tests used the tester’s own logged-in accounts, with different plans, native batch sizes and output settings. Flow and Luma used conversational agents that can interpret prompts; the other described submissions used prompt or tool interfaces. This is a practical workflow comparison, not a controlled benchmark of model architecture.
Measured: local execution-log times, recorded credit-counter changes and the media-property readings identified below. Observed: visual judgments, interface labels, queue messages and approximate hosted waits. Vendor claims: current plan and license descriptions, linked separately. Unresolved: unexplained charges, account changes and conflicting records. The publication review used the supplied written record; it did not rerun generations or independently remeasure the original media files.
Counts describe only these outputs, not population success rates. Hosted batches are not independent sessions. No blind panel or statistical significance test was used. A warm local generation time excludes installation and model download; an approximate hosted wait can include queueing. Different sizes, steps and conditioning prevent direct speed rankings. A zero-generation attempt receives no quality score.
Image comparison: check the whole frame
“Sign” means a standalone, legible OPEN sign, rather than text redirected onto a cup or clothing. A sign pass does not certify every other word, the number of signs, identity, or book count. Counts below remain separate even when one image passes several checks.
| Tested setup | Sample | Sign | Exactly 3 books | Recorded time / credits | What limits the result |
|---|---|---|---|---|---|
| SDXL / Juggernaut-XL | 5 images | 1/5 | 1/5 | 17.3–17.6s/image, warm | Five different-looking subjects; legible OPEN somewhere in 5/5, often misplaced. |
| SDXL, tuned negative prompt | 3 images | 1/3 | 0/3 | Not separately reported | One cleaner sign; extra or garbled text persisted. |
| SDXL Turbo | 5 images | 0/5 | 0/5 | 0.59–0.65s warm; 5.4s first | 512×512, softer detail; no legible OPEN anywhere. |
| FLUX.1 [schnell] FP8 | 5 images | 5/5 | 0/5 | 14.1–14.2s warm; 19.98s first | Different subjects; garbled book titles in all five; two had a second sign. |
| Firefly Image 3 / Free | 4 + 4 images | 0/4 → 0/4 | 0/4 → 0/4 | Under 10s/batch → about 8s | Legible OPEN anywhere: 1/4 → 0/4. Similar-looking subjects: 0/4 → 3/4. |
| Ideogram / Auto / Free | 4 + 8 images | 4/4 → 8/8 | 0/4 → 0/8 | Under 10s → well over 100s | 8 credits → 4 credits total; second request asked for four images but returned eight. |
| OpenArt / Flux 1 Dev / Free | 1 image | 1/1 | 0/1 | Under 8s; 5 credits | Model substituted to fit eight-credit balance; watermark; garbled secondary text. |
| OpenArt / Nano Banana Pro / Starter | 1 image | 1/1 | 1/1 | Under 20s; 40 credits | Clean sign and count together; no visible watermark. Model and plan both changed. |
| Recraft / V4.1 Pro via Auto / Free | 2 images | 1 clean; 1 cropped | 1/2 | Under 20s; 14 credits/batch | Both signs off the cup; correct-count image had text bleeding onto books. |
Video comparison: action, time and actual duration
Every text-to-video test requested five seconds. The image-to-video variation used a different lift-and-sip prompt. Values such as 5.153s and 5.041s are media readings, not literally exact five-second files. Selected resolution is a setting; measured resolution is identified explicitly in the details.
| Tested setup | Clips | Generation time | Output / setting | Result and constraint |
|---|---|---|---|---|
| LTX-Video 2B distilled / text-to-video | 2 | 58.8s / 64.4s | About 4s; 768×512 | Plausible grip; action reversed and camera static in both. Face not evaluable. |
| Wan2.2 5B / text-to-video | 1 | 27m 28s | About 5s; 1280×704 | Static camera; apparent reversed action; black jacket instead of red. |
| LTX-Video 2B distilled / image-to-video | 1 | 45.8s | About 4s; 512×512 | Lift-and-sip and push-in worked; source text defects carried into video. |
| Google Flow / Veo 3.1 - Fast / paid account | 2 | First not recorded; repeat about 109s | Reported 10s → timeline-confirmed 8s | Grip and push-in worked; five-second request not delivered. 15 → 20 credits. |
| Kling AI / Video 3.0 / apparently non-paid account | 2 | About 19–20m → under 3m | 5.153s both; 1080p selected | Pick-up and push-in repeated; 60 credits each. Audio presence checked, quality not assessed. |
| Luma / Ray3.2 / fresh Account 2, chat path | 3 | Under 2m / under 2m / about 4m | 5.041s each; 720p | Grip and push-in worked; silent, watermarked. 158 / 178 / 559 credits. |
Audio and watermark status were not established for every platform. Do not infer a clean export or commercial permission from an unreported watermark. Face consistency is unassessed where the face leaves the frame; sampled-frame inspection cannot guarantee every frame is artifact-free.
Detailed image findings
Local SDXL: a cached repeat is not a new character test
The five Juggernaut-XL baseline images were structurally clean. OPEN was legible somewhere in all five, but only one placed a legible sign behind the subject; in the others text migrated to clothing or props, with garbling or duplication in three. Exactly three books appeared in one image. All five subjects looked different. The baseline used 1024×1024 output and 25+20 base/refiner steps, with the default negative prompt “text, watermark.”
In the two-part seed-lock check, unchanged inputs returned a pixel-identical image from ComfyUI’s execution cache. That is evidence of cached reproducibility, not an independent recomputation or a guarantee across machines. Keeping the seed while changing the scene to sitting at a desk typing produced a different-looking woman. Three further runs with anti-garbling negative terms produced one cleaner sign, no correct book counts, and persistent text clutter. The small test does not establish a cause for those errors or a universal limit of SDXL.
SDXL Turbo and FLUX.1 [schnell]: speed and text are separate choices
Turbo used sd_xl_turbo_1.0_fp16, 512×512, one step, CFG 1 and euler_ancestral. Four warm runs took 590–650 milliseconds; the first took 5.4 seconds including loading. All five were coherent but softer than the larger baseline, with no standalone sign and no fully legible OPEN. The speed comparison includes a smaller output and different sampling, not just a model swap.
FLUX used the stock Flux.1 Schnell FP8 template, flux1-schnell-fp8.safetensors, 1024×1024, four steps, CFG 1, euler/simple and random seeds. Warm times were 14.1–14.2 seconds; the first was 19.98 seconds including loading. All five signs read correctly and no cup carried garbled text, but every image had illegible book titles, every book count failed, and subjects differed. Two images contained a second legible OPEN sign. The tested local FLUX checkpoint was [schnell]; Flux 1 Dev was used in the separate hosted OpenArt test.
Reference consistency: Canny structure versus Recraft identity
One SDXL Canny ControlNet run used controlnet_canny_sdxl_small_fp16 at strength 1.00 over the full denoising range, retaining the baseline prompt and scene. It held pose, framing and layout, but changed the face and retained wrong book counts and a duplicated cup; text bleed spread. No IPAdapter or comparable identity tool was installed. This was not a new-pose test. A separate review of 14 existing same-scenario local images found no fully clean source; that review pool is not an additional 14 independent generations to add to the scorecard.
Recraft’s initial Auto batch routed to V4.1 Pro according to each image’s metadata: two 2048×2048 images, 14 credits total, under 20 seconds. Both put the sign off the cup; one sign was cropped. The other image had three books but let OPEN bleed onto two, partly garbled. The subjects shared a general look but were visibly different.
On September 21, the tester used “Add as reference” on the first Recraft image, then requested the same woman sitting at a desk typing in an office. One image, under 30 seconds, cost 19 credits (84 → 65). Hairstyle, earrings, skin tone, facial structure and jacket carried over convincingly, and the pose was followed. This is one reference success, not a reliable identity rate; the original sign/book test did not apply to the new scene. An unrelated animation request already in the project history was rejected for requiring a paid subscription. Its origin and date remain unknown, so it is not counted as a new controlled video attempt.
Firefly: steady speed, unstable text and apparent identity
The free account used Firefly Image 3, Fast mode, square output and default Auto content type, not the offered Image 5 preview. Each click returned four images. The first batch completed in under ten seconds and the repeat in about eight. All eight were visually clean at the gross-artifact level.
The first batch had one legible OPEN on a mug, but no image combined clean sign placement and legibility. The repeat produced two separate sign-like objects, but one was obscured and the other had a malformed final letter: no fully legible OPEN anywhere. Both batches missed every book count. Four distinct-looking subjects became three similar-looking women in the repeat batch with the same settings. That unexplained swing does not prove a durable character-consistency feature. Credits used were not quantified in this record.
Ideogram: repeated sign success, changing wait and credit mechanics
The Personal/Free account used Auto, 1:1, four images requested and no style, character or color presets. The first batch delivered four images in under ten seconds despite a slow-queue message; the balance fell 12 → 4, or eight credits total. The identical repeat delivered eight images rather than four, took well over 100 seconds without precise stopwatch timing, and consumed the remaining four credits. That is two credits per image followed by 0.5 credits per image, not a stable rate to budget against.
All 12 outputs had readable standalone signs and blank cups, and subjects appeared consistent within each batch. All 12 missed the three-book instruction. A new-pose reference test was not performed. Neither the changed batch size nor the changed cost has a confirmed explanation; Auto’s underlying model routing was not established.
OpenArt: two useful single images, not a controlled repeat
On Free, the default Nano Banana Pro quote was 40 credits against an eight-credit balance. It was not submitted at that price. The tester substituted Flux 1 Dev at five credits, retaining 4:3 and 1K settings. One clean image arrived in under eight seconds with a legible standalone sign, roughly seven books and garbled secondary text. A large OpenArt watermark and the interface’s export-watermark disclosure limited the free result. Three credits remained.
The same account later displayed Starter and thousands of credits for an unknown reason. Nano Banana Pro then produced one image at the unchanged 40-credit quote, confirmed deducted, in under 20 seconds. The sign was clean; three books were countable, with two legible titles and a third partly visible. No watermark was visible. This is an important exception to the earlier book-count failures, but model, plan and balance all changed. Neither run tested repeatability or subject consistency, and the account change remains unexplained.
Detailed video findings
Local video: valid footage can still miss the action
Two LTXV-2B distilled text-to-video runs used 30 steps and 768×512. Each produced about four seconds from the template’s 97 frames at 24fps, despite five seconds in the prompt. Grip looked plausible and the scene was coherent, but both clips set down an already-held cup and kept the camera static. Framing prevented a face-identity assessment.
Wan2.2 5B produced one roughly five-second 1280×704 clip at 20 steps. It also appeared to reverse the action and ignored the push-in, while rendering a black leather jacket rather than red. It completed successfully in 27 minutes 28 seconds. System RAM fell below 500 MB free out of 32 GB during an extended decode/encode stage. Memory pressure was observed; precise VRAM use and the causal contribution of each stage were not instrumented. This is a hardware-fit warning for this setup, not a claim about all Wan or local video workflows.
The LTX image-to-video variation used a flawed SDXL source resized from 1024×1024 to 512×512, 30 steps and an LTXVImgToVideo node. In 45.8 seconds it produced about four seconds of a plausible lift-and-sip with a push-in and recognizable source composition. Garbled source text persisted; the final close-up obscured part of the face. Different prompt, conditioning and size mean it cannot isolate which change produced the better action result.
Google Flow: repeated grip and push-in, inconsistent duration and cost
Two paid-account Agent requests used the exact video prompt plus an instruction not to embellish it, with Veo 3.1 - Fast manually selected. The first cost 15 credits and was recorded as a ten-second clip; its generation time and measured pixel dimensions were not recorded. Visual review found a convincing grip, push-in and stable appearance in the checked frames.
The repeat explicitly refused five seconds, claiming the selected model supported only eight seconds on that account’s tier. The eight-second alternative was accepted, delivered and confirmed in the timeline. It cost 20 credits and took about 109 seconds including a high-demand queue message. Grip and push-in repeated. These are observed Agent behaviors, not proof of a universal Veo duration limit. Current documentation lists 4/6/8-second Fast generations, so the tier explanation and first ten-second result remain unverified. No cause is assigned to the 15 → 20 credit difference.
Kling: similar footage, very different waits
The account had no explicit Free label, but showed upgrade prompts and no active-subscription indicator. Video 3.0 defaults were 1080p, five seconds, 16:9, one clip and Native Audio on. The first run spent 60 of 66 credits. Its interface advertised a one-minute estimate and a Fast-Track message claiming two hours saved; actual observed completion took about 19–20 minutes. The unexperienced two-hour queue is a vendor claim, not measured waiting time.
Before the repeat, the balance rose from six to 336 credits without a recorded purchase or plan change. The second run also cost 60 credits, had no fast-track or wait estimate, and completed in under three minutes. Both media readings were 5.153 seconds. Checked frames showed the cup lifted from the table and a continuous push-in; the face was cropped out after about 2.6 seconds. Nonzero decoded audio bytes supported audio-track presence in both, but listening quality and synchronization were not evaluated. The balance increase and queue differences remain unresolved.
Luma: three successful chat clips, unresolved charges and resolution claims
The fresh Account 2 trial succeeded through chat after the direct Create Video node stayed disabled. Its first clip measured 5.041 seconds, used a reported 720p setting, took under two minutes and cost 158 credits. The usage/balance record showed 2,842 trial credits remaining, consistent with an approximately 3,000-credit starting allowance. Checked frames showed lift, grip and push-in without visible warping. The face was not sufficiently visible for identity evaluation. The record reports silence and a visible Luma AI watermark.
The first repeat used the tester’s paraphrase through chat: under two minutes, 178 credits, measured 1280×720 and 5.041 seconds, silent and watermarked, with convincing grip and push-in. The direct node now explicitly displayed “Requires Keyframe(s)”; Create Scene offered a ten-second minimum. The original entry already recorded a disabled direct node, so this evidence establishes a later explicit explanation, not a proven regression from a previously working direct text-to-video path.
Before repeat two, the tester reported personally upgrading to ensure enough credits. The interface continued to display Luma Trial; the reason for that label is unknown. The exact test wording through chat yielded another 5.041-second, 1280×720, silent, watermarked clip with the same checked action and camera behavior. It took about four minutes and cost 559 credits, about 3.1 times the preceding 178-credit run. The agent claimed 1920×1080/1080p, contradicting the delivered videoWidth/videoHeight measurements. The prompt and account funding conditions were not identical across repeats. Neither the charge difference nor the incorrect resolution report has a confirmed cause.
The two-account plan conflict below remains unresolved. These repeats used Account 2 again, not a third independent account.
Access results: zero generations is not a quality verdict
| Platform / account | Attempt and barrier | What this establishes |
|---|---|---|
| Runway / free Personal, 100 credits | Tool → Video; Gen-4.5, HDR, 16:9, 5s, no first frame. Every surfaced model showed Pro required. Upgrade opened a $35/month or $28/month billed yearly paywall; Image / Nano Banana 2 also gated. | Zero generations on this account and interface. Public free-access claims differ; do not generalize to every account. |
| Luma / previously used account, about nine months old | Ray3, Draft on, 16:9, no keyframe. Submission returned “You have reached your free usage limit.” | Zero on this account at that time; apparent account quota, not proof all fresh signups are blocked. Account 2 later succeeded. |
| Pika / pre-existing Free account | Pika 2.5 selected; 5s, 720p, audio off, Auto aspect. Twenty-credit quote against zero; “Not enough credits.” Usage log showed no credit activity. | No generation and no purchase. A fresh signup was not tested. Current public Free plan lists zero monthly credits. |
| Black Forest Labs / developer dashboard, 0.00 credits | Preselected flux-3 exposed video/audio controls. Add credits required a $10 minimum for 1,000 credits; tester declined purchase. | No image or video output to assess. A paid developer console is different from a consumer free allowance. |
| Midjourney / previously used, currently unsubscribed | Web prompt field disabled with “Subscribe to start creating…”; subscription paywall. Source also checked official Discord subscription requirement. | No generated images; no sign, book-count or quality assessment. No paid test was performed. |
Pika’s tested Video Studio initially selected Seedance 2.5 at roughly 243 credits for 10s/720p/audio on. Switching to Pika 2.5 changed the quote to about 20 credits for 5s/audio off. That is a different model and configuration, not a like-for-like 12× model-price comparison. The model behind a platform’s default matters.
Current vendor checks and unresolved terms
Public first-party pages were checked September 21–22, 2026. Dollar figures below are USD as displayed; monthly billing and annual commitments are distinguished. Historical account quotes are retained as observations, not offers to readers. No credit balance was converted into a cash cost without a verified purchase basis. Vendor documentation does not override a conflicting account record or establish output quality.
- Runway: public pricing advertises 125 one-time Free credits and selected generative models. Standard is $15 monthly or $12/month billed annually; Pro $35 or $28; Max $95 or $76. This contradicts treating the tested Pro gate as universal. The reason that account could not use its visible 100 credits remains unknown. Current plans.
- Pika: public Free now explicitly lists zero monthly credits and no commercial license. Starter is $10 monthly/$8 annually with 900 credits and also lacks a commercial license; Creator $35/$28 with 3,150 credits includes one; Fancy starts at $95/$76 with 8,550 credits. “No watermark” is listed even for Free and is not equivalent to commercial rights. Plan comparison.
- Google Flow: the current Fast rate is 20 credits per generation for non-Ultra subscribers and 10 for Ultra, with 4/6/8-second options. This supports the repeat’s 20-credit charge, not the first run’s 15-credit/ten-second record or the Agent’s eight-second-only explanation. Current credit table.
- Ideogram: Plus shows $20 monthly or $15/month billed annually; Pro $60 or $42. The page lists weekly Free slow credits for eligible accounts. These published terms do not explain eight images for four credits in the repeat. Current plans.
- Recraft: current public terms confirm Free images are Recraft-owned, public and not licensed for commercial use; paid-plan generations receive ownership and commercial rights that survive cancellation. The source account showed Basic $10 and Pro $16–$128 monthly, but numeric plan cards were not available in the retrieved public page, so those prices were not independently reconfirmed. Ownership and plans.
- OpenArt: the current page lists Starter with 4,000 monthly credits and watermark-free output. Its multiple discount displays were not resolved into a single verified billing quote. Neither this page nor a clean export explains the test account’s spontaneous Starter badge. Commercial permission for that specific tested Starter output was not independently established. Current plan details.
- Midjourney: current monthly Basic/Standard/Pro/Mega prices are $10/$30/$60/$120, with 20% off an annual commitment. General commercial terms have conditions: companies above $1 million annual gross revenue need Pro or Mega. No new output test was run. Plan and usage terms.
- Adobe Firefly: Adobe documents limited free generative credits and commercial use of Firefly output, subject to product-specific restrictions, including any stated beta exception. This is vendor guidance, not a guarantee against infringement or a claim about third-party models inside Firefly. This review did not establish a fresh paid-plan dollar quote. Credits and use guidance.
- Kling: the official Video 3.0 guide was located but could not be retrieved for a reliable current price/rights check. The 60-credit charges remain test observations, and current plan pricing, watermark and commercial-use conditions are not independently verified here. Official model guide.
- Black Forest Labs: the current site lists FLUX.2 variants and FLUX 3, and pay-as-you-go pricing. The source’s preselected FLUX 3 screen exposed video/audio controls; do not mistake that tested mode for the local FLUX.1 [schnell] image checkpoint. The current calculator’s displayed video rate differs from the source’s quote, without matched configuration. No current per-model price or universal “image-only/video-only” claim is inferred. Pricing · FLUX 3.
Luma pricing: two real accounts, no resolved quote
The older account showed Free/Lite/Plus/Unlimited at $0/$9.99/$29.99/$94.99 monthly. Lite showed 3,200 credits, watermark and non-commercial restrictions; Plus showed 10,000 credits, commercial use and no watermark; Unlimited added relaxed generation. Account 2 showed Plus/Pro/Ultra at $30/$90/$300 monthly, respectively 10,000/40,000/150,000 credits, or $25/$75/$250 per month with annual commitments. Account 2’s balance was labeled Luma Trial.
Both structures were recorded from live account billing screens on the same day. Public first-party search results also surface the Plus/Pro/Ultra structure, but the relevant full pricing pages could not be reliably retrieved in this review. That does not resolve why the accounts differed, what a new reader will see, or the repeat-generation charges. No workspace, trial or migration explanation is asserted. Check the actual account checkout and export rights before paying. Luma pricing.
Hosted app, API or local?
Hosted apps remove installation work and can provide useful reference controls, but the tests show that account eligibility, model selection, watermarking, queueing and billing are part of the product. Ideogram and local FLUX are candidates to retest for short sign text; Recraft’s reference workflow merits a larger new-pose test; OpenArt’s Nano Banana Pro result merits a same-model, same-plan batch. None earns a universal recommendation from this record.
Cloud APIs are a separate purchasing and engineering decision: programmatic model choice, billing, retries, logging and export handling. No funded generation test was completed against the separate Veo, Runway, Kling or OpenAI image APIs. The BFL dashboard attempt produced no output. Hosted app results do not validate their corresponding APIs, and subscription credits must not be assumed transferable.
Local generation gives control over the saved workflow and hardware, but requires installation, storage, updates, security and license review. This record did not measure clean-install time, available storage, power draw, thermals or peak VRAM. Hardware, electricity, maintenance, retries and human finishing remain costs even without per-generation vendor credits. The Wan run demonstrates why a valid render can still be impractical on a particular machine.
Local licenses are checkpoint-specific
Current original model cards list FLUX.1 [schnell] and Wan2.2 TI2V-5B under Apache 2.0. That does not audit every redistributed FP8 file or dependency. RunDiffusion’s Juggernaut-XL v9 README restricts paid API deployment without explicit licensing; the source records a differing CivitAI permissions presentation. That conflict is not resolved here, and no blanket commercial clearance is given.
The source records the original LTXV-2B weights under LTXV Open Weights License 0.X, with a separate agreement at annual revenue of $10 million or more; the exact license was not freshly retrievable. Original SDXL base/refiner, SDXL Turbo, newer Stability models, FLUX [dev] and [schnell] must not be treated as one license. ComfyUI’s software license does not license its model weights. Review the exact downloaded model, redistribution terms and intended use before production.
Choose by production job
| Job | Prioritize | Reject or retest when |
|---|---|---|
| Thumbnails and social graphics | Fast iteration, readable headline, intended-use rights | Text is garbled or the design needs more repair than a conventional layout. Consider adding final typography in an editor. |
| Product mockups and concept art | Precise objects, localized edits, adequate resolution | A concept could be mistaken for a photograph of the actual product. Label it clearly. |
| Character or brand illustration series | Same identity in different poses and scenes | The only evidence is a similar-looking batch or unchanged cached image. |
| B-roll and stock-footage replacement | Physical contact, camera motion, usable duration | Hands, water, fabric or crowds fail close inspection; timing does not fit the edit. |
| Social ads and short-form motion | Iteration cost, commercial and advertising permission | The plan allows a clean export but not the intended commercial use. |
| Talking-head or avatar explainers | Consented likeness, lip sync, pacing and audience trust | Synthetic presentation undermines the message. Avatar/lip-sync performance was not tested here. |
| Image-to-video animation | Clean source, preserved details, controlled motion | The source has defects that animation carries forward or output resolution drops below the delivery need. |
For longer edits, budget shot selection, continuity and stitching as well as generation. Use the AI voice guide for narration and the software-stack guide for the surrounding production workflow.
Copy-and-run test
Image
A woman in a red jacket holding a coffee cup with the word 'OPEN' printed on a sign behind her, standing in front of a bookshelf, photorealistic, three books visible on the shelf.
Generate this exact prompt five times with identical settings; record native batch size and seeds. Score sign placement separately from spelling and text elsewhere in the frame, count every book and cup, and compare identity. Then use the best result as a reference for the same subject in a new pose or setting. Record any prompt edits or model substitutions rather than silently improving one candidate’s test.
Video
Generate a five-second clip of "a person in a red jacket walking across a room and picking up a coffee cup from a wooden table, camera slowly pushing in."
Repeat with the same settings. Check the starting position of the cup, actual lift, fingers, camera movement, jacket color, visible face, warped or duplicated objects, and measured duration and resolution. Listen to audio separately. Save the prompt, model, plan, date, settings, before/after credit balance, quoted wait, actual elapsed time, file properties, watermark and review notes. Ask for a visible face if identity is part of your follow-up test and record that prompt change.
Your go/no-go scorecard
Rights and consent: pass/fail first. Document output permissions, training-data disclosures, authorized references and likenesses, intended commercial channels, and any post-cancellation restrictions. A quality score cannot compensate for missing rights.
| Criterion | Weight | Evidence to require |
|---|---|---|
| Consistency control | 25% | Character, product or style holds across repeated scenes with acceptable correction. |
| Use-case performance | 25% | The actual job passes: text, count, motion, lip sync or resolution. |
| Finished-asset economics | 20% | Subscriptions, credits, retries, review, finishing and discarded output are included. |
| Audience trust and disclosure | 15% | You can accurately explain what was generated and what a human checked. |
| Workflow fit | 15% | Files, resolution, exports and revision controls fit the editing pipeline. |
The guide’s proposed threshold is 80/100 after rights pass, with rejection of any tool that fails the essential job even if its total is high. These are editorial decision weights, not measured vendor scores.
Cost per approved asset
Use (allocated tool cost + human review + finishing + other production costs) ÷ approved assets. Illustrative example, not test spending or a current plan quote: a $20 plan, 60 attempts to obtain ten approved images, two review hours at $35 and one finishing hour at $35 total $125, or $12.50 per approved image. The attempt count is an assumption, not a measured typical rate. Compare that finished cost with stock licensing or commissioned work.
Publish with provenance
Keep the model, prompt/reference, date, permissions and human review record. A disclosure such as “This product mockup is an AI-generated concept visual, not a photo of the finished product” explains more than a vague AI label. For generated footage, state that it was generated and reviewed. Content Credentials can carry provenance; their presence does not itself establish accuracy or rights.
Investigate unclear training provenance, likeness permissions, commercial exceptions, cancellation terms and removal procedures before use. Prefer real product photography when accuracy matters, authentic documentary imagery, and a human creator when repair costs exceed conventional production. Obtain documented consent for identifiable people. Purely generated work raises copyrightability questions; a vendor’s permission to use output is different from copyright protection or freedom from third-party claims. Copyright Office guidance · YouTube disclosure rules.
Still untested: larger independent repeat sessions, reference consistency across a full series, fresh Pika signup behavior, paid Runway and Midjourney output, the funded APIs, avatar lip sync, Kling audio quality, and a clean local installation. Exact account-change causes, Luma charges/pricing, and the identified license questions remain open. The absence of a measurement is not a pass or a failure.
Research behind this guide
Google Flow — current generation credit costsRunway — plans, free credits and model accessPika — credits, plans and commercial-license differencesIdeogram — plans and model lineupRecraft — pricing, ownership and commercial useOpenArt — plans and export featuresMidjourney — subscription comparison and commercial termsAdobe — generative credits and commercial-use guidanceLuma — published pricing (account discrepancies remain unresolved)Black Forest Labs — pricing and model selectionFLUX.1 [schnell] — original model card and Apache 2.0 licenseWan2.2 TI2V-5B — model card and Apache 2.0 licenseRunDiffusion — Juggernaut-XL v9 commercial restrictionsU.S. Copyright Office — AI and copyrightabilityYouTube — disclosing altered or synthetic contentC2PA — content provenance standardHow we’ll keep this guide current
Material changes to the cited guidance, prices, availability, or commercial terms will trigger a review before this page is republished.
