This amazing new AI tool turns a scribble into a finished image in seconds
If you use AI to create visuals, there’s a new tool to add to your arsenal. Stable Doodle, a new tool from Stability, gives you (almost) full control of the image-making process through a clever interface that upends the traditional text-only prompting of most AI workflows.
Stable Doodle turns rudimentary doodles into colorful, fully rendered images. Available now for free as part of Stability AI’s Clipdrop generative AI tool suite, Stable Doodle allows you to scribble any rough idea, from the basic shape of object to a full professional storyboard-level drawing, and create an image within seconds. It’s akin to being an art director working with a team of great concept artists and illustrators.
Using it is as simple as this: first, draw anything you want in black and white on the Stable Doodle canvas. Then write a prompt to tell Stable Diffusion how to interpret it. Click the “generate” button and watch four different pics appear right in front of you, rendered with impressive precision. Check out how it works with this chair and the prompt “A comfy chair, “isometric’ style”:
Or this really rough outline, that gets transformed into a picture perfect, book cover-worthy illustration with the prompt “Castle on a hill, winter, ‘Anime’ Style”:
It also works with more detailed sketches, if you are better at drawing than most people. Like this living room sketch getting transformed into a “living room, comic-book style”:
The software is not limited to illustration styles. Stable Doodle can use any of the 14 styles available in SDXL, the latest incarnation of the Stability’s generative AI engine, which include cinematic, fantasy art, and the origami-style owl you see below.
Towards a new generative AI user experience
Stable Doodle is not the first generative AI capable of turning sketches into images, but it is the most advanced. Last year, Nvidia launched Canvas, a sketch-to-image tool that is adept at creating landscapes. But Stable Doodle is poised to be a great tool for speeding up workflows because of its simplicity. Once a basic sketch is in hand, artists can augment it with software like Photoshop, Sketch, or Illustrator. In the future, the tool will only become more compelling. Since Stability publishes its code, it’s only a matter of time before we see this same technology combined with custom models trained on the specific artwork of the professional using the tool. We’re getting close to a moment in which every artist can use AI to form a full-on studio, with a suite of AI helpers to execute on a vision, not unlike the painters of yore and their studios full of apprentices.
And it’s never been easier to do so. By allowing a visual input to guide the prompt, we are beginning to truly harness the power of these generative tools to obtain exactly what we want. Like with DragGAN, an interactive artificial intelligence app that allows users to change the position and perspective of subjects by dragging vector lines, the human hand becomes again the crucial element that gives shape to our imagination.
As the research paper for DragGAN noted, “synthesizing visual content that meets users’ needs often requires flexible and precise controllability of the pose, shape, expression, and layout of the generated objects.” Stable Doodle’s sketch direction does this for image generation, not just editing. As the creator of DragGAN—Xingang Pan—told me over video chat a few weeks ago, the future of the AI user experience is a combination of text input and direct object manipulation with your own hands.
Under the hood
Stability says Stable Doodle software combines two pieces of technology to make this magic work: Stable Diffusion XL and T2I-Adapter, an algorithm developed by the Advanced Research Center of Tencent, the powerful Chinese conglomerate that owns WeChat.
The TSI-Adapter adds parameters that guide diffusion models to generate the desired output. In the case of Stable Doodle, the company says the algorithm provides “supplementary guidance” to SDXL that enables the model to understand the outline of sketches and combine that understanding with text prompts.
According to a Stability spokesperson, Stable Doodle is yet another step in their strategy to turn Clipdrop into a toolbox for aficionados and professionals, designed to edit and create images in a simple way using generative AI models. The organization will keep adding more of these tools in the future. “We are constantly exploring combinations of models to offer new experiences,” a spokesperson said over email.
Stable Doodle is now available on both the Clipdrop website and its mobile app, which is available for iOS and Android. It is free, even without a login, but limited to a daily quota not yet specified by Stability. Like with other generative AI apps, you can pay credits to go beyond the limits and get faster processing times.
(6)
