Adobe claims its new image-generation model is its best yet

Adobe’s Firefly, a suite of generative AI models, has faced criticism within the creative community, particularly for its image-generation model. This specific model, known for its shortcomings compared to competitors like Midjourney and OpenAI’s DALL-E 3, has been disparaged for its tendency to produce distorted limbs and landscapes and to miss subtle nuances in prompts. However, Adobe is striving to address these concerns with the release of its third-generation model, Firefly Image 3, unveiled this week at the company’s Max London conference.

The latest iteration of the Firefly model, now integrated into Photoshop (beta) and Adobe’s Firefly web app, promises to deliver more “realistic” imagery compared to its predecessors, Image 1 and Image 2. This improvement is attributed to its enhanced ability to comprehend longer and more complex prompts and scenes, along with advancements in lighting and text-generation capabilities. Adobe asserts that Firefly Image 3 will offer improved rendering of typography, iconography, raster images, and line art. Additionally, the model is touted to excel in depicting dense crowds and individuals with “detailed features” and a wide range of emotions and expressions.

Notice the lighting in this headshot from Image 3 compared to the one below it, from Image 2:

From Image 3. Prompt: “Studio portrait of young woman.” Image Credits: Adobe

Same prompt as above, from Image 2. Image Credits: Adobe

The Image 3 output looks more detailed and lifelike to my eyes, with shadowing and contrast that’s largely absent from the Image 2 sample.

Here’s a set of images showing Image 3’s scene understanding at play:

From Image 3. Prompt: “An artist in her studio sitting at desk looking pensive with tons of paintings and ethereal.” Image Credits: Adobe

“An artist in his studio sitting at desk looking pensive with tons of paintings and ethereal.” From Image 2. Image Credits: Adobe

It’s worth noting that while the sample from Image 2 is relatively basic in terms of detail and expressiveness, the output from Image 3 showcases a significant improvement in both aspects. However, there are still noticeable imperfections, particularly in the shirt area of the subject in the Image 3 sample. Despite this, the pose in Image 3 is more intricate compared to the subject’s pose in Image 2, which may contribute to the complexity of the generated image. Additionally, it’s observed that the clothing in Image 2 also exhibits some inaccuracies.

Undoubtedly, some of the improvements seen in Image 3 can be attributed to a larger and more diverse training dataset. Similar to Image 1 and Image 2, Image 3 undergoes training on content sourced from Adobe Stock, which includes uploads from contributors, licensed content, and public domain material. As Adobe Stock continues to expand, so does the available training data for Firefly models.

In an attempt to differentiate itself from other generative AI vendors and uphold ethical standards, Adobe has implemented a program to compensate Adobe Stock contributors for their contributions to the training dataset. However, the specifics of this program remain somewhat opaque. Additionally, Adobe’s decision to train Firefly models on AI-generated images has sparked controversy, with critics labeling it as a form of data laundering.

Recent reporting by Bloomberg has shed light on the inclusion of AI-generated images from Adobe Stock in Firefly’s training data. This practice raises concerns, particularly regarding the potential inclusion of copyrighted material in the AI-generated images. Adobe defends this approach by asserting that AI-generated images constitute only a small fraction of the training data and undergo moderation to prevent the depiction of trademarks, recognizable characters, or references to artists’ names.

Nevertheless, despite efforts to curate diverse and ethically sourced training data, and the implementation of content filters and other safeguards, a flaw-free experience cannot be guaranteed. Instances such as users generating objectionable content, like people making offensive gestures with Image 2, highlight the inherent challenges. Ultimately, the true efficacy of Image 3 will be determined once it is widely adopted by the community and put to the test in practical applications.

New AI-powered features

Image 3 introduces a range of new features in Photoshop, expanding beyond its enhanced text-to-image capabilities. One notable addition is the inclusion of a “style engine,” accompanied by an auto-stylization toggle, which enables the model to generate a wider spectrum of colors, backgrounds, and subject poses. These features are complemented by the Reference Image option, allowing users to condition the model based on an image’s colors or tone for aligning future generated content.

Furthermore, Image 3 powers three new generative tools within Photoshop: Generate Background, Generate Similar, and Enhance Detail. Generate Background seamlessly replaces backgrounds with generated alternatives that blend seamlessly into existing images, while Generate Similar offers variations of selected portions of a photo, such as individuals or objects. Enhance Detail fine-tunes images to enhance sharpness and clarity, providing users with precision editing capabilities.

Although these features have been available in beta on the Firefly web app for some time, their integration into Photoshop marks their official debut. Additionally, Adobe is not overlooking its web app, introducing Structure Reference and Style Reference alongside the release of Image 3. Structure Reference enables users to generate images that match the structure of a reference image, while Style Reference facilitates style transfer while preserving the content of the original image.

These enhancements offer users advanced creative control, providing versatile tools for generating and editing images with greater precision and flexibility.

Here’s Structure Reference in action:

Original image. Image Credits: Adobe

Transformed with Structure Reference. Image Credits: Adobe

And Style Reference:

Original image. Image Credits: Adobe

Transformed with Style Reference. Image Credits: Adobe

Adobe confirmed that, despite the extensive upgrades to Firefly image-generation capabilities, its pricing structure will remain unchanged for the time being. The current tiers, including the cheapest Firefly premium plan priced at $4.99 per month, will remain in place, maintaining Adobe’s competitive edge against rivals like Midjourney and OpenAI.

Additionally, Adobe affirmed that its generative credit system will continue to be utilized, allowing users to access and generate content within their chosen plan parameters. The indemnity policy, which ensures Adobe will cover copyright claims related to works generated in Firefly, will also remain unchanged, providing users with peace of mind regarding legal concerns.

Furthermore, Adobe reiterated its commitment to watermarking AI-generated content and maintaining transparency regarding its origin. Content Credentials, metadata designed to identify AI-generated media, will continue to be automatically attached to all Firefly image generations, whether created from scratch or through the use of generative features in the web interface or Photoshop.

Overall, Adobe’s decision to retain its current pricing structure and policies reflects its dedication to providing accessible and reliable AI-powered creative tools while upholding ethical and legal standards in content generation and distribution.