Higher than JPEG? Researcher discovers that Secure Diffusion can compress photographs

Final week, Swiss software program engineer Matthias Bühlmann found that the favored picture synthesis mannequin Secure Diffusion might compress current bitmapped photographs with fewer visible artifacts than JPEG or WebP at excessive compression ratios, although there are important caveats.

Secure Diffusion is an AI picture synthesis mannequin that usually generates photographs based mostly on textual content descriptions (known as “prompts”). The AI mannequin realized this capability by learning tens of millions of photographs pulled from the Web. Through the coaching course of, the mannequin makes statistical associations between photographs and associated phrases, making a a lot smaller illustration of key details about every picture and storing them as “weights,” that are mathematical values that symbolize what the AI picture mannequin is aware of, so to talk.

When Secure Diffusion analyzes and “compresses” photographs into weight kind, they reside in what researchers name “latent house,” which is a manner of claiming that they exist as a form of fuzzy potential that may be realized into photographs as soon as they’re decoded. With Secure Diffusion 1.4, the weights file is roughly 4GB, but it surely represents information about tons of of tens of millions of photographs.

Whereas most individuals use Secure Diffusion with textual content prompts, Bühlmann minimize out the textual content encoder and as an alternative compelled his photographs via Secure Diffusion’s picture encoder course of, which takes a low-precision 512×512 picture and turns it right into a higher-precision 64×64 latent house illustration. At this level, the picture exists at a a lot smaller information dimension than the unique, however it might nonetheless be expanded (decoded) again right into a 512×512 picture with pretty good outcomes.

Whereas working exams, Bühlmann discovered that photographs compressed with Secure Diffusion seemed subjectively higher at greater compression ratios (smaller file dimension) than JPEG or WebP. In a single instance, he reveals a photograph of a sweet store that’s compressed down to five.68KB utilizing JPEG, 5.71KB utilizing WebP, and 4.98KB utilizing Secure Diffusion. The Secure Diffusion picture seems to have extra resolved particulars and fewer apparent compression artifacts than these compressed within the different codecs.

Bühlmann’s technique at present comes with important limitations, nevertheless: It isn’t good with faces or textual content, and in some circumstances, it might truly hallucinate detailed options within the decoded picture that weren’t current within the supply picture. (You in all probability don’t need your picture compressor inventing particulars in a picture that do not exist.) Additionally, decoding requires the 4GB Secure Diffusion weights file and additional decoding time.

Whereas this use of Secure Diffusion is unconventional and extra of a enjoyable hack than a sensible resolution, it might probably level to a novel future use of picture synthesis fashions. Bühlmann’s code may be discovered on Google Colab, and you will find extra technical particulars about his experiment in his publish on In the direction of AI.