Using only a single input image, HyperDreamBooth is able to personalize a text-to-image diffusion model 25x faster than DreamBooth, by using (1) a HyperNetwork to generate an initial prediction of a subset of network weights that are then (2) refined using fast finetuning for high fidelity to subject detail. Our method both conserves model integrity and style diversity while closely approximating the subject's essence and details...
Our approach consists of 3 core elements: Lightweight DreamBooth (LiDB), a HyperNetwork that predicts LiDB weight, and rank-relaxed fast fine-tuning.