NVIDIA has introduced a new AI Blueprint that enables users to generate images guided by 3D scenes using Blender.
This tool combines basic 3D scene design with depth mapping to help control image composition more precisely during AI generation. At the core of the process is FLUX.1-dev, an AI model developed by Black Forest Labs, which interprets the scene's spatial layout alongside text prompts to produce visuals that match the intended design.
Depth maps play a crucial role by providing the spatial context needed for the model to understand scene structure. This technique simplifies the process by removing the need for detailed textures or complex objects, instead relying on general spatial information. With the scenes rendered in 3D, users have the flexibility to move elements and adjust camera angles to suit their creative goals.
The Blueprint includes an NVIDIA NIM microservice that helps deploy the FLUX.1-dev model efficiently on RTX GPUs, using TensorRT for faster inference. It's packaged with an installer and comprehensive deployment instructions, making it accessible for AI artists looking to integrate generative tools into their workflow.
Beyond entry-level users, the Blueprint is also designed to accommodate advanced developers. It offers a customisable pipeline that can be modified for more sophisticated needs. NVIDIA provides supporting materials like sample assets, detailed documentation, and a preconfigured environment to help streamline experimentation and creation.
Optimised for NVIDIA RTX AI PCs and workstations, the solution benefits from the company's Blackwell architecture. The FLUX.1-dev model has been fine-tuned using TensorRT and quantised to FP4 precision, resulting in more than double the inference speed compared to traditional FP16 PyTorch implementations. There are also FP8 model versions tailored for GPUs based on the Ada Lovelace architecture, further expanding compatibility and performance.
Quantising to FP4 significantly reduces the model size, lowering memory requirements while maintaining high performance, cutting VRAM needs by more than half compared to FP16. Currently, NVIDIA offers ten NIM microservices across areas like image generation, natural language processing, speech AI, and computer vision. The company plans to continue building out its portfolio with more AI blueprints and services designed to accelerate creative and technical workflows.
