Using layer information to improve control of image generation models