Is Nano Banana Pro a Low-Level Vision All-Rounder? A Comprehensive Evaluation on 14 Tasks and 40 Datasets

Abstract: The rapid evolution of text-to-image generation models has revolutionized visual content creation. While commercial products like Nano Banana Pro have garnered significant attention, their potential as generalist solvers for traditional low-level vision challenges remains largely underexplored. In this study, we investigate the critical question: Is Nano Banana Pro a Low-Level Vision All-Rounder? We conducted a comprehensive zero-shot evaluation across 14 distinct low-level tasks spanning 40 diverse datasets. By utilizing simple textual prompts without fine-tuning, we benchmarked Nano Banana Pro against state-of-the-art specialist models. Our extensive analysis reveals a distinct performance dichotomy: while \textbf{Nano Banana Pro demonstrates superior subjective visual quality}, often hallucinating plausible high-frequency details that surpass specialist models, it lags behind in traditional reference-based quantitative metrics. We attribute this discrepancy to the inherent stochasticity of generative models, which struggle to maintain the strict pixel-level consistency required by conventional metrics. This report identifies Nano Banana Pro as a capable zero-shot contender for low-level vision tasks, while highlighting that achieving the high fidelity of domain specialists remains a significant hurdle.

• The paper demonstrates that Nano Banana Pro excels in subjective visual quality in a zero-shot evaluation across 14 low-level vision tasks but struggles with traditional pixel-level fidelity metrics.
• It employs a comprehensive evaluation using 40 diverse datasets to compare its generative performance against specialist models in tasks like dehazing and deraining.
• The study suggests integrating generative models with task-specific constraints to overcome fidelity challenges and improve practical applications in low-level vision.

Comprehensive Evaluation of Nano Banana Pro as a Low-Level Vision All-Rounder

Nano Banana Pro, a commercial text-to-image generation model, has attracted considerable interest due to its versatile capabilities. This paper investigates the potential of Nano Banana Pro to act as a generalist tool for low-level vision tasks, which traditionally require specialized models. The study involves a zero-shot evaluation of Nano Banana Pro across 14 distinct low-level vision tasks using 40 diverse datasets. The model's performance in generating visually superior outputs is juxtaposed with its limitations in achieving traditional reference-based quantitative metrics.

Experimental Setup

The evaluation of Nano Banana Pro was conducted without fine-tuning, using simple textual prompts, which contrasts starkly with the complex fine-tuning pipelines typically employed for specialist models. The zero-shot evaluation spans a variety of tasks, including image restoration, enhancement, and fusion.

(Below is a visual representation of some exemplary zero-shot results.)

Figure 1: Exemplary zero-shot results of Nano Banana Pro across 14 low-level vision tasks. This figure illustrates the model's potential to restore and enhance images despite no specific task training.

Findings

Perceptual Quality vs. Quantitative Metrics

The study reveals a distinct performance dichotomy. Nano Banana Pro consistently excels in subjective visual quality, often generating plausible high-frequency details that surpass those produced by specialist models. However, the model struggles to maintain the pixel-level consistency required by conventional metrics like PSNR and SSIM. This discrepancy is primarily attributed to the inherent stochasticity of generative models.

Image Restoration Tasks

Nano Banana Pro performs competitively in tasks such as dehazing and deraining, where it generates sharp edges and realistic textures pleasing to human observers. Yet, these outputs frequently score lower on reference-based metrics due to deviations from pixel-perfect fidelity.

(Figure 2 and Figure 3 showcase specific examples.)

Figure 2: NB Pro dehazing visual results on the RTTS dataset. Despite hazy conditions, NB Pro effectively recovers blurred details.

Figure 3: A comparison of visual results between NB Pro and other methods, illustrating NB Pro's superior clarity but issues with over-enhancement artifacts.

Limitations and Challenges

Despite its strengths, Nano Banana Pro's generative nature imposes limitations. It often sacrifices strict fidelity for semantic plausibility, which undermines its effectiveness in tasks demanding precise correspondence to ground truth data. The model struggles with maintaining spectral fidelity and atmospheric consistency, as evidenced by noticeable color shifts and hallucinated content in some scenarios.

(Figure 4 highlights some failure cases.)

Figure 4: Anamorphic example images showcasing color fidelity issues in dehazing tasks.

Implications and Future Directions

The paper suggests that while Nano Banana Pro is a promising zero-shot contender for low-level tasks, achieving high fidelity akin to domain-specific specialists remains a significant hurdle. The inherent trade-off between perceptual quality and pixel fidelity calls for the development of new evaluation paradigms that reconcile these aspects. Exploring hybrid model architectures that combine NB Pro's generative strengths with task-specific physical constraints could enhance its utility.

Conclusion

Nano Banana Pro demonstrates notable potential as a generalist tool for zero-shot low-level vision tasks, particularly in contexts where perceptual quality is paramount. However, its inability to consistently adhere to traditional metrics highlights the need for further research into integrating generative models with domain-specific constraints and developing advanced evaluation frameworks.