Image Analysis | AI Application Building

Gemini Is Multimodal

Gemini can process more than text. A request may also include images, documents, audio, and video.

When an application combines different types of information in one request, the process is described as multimodal prompting.

The request now contains two inputs

The text tells Gemini what task to perform. The image supplies the visual information Gemini should examine.

What Image Analysis Can Do

Depending on the image and instructions, Gemini can help with:

Describing visible content
Reading visible text
Reviewing screenshots and interfaces
Summarizing charts or diagrams
Finding prominent objects
Comparing visual elements
Evaluating layout and presentation
Answering questions about an uploaded image

Image analysis is not guaranteed truth

A model may overlook details, misread small text, confuse objects, or make an unsupported interpretation. Important results still require human review.

Try the Image Analyzer

Upload a JPEG, PNG, WebP, or BMP image. Then tell Gemini what you want it to examine.

LIVE MULTIMODAL EXERCISE

Gemini Image Analyzer

Upload one image and send it with a text instruction through the secure PHP endpoint.

Ready

Upload an Image

🖼️ Drop an image here or select one from your device JPEG, PNG, WebP, or BMP • Maximum 8 MB

SAMPLE ANALYSIS TASKS

Analysis Instruction 0 / 2,000

How the Image Reaches Gemini

The browser first uploads the image to our PHP endpoint. PHP validates the file before sending anything to Gemini.

The server checks:

Whether an upload was supplied
Whether the upload completed successfully
The actual MIME type
The file size
The image dimensions
Whether PHP can read it as an image
The current course security token
The request rate for the session

Never Trust the Filename

A filename such as:

company-photo.jpg

does not prove that the file contains a JPEG image.

The endpoint uses PHP Fileinfo to inspect the uploaded bytes and determine the actual MIME type.

File extensions are labels

Users can rename files. Application security must inspect the uploaded content rather than trusting its name or browser-supplied type.

Inline Image Data

After validation, PHP reads the temporary image and converts its bytes into base64 text:

$imageBytes = file_get_contents($temporaryPath);

$base64Image = base64_encode($imageBytes);

The image becomes part of the Gemini request:

{
    "model": "gemini-3.5-flash",

    "input": [
        {
            "type": "text",
            "text": "Describe the important visible content."
        },

        {
            "type": "image",
            "data": "BASE64_ENCODED_IMAGE",
            "mime_type": "image/jpeg"
        }
    ]
}

Google currently allows inline image data when the complete request remains within its inline request-size limit. Larger or reusable files should use the Gemini Files API instead.

Why the Prompt Comes First

Our input array places the text instruction before the image:

"input": [
    {
        "type": "text",
        "text": "Analyze this screenshot."
    },

    {
        "type": "image",
        "data": "...",
        "mime_type": "image/png"
    }
]

This gives the model the task before it processes the supplied visual information.

The Image Is Not Permanently Stored

PHP receives uploads in a temporary server location. This exercise reads the image from that location and sends it to Gemini without moving it into a permanent public folder.

The application does not need to save every uploaded image merely because it needs to analyze it.

Process only what you need

Temporary processing reduces unnecessary storage, cleanup work, exposed file URLs, and retained user data.

The Browser Preview Is Separate

JavaScript creates a temporary browser preview using the selected local file.

That preview does not mean the image has already been uploaded. The upload begins only when the user clicks Analyze Image.

Image Size and Token Usage

Image inputs also consume tokens. Larger images may be divided into visual tiles during model processing.

This means image dimensions can affect:

Token usage
Processing time
Request cost
The model's ability to examine small details

Bigger is not automatically better. A clear image cropped to the relevant content can be more useful than an enormous image containing unnecessary empty space.

Write a Specific Analysis Prompt

This prompt is vague:

What is this?

This version gives the application a clearer task:

Review this website screenshot.

Describe the page structure, identify the primary call to action,
and list three usability improvements.

Separate visible facts from subjective recommendations.

A useful image prompt explains:

What visual information matters
What questions should be answered
How the answer should be organized
What the model should avoid assuming

Reading Text From Images

Gemini can often read text visible in screenshots, signs, forms, labels, charts, and photographs.

Results become less dependable when text is:

Very small
Blurry
Partially hidden
Distorted
Low contrast
Rotated incorrectly
Written in an unusual typeface

Ask the model to identify uncertain words instead of pretending every character was read confidently.

Observations Versus Interpretations

An image may clearly show a person holding an open laptop. It may not prove why the person is holding it, what they are working on, or where the image was taken.

Good analysis distinguishes:

Visible Observation

A laptop is open on a desk beside a coffee cup.

Uncertain Interpretation

The setting may be a home office, but the location cannot be
confirmed from the image alone.

Do not turn guesses into facts

Instructions should ask Gemini to label uncertainty and avoid inventing names, locations, motives, events, or circumstances.

Private and Sensitive Images

Before accepting user images, application owners should decide what content is appropriate for the tool.

Avoid casually uploading images containing:

Private identification documents
Financial account information
Medical records
Private customer information
Passwords or API keys
Confidential business documents

A technically functioning upload form does not remove the need for responsible data-handling rules.

Inline Data Versus the Files API

Inline Data

Best for relatively small files used in one immediate request.

Files API

Better when a file is larger, must be reused, or will be referenced across multiple requests.

This lesson uses inline data because the image is small, temporary, and used once.

Practical Exercise

Upload a clear photograph or screenshot.
Run the general-description prompt.
Review the reported token usage.
Run a more specific original instruction.
Ask Gemini to separate visible observations from uncertain interpretations.
Test an image containing visible text.
Compare what Gemini reads correctly with anything it marks as uncertain.
Copy one useful analysis.

Completion goal

You should understand how a browser upload is validated, converted into inline image data, combined with a text prompt, and processed by a multimodal Gemini model.

Official Resources

Gemini Image Understanding Guide ↗

Gemini File Input Methods ↗

Lesson Summary

A multimodal Gemini request can contain both text instructions and image data.

The browser uploads the image to PHP. PHP validates its size, real MIME type, readability, and dimensions before encoding it and sending it to Gemini.

The API key remains on the server, the image is not permanently stored by this course, and the browser receives only the generated analysis and safe file metadata.