> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/googleapis/python-genai/llms.txt
> Use this file to discover all available pages before exploring further.

# Multimodal Input

> Learn how to work with images, audio, video, and PDFs as input to Gemini models using the Google Gen AI Python SDK.

Gemini models can process multiple types of media in addition to text. This guide covers how to provide images, audio, video, and PDF files as input.

## Images

There are three main ways to provide image input:

<Tabs>
  <Tab title="Cloud Storage (GCS)">
    Use `Part.from_uri` for images stored in Google Cloud Storage:

    ```python theme={null}
    from google import genai
    from google.genai import types

    client = genai.Client(api_key='your-api-key')

    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[
            'What is this image about?',
            types.Part.from_uri(
                file_uri='gs://generativeai-downloads/images/scones.jpg',
                mime_type='image/jpeg',
            ),
        ],
    )
    print(response.text)
    ```

    Supported image formats: JPEG, PNG, WebP, GIF
  </Tab>

  <Tab title="Local Files (Bytes)">
    Use `Part.from_bytes` for local image files:

    ```python theme={null}
    from google.genai import types

    with open('your_image_path.jpg', 'rb') as f:
        image_bytes = f.read()

    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[
            'What is this image about?',
            types.Part.from_bytes(
                data=image_bytes,
                mime_type='image/jpeg'
            ),
        ],
    )
    print(response.text)
    ```

    <Note>
      Best for images under 20MB. For larger files, use the File API.
    </Note>
  </Tab>

  <Tab title="File API">
    Upload images to the File API first (Gemini Developer API only):

    ```python theme={null}
    # Upload the image
    file = client.files.upload(file='image.jpg')

    # Use it in generate_content
    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=['Describe this image', file]
    )
    print(response.text)
    ```

    Best for large images or when reusing the same image multiple times.
  </Tab>
</Tabs>

## Audio

Process audio files for transcription, analysis, or understanding:

<Tabs>
  <Tab title="Local Audio (Bytes)">
    ```python theme={null}
    from google.genai import types

    with open('audio_sample.mp3', 'rb') as f:
        audio_bytes = f.read()

    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[
            types.Part.from_bytes(
                data=audio_bytes,
                mime_type='audio/mp3',
            ),
            'Transcribe this audio.'
        ]
    )
    print(response.text)
    ```

    Supported audio formats: MP3, WAV, FLAC, AAC
  </Tab>

  <Tab title="Cloud Storage Audio">
    ```python theme={null}
    from google.genai import types

    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[
            'What is being discussed in this audio?',
            types.Part.from_uri(
                file_uri='gs://your-bucket/audio.mp3',
                mime_type='audio/mp3',
            ),
        ],
    )
    print(response.text)
    ```
  </Tab>

  <Tab title="File API (Long Audio)">
    For long audio files, upload to the File API first:

    ```python theme={null}
    # Upload
    audio_file = client.files.upload(file='podcast.mp3')

    # Wait for processing to complete
    while audio_file.state == 'PROCESSING':
        time.sleep(2)
        audio_file = client.files.get(name=audio_file.name)

    # Generate content
    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[audio_file, 'Summarize this podcast.']
    )
    print(response.text)
    ```
  </Tab>
</Tabs>

## Video

Analyze video content for descriptions, summaries, or specific questions:

<Tabs>
  <Tab title="Cloud Storage Video">
    ```python theme={null}
    from google.genai import types

    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[
            'What happens in this video?',
            types.Part.from_uri(
                file_uri='gs://your-bucket/video.mp4',
                mime_type='video/mp4',
            ),
        ],
    )
    print(response.text)
    ```

    Supported video formats: MP4, MOV, AVI, WebM, FLV, MPG
  </Tab>

  <Tab title="File API (Recommended)">
    The File API is recommended for videos:

    ```python theme={null}
    # Upload
    video_file = client.files.upload(file='video.mp4')

    # Wait for processing
    import time
    while video_file.state == 'PROCESSING':
        time.sleep(5)
        video_file = client.files.get(name=video_file.name)

    # Generate content
    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[video_file, 'What happens in this video?']
    )
    print(response.text)
    ```

    <Warning>
      Video files typically require processing time. Always check the file state before using it.
    </Warning>
  </Tab>

  <Tab title="Video Frames">
    Extract and analyze specific frames:

    ```python theme={null}
    from google.genai import types

    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[
            'Describe what happens at the 30 second mark',
            types.Part.from_uri(
                file_uri='gs://your-bucket/video.mp4',
                mime_type='video/mp4',
            ),
        ],
    )
    print(response.text)
    ```
  </Tab>
</Tabs>

## PDFs

Extract information from PDF documents:

<Tabs>
  <Tab title="File API (Gemini Developer API)">
    ```python theme={null}
    # Upload PDF
    pdf_file = client.files.upload(file='document.pdf')

    # Wait for processing
    import time
    while pdf_file.state == 'PROCESSING':
        time.sleep(2)
        pdf_file = client.files.get(name=pdf_file.name)

    # Ask questions about the PDF
    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=['Summarize this document', pdf_file]
    )
    print(response.text)
    ```
  </Tab>

  <Tab title="Cloud Storage (Vertex AI)">
    ```python theme={null}
    from google.genai import types

    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[
            'What are the key findings in this research paper?',
            types.Part.from_uri(
                file_uri='gs://your-bucket/research-paper.pdf',
                mime_type='application/pdf',
            ),
        ],
    )
    print(response.text)
    ```
  </Tab>

  <Tab title="Multiple PDFs">
    Analyze multiple PDFs together:

    ```python theme={null}
    from google.genai import types

    # Upload two PDFs
    file1 = client.files.upload(file='paper1.pdf')
    file2 = client.files.upload(file='paper2.pdf')

    # Use them together
    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[
            'Compare and contrast these two research papers.',
            file1,
            file2
        ]
    )
    print(response.text)
    ```
  </Tab>
</Tabs>

## Combining Multiple Modalities

You can mix different media types in a single request:

```python theme={null}
from google.genai import types

# Upload files
image_file = client.files.upload(file='chart.png')
audio_file = client.files.upload(file='presentation.mp3')

response = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[
        'Based on this chart and audio presentation, ',
        image_file,
        audio_file,
        'what are the main conclusions?'
    ]
)
print(response.text)
```

## MIME Types Reference

Common MIME types for different media:

| Media Type | MIME Type Examples                                   |
| ---------- | ---------------------------------------------------- |
| Images     | `image/jpeg`, `image/png`, `image/webp`, `image/gif` |
| Audio      | `audio/mp3`, `audio/wav`, `audio/flac`, `audio/aac`  |
| Video      | `video/mp4`, `video/mov`, `video/avi`, `video/webm`  |
| PDF        | `application/pdf`                                    |

## File API Management

Manage uploaded files:

```python theme={null}
# Upload
file = client.files.upload(file='document.pdf')
print(f"Uploaded: {file.name}")

# Get file info
file_info = client.files.get(name=file.name)
print(f"State: {file_info.state}")
print(f"Size: {file_info.size_bytes} bytes")

# List all files
for f in client.files.list():
    print(f"{f.name}: {f.state}")

# Delete when done
client.files.delete(name=file.name)
```

## Streaming with Multimodal Input

You can stream responses for multimodal inputs:

```python theme={null}
from google.genai import types

with open('image.jpg', 'rb') as f:
    image_bytes = f.read()

for chunk in client.models.generate_content_stream(
    model='gemini-2.5-flash',
    contents=[
        'Describe this image in detail',
        types.Part.from_bytes(data=image_bytes, mime_type='image/jpeg'),
    ],
):
    print(chunk.text, end='')
```

## Use Cases

<CardGroup cols={2}>
  <Card title="Document Analysis" icon="file-pdf">
    Extract insights from PDFs, images of documents, and scanned files
  </Card>

  <Card title="Video Understanding" icon="video">
    Analyze video content, generate descriptions, and answer questions
  </Card>

  <Card title="Audio Transcription" icon="microphone">
    Transcribe and analyze audio content, podcasts, and meetings
  </Card>

  <Card title="Visual Q&A" icon="image">
    Answer questions about images, charts, and diagrams
  </Card>
</CardGroup>

## Best Practices

* Use `Part.from_uri` for large files or files already in cloud storage
* Use `Part.from_bytes` for small files (\< 20MB) from local filesystem
* Use the File API for files that need preprocessing (video, long audio, PDFs)
* Always specify the correct MIME type for your media
* Check file state (`PROCESSING`, `ACTIVE`) before using uploaded files
* Delete files after use to manage storage costs
* Combine multiple modalities when relevant to your use case
* For Gemini Developer API, use the File API for all large files
* For Vertex AI, you can use GCS URIs directly with `Part.from_uri`