diff --git a/.gemini/styleguide.md b/.gemini/styleguide.md index d578751f..ba6ee372 100644 --- a/.gemini/styleguide.md +++ b/.gemini/styleguide.md @@ -70,6 +70,7 @@ sealed class ScreenState { } ``` +* **Strongly recommended:** When a view model or the business logic is modified in the code of a sample, verify that these changes are properly reflected in the README.md of this sample including the versions of the model used. * **Recommended:** Do not use `AndroidViewModel`. Use the `ViewModel` class. Avoid using the `Application` class in ViewModels; move the dependency to the UI or data layer. * **Recommended:** Don't use `LiveData`, use state flow instead. * **Recommended:** Expose a UI state. Use a single `uiState` property (a `StateFlow`) for data exposure. Multiple properties can be used for unrelated data. Use `stateIn` with `WhileSubscribed(5000)` for data streams. For simpler cases, use a `MutableStateFlow` exposed as an immutable `StateFlow`. Consider using a data class or sealed class for the `UiState`. diff --git a/samples/gemini-chatbot/README.md b/samples/gemini-chatbot/README.md index 9b207814..bc430978 100644 --- a/samples/gemini-chatbot/README.md +++ b/samples/gemini-chatbot/README.md @@ -31,4 +31,4 @@ fun sendMessage(message: String) { } ``` -Read more about [getting started with Gemini](https://developer.android.com/ai/gemini/get-started) in the Android Documentation. +Read more about the [Gemini API](https://developer.android.com/ai/gemini) in the Android Documentation. diff --git a/samples/gemini-chatbot/src/main/java/com/android/ai/samples/geminichatbot/GeminiChatbotViewModel.kt b/samples/gemini-chatbot/src/main/java/com/android/ai/samples/geminichatbot/GeminiChatbotViewModel.kt index 4f422422..706ac3c7 100644 --- a/samples/gemini-chatbot/src/main/java/com/android/ai/samples/geminichatbot/GeminiChatbotViewModel.kt +++ b/samples/gemini-chatbot/src/main/java/com/android/ai/samples/geminichatbot/GeminiChatbotViewModel.kt @@ -50,7 +50,7 @@ class GeminiChatbotViewModel @Inject constructor() : ViewModel() { private val generativeModel by lazy { Firebase.ai(backend = GenerativeBackend.googleAI()).generativeModel( - "gemini-2.5-flash", + "gemini-2.0-flash", generationConfig = generationConfig { temperature = 0.9f topK = 32 diff --git a/samples/gemini-image-chat/README.md b/samples/gemini-image-chat/README.md index 1e5af4ff..b268e289 100644 --- a/samples/gemini-image-chat/README.md +++ b/samples/gemini-image-chat/README.md @@ -26,4 +26,4 @@ val content = content { val response = chat.sendMessage(content) ``` -Read more about [image generation with Gemini](https://developer.android.com/ai/gemini/developer-api#generate-images) in the Android Documentation. \ No newline at end of file +Read more about [image generation with Gemini](https://developer.android.com/ai/gemini/developer-api#generate-images) in the Android Documentation. diff --git a/samples/gemini-live-todo/README.md b/samples/gemini-live-todo/README.md new file mode 100644 index 00000000..e43de3f7 --- /dev/null +++ b/samples/gemini-live-todo/README.md @@ -0,0 +1,39 @@ +# Gemini Live Todo Sample + +This sample is part of the [AI Sample Catalog](../../). To build and run this sample, you should clone the entire repository. + +## Description + +This sample demonstrates how to use the Gemini Live API for real-time, voice-based interactions in a simple ToDo application. Users can add, remove, and update tasks by speaking to the app, showcasing a hands-free, conversational user experience powered by the Gemini API. + +
+Gemini Live Todo in action +
+ +## How it works + +The application uses the Firebase AI SDK (see [How to run](../../#how-to-run)) for Android to interact with the `gemini-2.0-flash-live-preview-04-09` model. The core logic is in the [`TodoScreenViewModel.kt`](./src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreenViewModel.kt) file. A `liveModel` is initialized with a set of function declarations (`addTodo`, `removeTodo`, `toggleTodoStatus`, `getTodoList`) that allow the model to interact with the ToDo list. When the user starts a voice conversation, the model processes the spoken commands and executes the corresponding functions to manage the tasks. + +Here is the key snippet of code that initializes the model and connects to a live session: + +```kotlin +val generativeModel = Firebase.ai(backend = GenerativeBackend.vertexAI()).liveModel( + "gemini-2.0-flash-live-preview-04-09", + generationConfig = liveGenerationConfig, + systemInstruction = systemInstruction, + tools = listOf( + Tool.functionDeclarations( + listOf(getTodoList, addTodo, removeTodo, toggleTodoStatus), + ), + ), +) + +try { + session = generativeModel.connect() +} catch (e: Exception) { + Log.e(TAG, "Error connecting to the model", e) + liveSessionState.value = LiveSessionState.Error +} +``` + +Read more about the [Gemini Live API](https://developer.android.com/ai/gemini/live) in the Android Documentation. diff --git a/samples/gemini-multimodal/README.md b/samples/gemini-multimodal/README.md new file mode 100644 index 00000000..db235854 --- /dev/null +++ b/samples/gemini-multimodal/README.md @@ -0,0 +1,53 @@ +# Gemini Multimodal Sample + +This sample is part of the [AI Sample Catalog](../../). To build and run this sample, you should clone the entire repository. + +## Description + +This sample demonstrates a multimodal (image and text) prompt, using the `Gemini 2.5 Flash` model. Users can select an image and provide a text prompt, and the generative model will respond based on both inputs. This showcases how to build a simple, yet powerful, multimodal AI with the Gemini API. + +
+Gemini Multimodal in action +
+ +## How it works + +The application uses the Firebase AI SDK (see [How to run](../../#how-to-run)) for Android to interact with the `gemini-2.5-flash` model. The core logic is in the [`GeminiDataSource.kt`](./src/main/java/com/android/ai/samples/geminimultimodal/data/GeminiDataSource.kt) file. A `generativeModel` is initialized, and then a `chat` session is started from it. When a user provides an image and a text prompt, they are combined into a multimodal prompt and sent to the model, which then generates a text response. + +Here is the key snippet of code that initializes the generative model: + +```kotlin +private val generativeModel by lazy { + Firebase.ai(backend = GenerativeBackend.googleAI()).generativeModel( + "gemini-2.5-flash", + generationConfig = generationConfig { + temperature = 0.9f + topK = 32 + topP = 1f + maxOutputTokens = 4096 + }, + safetySettings = listOf( + SafetySetting(HarmCategory.HARASSMENT, HarmBlockThreshold.MEDIUM_AND_ABOVE), + SafetySetting(HarmCategory.HATE_SPEECH, HarmBlockThreshold.MEDIUM_AND_ABOVE), + SafetySetting(HarmCategory.SEXUALLY_EXPLICIT, HarmBlockThreshold.MEDIUM_AND_ABOVE), + SafetySetting(HarmCategory.DANGEROUS_CONTENT, HarmBlockThreshold.MEDIUM_AND_ABOVE), + ), + ) +} +``` + +Here is the key snippet of code that calls the [`generateText`](./src/main/java/com/android/ai/samples/geminimultimodal/data/GeminiDataSource.kt) function: + +```kotlin +suspend fun generateText(bitmap: Bitmap, prompt: String): String { + val multimodalPrompt = content { + image(bitmap) + text(prompt) + } + val result = generativeModel.generateContent(multimodalPrompt) + return result.text ?: "" +} +``` + +Read more about [the Gemini API](https://developer.android.com/ai/gemini) in the Android Documentation. + diff --git a/samples/gemini-video-metadata-creation/README.md b/samples/gemini-video-metadata-creation/README.md new file mode 100644 index 00000000..128077d5 --- /dev/null +++ b/samples/gemini-video-metadata-creation/README.md @@ -0,0 +1,50 @@ +# Gemini Video Metadata Creation Sample + +This sample is part of the [AI Sample Catalog](../../). To build and run this sample, you should clone the entire repository. + +## Description + +This sample demonstrates how to generate various types of video metadata (description, hashtags, chapters, account tags, links, and thumbnails) using the `Gemini 2.5 Flash` model. Users can select a video, and the generative model will analyze its content to provide relevant metadata, showcasing how to enrich video content with AI-powered insights. + +
+Gemini Video Metadata Creation in action +
+ +## How it works + +The application uses the Firebase AI SDK (see [How to run](../../#how-to-run)) for Android to interact with the `gemini-2.5-flash` model. The core logic involves several functions (e.g., [`generateDescription`](./src/main/java/com/android/ai/samples/geminivideometadatacreation/GenerateDescription.kt), `generateHashtags`, `generateChapters`, `generateAccountTags`, `generateLinks`, `generateThumbnails`) that send video content to the Gemini API for analysis. The model processes the video and returns structured metadata based on the specific prompt. + +Here is a key snippet of code that generates a video description: + +```kotlin +suspend fun generateDescription(videoUri: Uri): @Composable () -> Unit { + val response = Firebase.ai(backend = GenerativeBackend.vertexAI()) + .generativeModel(modelName = "gemini-2.5-flash") + .generateContent( + content { + fileData(videoUri.toString(), "video/mp4") + text( + """ + Provide a compelling and concise description for this video in less than 100 words. + Don't assume if you don't know. + The description should be engaging and accurately reflect the video's content. + You should output your responses in HTML format. Use styling sparingly. You can use the following tags: + * Bold: + * Italic: + * Underline: + * Bullet points:
    ,
  • + """.trimIndent(), + ) + }, + ) + + val responseText = response.text + return if (responseText != null) { + { DescriptionUi(responseText) } + } else { + { ErrorUi(response.promptFeedback?.blockReasonMessage) } + } +} +``` + +Read more about [the Gemini API](https://developer.android.com/ai/gemini) in the Android Documentation. diff --git a/samples/gemini-video-summarization/README.md b/samples/gemini-video-summarization/README.md new file mode 100644 index 00000000..792f5d44 --- /dev/null +++ b/samples/gemini-video-summarization/README.md @@ -0,0 +1,34 @@ +# Gemini Video Summarization Sample + +This sample is part of the [AI Sample Catalog](../../). To build and run this sample, you should clone the entire repository. + +## Description + +This sample demonstrates how to generate a text summary of a video using the `Gemini 2.5 Flash` model. Users can select a video, and the generative model will analyze its content to provide a concise summary, showcasing how to extract key information from video content with the Gemini API. + +
    +Gemini Video Summarization in action +
    + +## How it works + +The application uses the Firebase AI SDK (see [How to run](../../#how-to-run)) for Android to interact with the `gemini-2.5-flash` model. The core logic is in the [`VideoSummarizationViewModel.kt`](./src/main/java/com/android/ai/samples/geminivideosummary/viewmodel/VideoSummarizationViewModel.kt) file. A `generativeModel` is initialized. When a user requests a summary, the video content and a text prompt are sent to the model, which then generates a text summary. + +Here is the key snippet of code that calls the generative model: + +```kotlin +val generativeModel = + Firebase.ai(backend = GenerativeBackend.vertexAI()) + .generativeModel("gemini-2.5-flash") + +val requestContent = content { + fileData(videoSource.toString(), "video/mp4") + text(promptData) +} +val outputStringBuilder = StringBuilder() +generativeModel.generateContentStream(requestContent).collect { response -> + outputStringBuilder.append(response.text) +} +``` + +Read more about [getting started with Gemini](https://developer.android.com/ai/gemini) in the Android Documentation. diff --git a/samples/genai-image-description/README.md b/samples/genai-image-description/README.md new file mode 100644 index 00000000..83634b3b --- /dev/null +++ b/samples/genai-image-description/README.md @@ -0,0 +1,39 @@ +# Image Description with Nano Sample + +This sample is part of the [AI Sample Catalog](../../). To build and run this sample, you should clone the entire repository. + +## Description + +This sample demonstrates how to generate short descriptions of images on-device using the GenAI API powered by Gemini Nano. Users can select an image, and the model will generate a descriptive text, showcasing the power of on-device multimodal AI. + +
    +Image Description with Nano in action +
    + +## How it works + +The application uses the ML Kit GenAI Image Description API to interact with the on-device Gemini Nano model. The core logic is in the [`GenAIImageDescriptionViewModel.kt`](https://github.com/android/ai-samples/blob/main/samples/genai-image-description/src/main/java/com/android/ai/samples/genai_image_description/GenAIImageDescriptionViewModel.kt) file. An `ImageDescriber` client is initialized. When a user provides an image, it's converted to a bitmap and sent to the `runInference` method, which streams back the generated description. + +Here is the key snippet of code that calls the generative model: + +```kotlin +private var imageDescriber: ImageDescriber = ImageDescription.getClient( + ImageDescriberOptions.builder(context).build(), +) +//... + +private suspend fun generateImageDescription(imageUri: Uri) { + _uiState.value = GenAIImageDescriptionUiState.Generating("") + val bitmap = MediaStore.Images.Media.getBitmap(context.contentResolver, imageUri) + val request = ImageDescriptionRequest.builder(bitmap).build() + + imageDescriber.runInference(request) { newText -> + _uiState.update { + (it as? GenAIImageDescriptionUiState.Generating)?.copy(partialOutput = it.partialOutput + newText) ?: it + } + }.await() + // ... +} +``` + +Read more about [Gemini Nano](https://developer.android.com/ai/gemini-nano/ml-kit-genai) in the Android Documentation. diff --git a/samples/genai-summarization/README.md b/samples/genai-summarization/README.md new file mode 100644 index 00000000..644509d2 --- /dev/null +++ b/samples/genai-summarization/README.md @@ -0,0 +1,38 @@ +# Summarization with Nano Sample + +This sample is part of the [AI Sample Catalog](../../). To build and run this sample, you should clone the entire repository. + +## Description + +This sample demonstrates how to summarize articles and conversations on-device using the GenAI API powered by Gemini Nano. Users can input text, and the model will generate a concise summary, showcasing the power of on-device text processing with AI. + +
    +Summarization with Nano in action +
    + +## How it works + +The application uses the ML Kit GenAI Summarization API to interact with the on-device Gemini Nano model. The core logic is in the `GenAISummarizationViewModel.kt` file. A `Summarizer` client is initialized. When a user provides text, it's passed to the `runInference` method, which streams back the generated summary. + +Here is the key snippet of code that calls the generative model from [`GenAISummarizationViewModel.kt`](./src/main/java/com/android/ai/samples/genai_summarization/GenAISummarizationViewModel.kt): + +```kotlin +private suspend fun generateSummarization(summarizer: Summarizer, textToSummarize: String) { + _uiState.value = GenAISummarizationUiState.Generating("") + val summarizationRequest = SummarizationRequest.builder(textToSummarize).build() + + try { + // Instead of using await() here, alternatively you can attach a FutureCallback + summarizer.runInference(summarizationRequest) { newText -> + (_uiState.value as? GenAISummarizationUiState.Generating)?.let { generatingState -> + _uiState.value = generatingState.copy(generatedOutput = generatingState.generatedOutput + newText) + } + }.await() + } catch (genAiException: GenAiException) { + // ... + } + // ... +} +``` + +Read more about [Gemini Nano](https://developer.android.com/ai/gemini-nano/ml-kit-genai) in the Android Documentation. diff --git a/samples/genai-writing-assistance/README.md b/samples/genai-writing-assistance/README.md new file mode 100644 index 00000000..96472bd4 --- /dev/null +++ b/samples/genai-writing-assistance/README.md @@ -0,0 +1,31 @@ +# Writing Assistance with Nano Sample + +This sample is part of the [AI Sample Catalog](../../). To build and run this sample, you should clone the entire repository. + +## Description + +This sample demonstrates how to proofread and rewrite short content on-device using the ML Kit GenAI APIs powered by Gemini Nano. Users can input text and choose to either proofread it for grammar and spelling errors or rewrite it in various styles, showcasing on-device text manipulation with AI. + +
    +Writing Assistance with Nano in action +
    + +## How it works + +The application uses the ML Kit GenAI Proofreading and Rewriting APIs to interact with the on-device Gemini Nano model. The core logic is in the [`GenAIWritingAssistanceViewModel.kt`](https://github.com/android/ai-samples/blob/main/samples/genai-writing-assistance/src/main/java/com/android/ai/samples/genai_writing_assistance/GenAIWritingAssistanceViewModel.kt) file. `Proofreader` and `Rewriter` clients are initialized. When a user provides text, it's passed to either the `runProofreadingInference` or `runRewritingInference` method, which then returns the polished text. + +Here is the key snippet of code that runs the proofreading inference from [`GenAIWritingAssistanceViewModel.kt`](.src/main/java/com/android/ai/samples/genai_writing_assistance/GenAIWritingAssistanceViewModel.kt): + +```kotlin +private suspend fun runProofreadingInference(textToProofread: String) { + val proofreadRequest = ProofreadingRequest.builder(textToProofread).build() + // More than 1 result may be generated. Results are returned in descending order of + // quality of confidence. Here we use the first result which has the highest quality + // of confidence. + _uiState.value = GenAIWritingAssistanceUiState.Generating + val results = proofreader.runInference(proofreadRequest).await() + _uiState.value = GenAIWritingAssistanceUiState.Success(results.results[0].text) +} +``` + +Read more about [Gemini Nano](https://developer.android.com/ai/gemini-nano/ml-kit-genai) in the Android Documentation. diff --git a/samples/imagen-editing/README.md b/samples/imagen-editing/README.md new file mode 100644 index 00000000..0920e5da --- /dev/null +++ b/samples/imagen-editing/README.md @@ -0,0 +1,37 @@ +# Imagen Image Editing Sample + +This sample is part of the [AI Sample Catalog](../../). To build and run this sample, you should clone the entire repository. + +## Description + +This sample demonstrates how to edit images using the Imagen editing model. Users can generate an image, then draw a mask on it and provide a text prompt to inpaint (fill in) the masked area, showcasing advanced image manipulation capabilities with the Imagen API. + +
    +Imagen Image Editing in action +
    + +## How it works + +The application uses the Firebase AI SDK (see [How to run](../../#how-to-run)) for Android to interact with the `imagen-4.0-ultra-generate-001` and `imagen-3.0-capability-001` models. The core logic is in the [`ImagenEditingDataSource.kt`](https://github.com/android/ai-samples/blob/main/samples/imagen-editing/src/main/java/com/android/ai/samples/imagenediting/data/ImagenEditingDataSource.kt) file. It first generates a base image using the generation model. Then, for editing, it takes the source image, a user-drawn mask, and a text prompt, and sends them to the editing model's `editImage` method to perform inpainting. + +Here is the key snippet of code that performs inpainting from [`ImagenEditingDataSource.kt`](./src/main/java/com/android/ai/samples/imagenediting/data/ImagenEditingDataSource.kt): + +```kotlin +@OptIn(PublicPreviewAPI::class) +suspend fun inpaintImageWithMask(sourceImage: Bitmap, maskImage: Bitmap, prompt: String, editSteps: Int = DEFAULT_EDIT_STEPS): Bitmap { + val imageResponse = editingModel.editImage( + referenceImages = listOf( + ImagenRawImage(sourceImage.toImagenInlineImage()), + ImagenRawMask(maskImage.toImagenInlineImage()), + ), + prompt = prompt, + config = ImagenEditingConfig( + editMode = ImagenEditMode.INPAINT_INSERTION, + editSteps = editSteps, + ), + ) + return imageResponse.images.first().asBitmap() +} +``` + +Read more about [Imagen](https://developer.android.com/ai/imagen) in the Android Documentation. diff --git a/samples/imagen/README.md b/samples/imagen/README.md new file mode 100644 index 00000000..36d17bfd --- /dev/null +++ b/samples/imagen/README.md @@ -0,0 +1,30 @@ +# Imagen Image Generation Sample + +This sample is part of the [AI Sample Catalog](../../). To build and run this sample, you should clone the entire repository. + +## Description + +This sample demonstrates how to generate images from text prompts using the Imagen model. Users can input a text description, and the generative model will create an image based on that prompt, showcasing the power of text-to-image generation with the Imagen API. + +
    +Imagen Image Generation in action +
    + +## How it works + +The application uses the Firebase AI SDK (see [How to run](../../#how-to-run)) for Android to interact with the `imagen-4.0-generate-preview-06-06` model. The core logic is in the [`ImagenDataSource.kt`](./src/main/java/com/android/ai/samples/imagen/data/ImagenDataSource.kt) file. An `imagenModel` is initialized with specific generation configurations (e.g., number of images, aspect ratio, image format). When a user provides a text prompt, it's passed to the `generateImages` method, which returns the generated image as a bitmap. + +Here is the key snippet of code that calls the generative model from [`ImagenDataSource.kt`](./src/main/java/com/android/ai/samples/imagen/data/ImagenDataSource.kt): + +```kotlin +@OptIn(PublicPreviewAPI::class) +suspend fun generateImage(prompt: String): Bitmap { + val imageResponse = imagenModel.generateImages( + prompt = prompt, + ) + val image = imageResponse.images.first() + return image.asBitmap() +} +``` + +Read more about [Imagen](https://developer.android.com/ai/imagen) in the Android Documentation. diff --git a/samples/magic-selfie/README.md b/samples/magic-selfie/README.md new file mode 100644 index 00000000..fcd2bf34 --- /dev/null +++ b/samples/magic-selfie/README.md @@ -0,0 +1,36 @@ +# Magic Selfie Sample + +This sample is part of the [AI Sample Catalog](../../). To build and run this sample, you should clone the entire repository. + +## Description + +This sample demonstrates how to create a "magic selfie" by replacing the background of a user's photo with a generated image. It uses the ML Kit Subject Segmentation API to isolate the user from their original background and the Imagen API to generate a new background from a text prompt. + +
    +Magic Selfie in action +
    + +## How it works + +The application uses two main components. First, the ML Kit Subject Segmentation API processes the user's selfie to create a bitmap containing only the foreground (the person). Second, the Firebase AI SDK (see [How to run](../../#how-to-run)) for Android interacts with the `imagen-4.0-generate-preview-06-06` model to generate a new background image from a user-provided text prompt. Finally, the application combines the foreground bitmap with the newly generated background to create the final magic selfie. The core logic for this process is in the [`MagicSelfieViewModel.kt`](./src/main/java/com/android/ai/samples/magicselfie/ui/MagicSelfieViewModel.kt) and [`MagicSelfieRepository.kt`](./src/main/java/com/android/ai/samples/magicselfie/data/MagicSelfieRepository.kt) files. + +Here is the key snippet of code that orchestrates the magic selfie creation from [`MagicSelfieViewModel.kt`](./src/main/java/com/android/ai/samples/magicselfie/ui/MagicSelfieViewModel.kt): + +```kotlin +fun createMagicSelfie(bitmap: Bitmap, prompt: String) { + viewModelScope.launch { + try { + _uiState.value = MagicSelfieUiState.RemovingBackground + val foregroundBitmap = magicSelfieRepository.generateForegroundBitmap(bitmap) + _uiState.value = MagicSelfieUiState.GeneratingBackground + val backgroundBitmap = magicSelfieRepository.generateBackground(prompt) + val resultBitmap = magicSelfieRepository.combineBitmaps(foregroundBitmap, backgroundBitmap) + _uiState.value = MagicSelfieUiState.Success(resultBitmap) + } catch (e: Exception) { + _uiState.value = MagicSelfieUiState.Error(e.message) + } + } +} +``` + +Read more about the [Gemini API](https://developer.android.com/ai/gemini) in the Android Documentation.