Editing article

Title

Summary

Content

<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*4xCKDkXmAXkIXMbs.png" /></figure><h3>Abstract</h3>The Gemini API allows the generating of text from uploaded files using Google Apps Script. It expands the potential of various scripting languages for diverse applications.<h3>Introduction</h3>With the release of the LLM model Gemini as an API on Vertex AI and Google AI Studio, a world of possibilities has opened up. <a href="https://deepmind.google/technologies/gemini/#introduction">Ref</a> The Gemini API significantly expands the potential of various scripting languages and paves the way for diverse applications. Also, recently, Gemini 1.5 in AI Studio has been released. <a href="https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/#sundar-note">Ref</a> In the near future, Gemini 1.5 API will be also released soon.Recently, the files got to be able to be uploaded with Gemini API. <a href="https://ai.google.dev/api/rest/v1beta/files">Ref1</a> and <a href="https://ai.google.dev/api/rest/v1beta/media">Ref2</a> When this is used, the text is generated using the uploaded files. This report introduces the sample scripts for uploading the files and generating the texts using Google Apps Script.<h3>Usage</h3>In order to test this script, please do the following steps.<h3>1. Create an API key</h3>Please access <a href="https://makersuite.google.com/app/apikey">https://makersuite.google.com/app/apikey</a> and create your API key. At that time, please enable Generative Language API at the API console. This API key is used for this sample script.This official document can be also seen. <a href="https://ai.google.dev/">Ref</a>.<h3>2. Create a Google Apps Script project</h3>In this report, Google Apps Script is used. Of course, the method introducing this report can be also used in other languages.Please create a standalone Google Apps Script project. Of course, this script can be also used with the container-bound script.And, please open the script editor of the Google Apps Script project.<h3>3. Steps of script</h3>Here, it introduces the following 4 sample scripts.<ol><li>Upload a file with <a href="https://ai.google.dev/api/rest/v1beta/media">“Method: files.list”</a>.</li><li>Confirm the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/files">“Method: media.upload”</a>.</li><li>Generate content using the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/models/generateContent">“Method: models.generateContent”</a>.</li><li>Delete the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/files/delete">“Method: files.delete”</a>.</li></ol>Limitation of the uploaded file <a href="https://github.com/google-gemini/gemini-api-cookbook/blob/main/preview/file-api/File_API.ipynb">Ref</a><blockquote>Can only be used with model.generateContent or model.streamGenerateContent Automatic file deletion after 2 days Maximum 2GB per file, 20GB limit per project No downloads allowed</blockquote><h3>4. Script</h3><h3>Upload a file</h3>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/media/upload">Method: media.upload</a>. The sample script of Google Apps Script is as follows.Please set your API key and the file ID of the image file. Here, PNG image is used.<pre>function sample1() { const apiKey = &quot;###&quot;; // Please set your API key. const fileId = &quot;###&quot;; // Please set the file ID of the image file. Here, PNG image is used. const url = `https://generativelanguage.googleapis.com/upload/v1beta/files?uploadType=multipart&amp;key=${apiKey}`; const metadata = { file: { displayName: DriveApp.getFileById(fileId).getName() }, }; const payload = { metadata: Utilities.newBlob(JSON.stringify(metadata), &quot;application/json&quot;), file: UrlFetchApp.fetch( `https://drive.google.com/thumbnail?sz=w1000&amp;id=${fileId}`, { headers: { authorization: &quot;Bearer &quot; + ScriptApp.getOAuthToken() } } ).getBlob(), }; const options = { method: &quot;post&quot;, payload: payload, muteHttpExceptions: true, }; const res = UrlFetchApp.fetch(url, options).getContentText(); console.log(res); }</pre>When this script is run, the following value is returned.<pre>{ &quot;file&quot;: { &quot;name&quot;: &quot;files/###&quot;, &quot;displayName&quot;: &quot;###&quot;, &quot;mimeType&quot;: &quot;image/jpeg&quot;, &quot;sizeBytes&quot;: &quot;123456&quot;, &quot;createTime&quot;: &quot;2024-03-30T01:23:00.000000Z&quot;, &quot;updateTime&quot;: &quot;2024-03-30T01:23:00.000000Z&quot;, &quot;expirationTime&quot;: &quot;2024-04-01T01:23:00.000000Z&quot;, &quot;sha256Hash&quot;: &quot;###&quot;, &quot;uri&quot;: &quot;https://generativelanguage.googleapis.com/v1beta/files/###&quot; } }</pre>The values of mimeType and uri are used with generateContent.In this sample, I used uploadType=multipart because of the small size of the image file. If you want to upload a large file, I think that resumable upload can be also used.<h3>Get the file list</h3>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/files/list">Method: files.list</a>. The sample script of Google Apps Script is as follows.Please set your API key.<pre>function sample2() { const apiKey = &quot;###&quot;; // Please set your API key. const url = `https://generativelanguage.googleapis.com/v1beta/files?pageSize=100&amp;key=${apiKey}`; const res = UrlFetchApp.fetch(url); console.log(res.getContentText()); }</pre>When this script is run, the following value is returned.<pre>{ &quot;files&quot;: [ { &quot;name&quot;: &quot;files/###&quot;, &quot;displayName&quot;: &quot;###&quot;, &quot;mimeType&quot;: &quot;image/jpeg&quot;, &quot;sizeBytes&quot;: &quot;123456&quot;, &quot;createTime&quot;: &quot;2024-03-30T01:23:00.000000Z&quot;, &quot;updateTime&quot;: &quot;2024-03-30T01:23:00.000000Z&quot;, &quot;expirationTime&quot;: &quot;2024-04-01T01:23:00.000000Z&quot;, &quot;sha256Hash&quot;: &quot;###&quot;, &quot;uri&quot;: &quot;https://generativelanguage.googleapis.com/v1beta/files/###&quot; }, , , , ] }</pre>When the number of files is more than 100, please retrieve all files using pageToken.<h3>Generate content from the uploaded file</h3>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/models/generateContent">Method: models.generateContent</a>. The sample script of Google Apps Script is as follows.Please set your API key, the URI of the uploaded file, and the mimeType of the file.<pre>function sample3() { const apiKey = &quot;###&quot;; // Please set your API key. const fileUri = &quot;https://generativelanguage.googleapis.com/v1beta/files/###&quot;; // Please set your file uri of the uploaded file. const mimeType = &quot;image/jpeg&quot;; // Please set the mimeType of the uploaded file. const q = &quot;Describe the image and count apples in the image.&quot;; const model = &quot;models/gemini-1.5-pro-gf-fc&quot;; const baseUrl = `https://generativelanguage.googleapis.com/v1beta/${model}`; const payload = { contents: [{ parts: [{ text: q }, { fileData: { fileUri, mimeType } }] }], }; const options = { payload: JSON.stringify(payload), contentType: &quot;application/json&quot;, muteHttpExceptions: true, }; const res = UrlFetchApp.fetch( `${baseUrl}:generateContent?key=${apiKey}`, options ); console.log(res.getContentText()); }</pre>In this sample, the following image created by Gemini was uploaded as a sample file and was used.<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*2K3xjxp7uwYgCx7-.png" /></figure>There are 12 apples including 7 red apples, 4 green apples, and 1 yellow apple are shown in the image. This image was generated by Gemini.When this script is run, the following generated contents are returned.<ul><li>At the model models/gemini-1.0-pro-latest, Image input modality is not enabled for models/gemini-1.0-pro-latest was returned.</li><li>At the model models/gemini-1.0-pro-vision-latest, There are 10 apples in the image. Four red, five green, and one yellow. was returned.</li><li>At the model models/gemini-1.5-pro-latest, The image shows a group of apples on a wooden table. There are red, green, and yellow apples. There are 14 apples in total. was returned.</li><li>At the model models/gemini-1.5-pro-gf-fc, The image shows a group of apples on a wooden table. There are 13 apples in total. The apples are of different colors, including red, green, and yellow. The apples are arranged in a random pattern on the table. The light is coming from the left side of the image, and it is casting shadows on the apples and the table. was returned.</li></ul><h3>Delete the uploaded file</h3>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/files/delete">Method: files.delete</a>. The sample script of Google Apps Script is as follows.Please set your API key and the name of the uploaded file.<pre>function sample4() { const apiKey = &quot;###&quot;; // Please set your API key. const name = &quot;files/###&quot;; // Please set the name of the uploaded file. const url = `https://generativelanguage.googleapis.com/v1beta/${name}?key=${apiKey}`; const res = UrlFetchApp.fetch(url, { method: &quot;delete&quot; }); console.log(res.getContentText()); // {} }</pre>In this case, an empty object like {} is returned.In the current stage, the expiration time of the uploaded file is 2 days. So, the uploaded file is automatically deleted 2 days later.<h3>Summary</h3>In this report, we present sample scripts for using the Gemini API’s generateContent function with uploaded files. Our findings are as follows:<ul><li>Uploading files, retrieving file lists, and deleting files all functioned smoothly using an API key. Also, I heard that at Google APIs, both “snake_case” and “camelCase” within the request body. This was confirmed through testing.</li><li>For generating content from uploaded image files, Gemini 1.5 API models models/gemini-1.5-pro-latest and models/gemini-1.5-pro-gf-fc can be used for image analysis. However, accurately counting objects within the image might still be challenging.</li><li>Currently, uploading text, CSV, and PDF files results in an error message like “Request contains an invalid argument.” It appears that only image and movie files are supported at this stage. We anticipate this limitation to be addressed in a future update.</li></ul><h3>Note</h3><ul><li>The top illustration was created by <a href="https://gemini.google.com/">Gemini</a> with giving the abstract.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5777f1c902ab" width="1" height="1" alt=""><hr><a href="https://medium.com/google-cloud/generating-texts-using-files-uploaded-by-gemini-1-5-api-5777f1c902ab">Generating Texts using Files Uploaded by Gemini 1.5 API</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Author

Link

Published date

Image url

Feed url

Guid

Hidden blurb

--- !ruby/object:Feedjira::Parser::RSSEntry
title: Generating Texts using Files Uploaded by Gemini 1.5 API
url: https://medium.com/google-cloud/generating-texts-using-files-uploaded-by-gemini-1-5-api-5777f1c902ab?source=rss----e52cf94d98af---4
author: Kanshi Tanaike
categories:
- gemini
- google-apps-script
- generative-ai
- google-cloud-platform
- machine-learning
published: 2024-03-31 02:50:43.000000000 Z
entry_id: !ruby/object:Feedjira::Parser::GloballyUniqueIdentifier
 is_perma_link: 'false'
 guid: https://medium.com/p/5777f1c902ab
carlessian_info:
 news_filer_version: 2
 newspaper: Google Cloud - Medium
 macro_region: Blogs
rss_fields:
- title
- url
- author
- categories
- published
- entry_id
- content
content: '<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*4xCKDkXmAXkIXMbs.png"
 /></figure><h3>Abstract</h3>The Gemini API allows the generating of text from
 uploaded files using Google Apps Script. It expands the potential of various scripting
 languages for diverse applications.<h3>Introduction</h3>With the release
 of the LLM model Gemini as an API on Vertex AI and Google AI Studio, a world of
 possibilities has opened up. <a href="https://deepmind.google/technologies/gemini/#introduction">Ref</a>
 The Gemini API significantly expands the potential of various scripting languages
 and paves the way for diverse applications. Also, recently, Gemini 1.5 in AI Studio
 has been released. <a href="https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/#sundar-note">Ref</a>
 In the near future, Gemini 1.5 API will be also released soon.Recently, the
 files got to be able to be uploaded with Gemini API. <a href="https://ai.google.dev/api/rest/v1beta/files">Ref1</a>
 and <a href="https://ai.google.dev/api/rest/v1beta/media">Ref2</a> When this is
 used, the text is generated using the uploaded files. This report introduces the
 sample scripts for uploading the files and generating the texts using Google Apps Script.<h3>Usage</h3>In
 order to test this script, please do the following steps.<h3>1. Create an API key</h3>Please
 access <a href="https://makersuite.google.com/app/apikey">https://makersuite.google.com/app/apikey</a>
 and create your API key. At that time, please enable Generative Language API at
 the API console. This API key is used for this sample script.This official
 document can be also seen. <a href="https://ai.google.dev/">Ref</a>.<h3>2. Create
 a Google Apps Script project</h3>In this report, Google Apps Script is used.
 Of course, the method introducing this report can be also used in other languages.Please
 create a standalone Google Apps Script project. Of course, this script can be also
 used with the container-bound script.And, please open the script editor of
 the Google Apps Script project.<h3>3. Steps of script</h3>Here, it introduces
 the following 4 sample scripts.<ol><li>Upload a file with <a href="https://ai.google.dev/api/rest/v1beta/media">“Method:
 files.list”</a>.</li><li>Confirm the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/files">“Method:
 media.upload”</a>.</li><li>Generate content using the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/models/generateContent">“Method:
 models.generateContent”</a>.</li><li>Delete the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/files/delete">“Method:
 files.delete”</a>.</li></ol>Limitation of the uploaded file <a href="https://github.com/google-gemini/gemini-api-cookbook/blob/main/preview/file-api/File_API.ipynb">Ref</a><blockquote>Can
 only be used with model.generateContent or model.streamGenerateContent Automatic
 file deletion after 2 days Maximum 2GB per file, 20GB limit per project No downloads
 allowed</blockquote><h3>4. Script</h3><h3>Upload a file</h3>You can see
 the official document at <a href="https://ai.google.dev/api/rest/v1beta/media/upload">Method:
 media.upload</a>. The sample script of Google Apps Script is as follows.Please
 set your API key and the file ID of the image file. Here, PNG image is used.<pre>function
 sample1() { const apiKey = &quot;###&quot;; // Please set your API key. const
 fileId = &quot;###&quot;; // Please set the file ID of the image file. Here, PNG
 image is used. const url = `https://generativelanguage.googleapis.com/upload/v1beta/files?uploadType=multipart&amp;key=${apiKey}`; const
 metadata = { file: { displayName: DriveApp.getFileById(fileId).getName()
 }, }; const payload = { metadata: Utilities.newBlob(JSON.stringify(metadata),
 &quot;application/json&quot;), file: UrlFetchApp.fetch( `https://drive.google.com/thumbnail?sz=w1000&amp;id=${fileId}`, {
 headers: { authorization: &quot;Bearer &quot; + ScriptApp.getOAuthToken() } } ).getBlob(), }; const
 options = { method: &quot;post&quot;, payload: payload, muteHttpExceptions:
 true, }; const res = UrlFetchApp.fetch(url, options).getContentText(); console.log(res); }</pre>When
 this script is run, the following value is returned.<pre>{ &quot;file&quot;:
 { &quot;name&quot;: &quot;files/###&quot;, &quot;displayName&quot;:
 &quot;###&quot;, &quot;mimeType&quot;: &quot;image/jpeg&quot;, &quot;sizeBytes&quot;:
 &quot;123456&quot;, &quot;createTime&quot;: &quot;2024-03-30T01:23:00.000000Z&quot;, &quot;updateTime&quot;:
 &quot;2024-03-30T01:23:00.000000Z&quot;, &quot;expirationTime&quot;: &quot;2024-04-01T01:23:00.000000Z&quot;, &quot;sha256Hash&quot;:
 &quot;###&quot;, &quot;uri&quot;: &quot;https://generativelanguage.googleapis.com/v1beta/files/###&quot; } }</pre>The
 values of mimeType and uri are used with generateContent.In this sample,
 I used uploadType=multipart because of the small size of the image file. If you
 want to upload a large file, I think that resumable upload can be also used.<h3>Get
 the file list</h3>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/files/list">Method:
 files.list</a>. The sample script of Google Apps Script is as follows.Please
 set your API key.<pre>function sample2() { const apiKey = &quot;###&quot;;
 // Please set your API key. const url = `https://generativelanguage.googleapis.com/v1beta/files?pageSize=100&amp;key=${apiKey}`; const
 res = UrlFetchApp.fetch(url); console.log(res.getContentText()); }</pre>When
 this script is run, the following value is returned.<pre>{ &quot;files&quot;:
 [ { &quot;name&quot;: &quot;files/###&quot;, &quot;displayName&quot;:
 &quot;###&quot;, &quot;mimeType&quot;: &quot;image/jpeg&quot;, &quot;sizeBytes&quot;:
 &quot;123456&quot;, &quot;createTime&quot;: &quot;2024-03-30T01:23:00.000000Z&quot;, &quot;updateTime&quot;:
 &quot;2024-03-30T01:23:00.000000Z&quot;, &quot;expirationTime&quot;: &quot;2024-04-01T01:23:00.000000Z&quot;, &quot;sha256Hash&quot;:
 &quot;###&quot;, &quot;uri&quot;: &quot;https://generativelanguage.googleapis.com/v1beta/files/###&quot; }, , , , ] }</pre>When
 the number of files is more than 100, please retrieve all files using pageToken.<h3>Generate
 content from the uploaded file</h3>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/models/generateContent">Method:
 models.generateContent</a>. The sample script of Google Apps Script is as follows.Please
 set your API key, the URI of the uploaded file, and the mimeType of the file.<pre>function
 sample3() { const apiKey = &quot;###&quot;; // Please set your API key. const
 fileUri = &quot;https://generativelanguage.googleapis.com/v1beta/files/###&quot;;
 // Please set your file uri of the uploaded file. const mimeType = &quot;image/jpeg&quot;;
 // Please set the mimeType of the uploaded file. const q = &quot;Describe
 the image and count apples in the image.&quot;; const model = &quot;models/gemini-1.5-pro-gf-fc&quot;; const
 baseUrl = `https://generativelanguage.googleapis.com/v1beta/${model}`; const
 payload = { contents: [{ parts: [{ text: q }, { fileData: { fileUri, mimeType
 } }] }], }; const options = { payload: JSON.stringify(payload), contentType:
 &quot;application/json&quot;, muteHttpExceptions: true, }; const
 res = UrlFetchApp.fetch( `${baseUrl}:generateContent?key=${apiKey}`, options ); console.log(res.getContentText()); }</pre>In
 this sample, the following image created by Gemini was uploaded as a sample file
 and was used.<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*2K3xjxp7uwYgCx7-.png"
 /></figure>There are 12 apples including 7 red apples, 4 green apples, and 1
 yellow apple are shown in the image. This image was generated by Gemini.When
 this script is run, the following generated contents are returned.<ul><li>At
 the model models/gemini-1.0-pro-latest, Image input modality is not enabled for
 models/gemini-1.0-pro-latest was returned.</li><li>At the model models/gemini-1.0-pro-vision-latest,
 There are 10 apples in the image. Four red, five green, and one yellow. was returned.</li><li>At
 the model models/gemini-1.5-pro-latest, The image shows a group of apples on a wooden
 table. There are red, green, and yellow apples. There are 14 apples in total. was
 returned.</li><li>At the model models/gemini-1.5-pro-gf-fc, The image shows a group
 of apples on a wooden table. There are 13 apples in total. The apples are of different
 colors, including red, green, and yellow. The apples are arranged in a random pattern
 on the table. The light is coming from the left side of the image, and it is casting
 shadows on the apples and the table. was returned.</li></ul><h3>Delete the uploaded file</h3>You
 can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/files/delete">Method:
 files.delete</a>. The sample script of Google Apps Script is as follows.Please
 set your API key and the name of the uploaded file.<pre>function sample4() { const
 apiKey = &quot;###&quot;; // Please set your API key. const name = &quot;files/###&quot;;
 // Please set the name of the uploaded file. const url = `https://generativelanguage.googleapis.com/v1beta/${name}?key=${apiKey}`; const
 res = UrlFetchApp.fetch(url, { method: &quot;delete&quot; }); console.log(res.getContentText());
 // {} }</pre>In this case, an empty object like {} is returned.In the
 current stage, the expiration time of the uploaded file is 2 days. So, the uploaded
 file is automatically deleted 2 days later.<h3>Summary</h3>In this report,
 we present sample scripts for using the Gemini API’s generateContent function with
 uploaded files. Our findings are as follows:<ul><li>Uploading files, retrieving
 file lists, and deleting files all functioned smoothly using an API key. Also, I
 heard that at Google APIs, both “snake_case” and “camelCase” within the request
 body. This was confirmed through testing.</li><li>For generating content from uploaded
 image files, Gemini 1.5 API models models/gemini-1.5-pro-latest and models/gemini-1.5-pro-gf-fc
 can be used for image analysis. However, accurately counting objects within the
 image might still be challenging.</li><li>Currently, uploading text, CSV, and PDF
 files results in an error message like “Request contains an invalid argument.” It
 appears that only image and movie files are supported at this stage. We anticipate
 this limitation to be addressed in a future update.</li></ul><h3>Note</h3><ul><li>The
 top illustration was created by <a href="https://gemini.google.com/">Gemini</a>
 with giving the abstract.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5777f1c902ab"
 width="1" height="1" alt=""><hr><a href="https://medium.com/google-cloud/generating-texts-using-files-uploaded-by-gemini-1-5-api-5777f1c902ab">Generating
 Texts using Files Uploaded by Gemini 1.5 API</a> was originally published in <a
 href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where
 people are continuing the conversation by highlighting and responding to this story.'

Language

Active

Ricc internal notes

Ricc source

Show this article Back to articles