Editing article

Title

Summary

Content

<figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*cwsJqTsvzudgNKgCZqg8ow.png" /><figcaption>Navigate better with deterministic Generative AI &amp; Reverse Geocoding API</figcaption></figure>Generative AI models are remarkable at understanding and responding to natural language. But what if you need precise, predictable outputs for critical tasks like address standardization? Traditional generative models can sometimes provide different responses at different times for the same prompts, potentially leading to inconsistencies. That’s where Gemini’s Function Calling capability shines, allowing you to deterministically control elements of the AI’s response.In this blog, we’ll illustrate this concept with the address completion and standardization use case. For this we will be building a Java Cloud Function that:<ol><li>Takes latitude and longitude coordinates</li><li>Calls the Google Maps Geocoding API to get corresponding addresses</li><li>Uses Gemini 1.o Pro Function Calling feature to deterministically standardize and summarize those addresses in a specific format that we need</li></ol>Let’s dive in!<h3>Gemini Function Calling</h3>Gemini Function Calling stands out in the Generative AI era because it lets you blend the flexibility of generative language models with the precision of traditional programming. Here’s how it works:Defining Functions: You describe functions as if you were explaining them to a coworker. These descriptions include:<ol><li>The function’s name (e.g., “getAddress”)</li><li>The parameters it expects (e.g., “latlng” as a string)</li><li>The type of data it returns (e.g., a list of address strings)</li></ol>“Tools” for Gemini: You package function descriptions in the form of API specification into “Tools”. Think of a tool as a specialized toolbox Gemini can use to understand the functionality of the API.Gemini as API Orchestrator: When you send a prompt to Gemini, it can analyze your request and recognize where it can use the tools you’ve provided. Gemini then acts as a smart orchestrator:<ol><li>Generates API Parameters: It produces the necessary parameters to call your defined functions.</li><li>Calls External APIs: Gemini doesn’t call the API on your behalf. You call the API based on the parameters and signature that Gemini function calling has generated for you.</li><li>Processes Results: Gemini feeds the results from your API calls back into its generation, letting it incorporate structured information into its final response which you can process in any way you desire for your application.</li></ol><h3>High Level Flow Diagram</h3>This diagram represents the flow of data and steps involved in the implementation. Please note that the owner for the respective step is mentioned in the text underneath.<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2-HYu2sGPsNwe46UdJuVdA.png" /><figcaption>High level flow diagram of the implementation</figcaption></figure><h3>Industry Use cases and why it matters</h3>Below are some of the examples and industry specific use cases for function calling with Gemini.<ol><li>Geocoding (Our Use Case): You’ve seen how to define a “getAddress” function and use it within the context of address standardization.</li><li>Data Validation: Imagine a function called “validateEmail” that takes a string and checks it against an email validation service. Gemini can help you formulate the parameters string so you can call the email validation API to ensure the quality of generated responses. Remember, Gemini does not make the API call for you.</li><li>Fact-Checking: Define a “lookupFact” function. Gemini could use this to consult a trusted knowledge base, making its responses more reliable within specific domains.</li></ol>Why Function Calling MattersBridging Two Worlds: LLMs can’t know everything (especially private information, customer details, news that are more recent than its knowledge cut-off date), and by integrating function calling, it’s able to extend its knowledge and capabilities to bridge the gap between open-ended generative AI and the deterministic execution of traditional code.Controlled Creativity: Function Calling injects structured, predictable elements into Gemini’s creative process, ideal for critical use cases or maintaining consistency.Building Agent: Chain together multiple function calls and Gemini processing steps, enabling multi-stage generative AI workflows.<h3>Create the Java Cloud Function</h3>This is the Gen 2 Cloud Function implementation where we will invoke the Gemini model to orchestrate the input for function Calling, invoke the API and then process the response in another Gemini call and deploy it to a REST endpoint.<h3>Setup</h3><ol><li>In the <a href="https://console.cloud.google.com/">Google Cloud Console</a>, on the project selector page, select or create a Google Cloud <a href="https://cloud.google.com/resource-manager/docs/creating-managing-projects">project</a>.</li><li>Make sure that billing is enabled for your Cloud project. Learn how to <a href="https://cloud.google.com/billing/docs/how-to/verify-billing-enabled">check if billing is enabled on a project</a>.</li><li>You will use Cloud Shell, a command-line environment running in Google Cloud that comes preloaded with bq. From the Cloud Console, click Activate Cloud Shell on the top right corner.</li><li>Just for support with building and delivering the app, let’s enable Duet AI. Navigate to <a href="https://console.cloud.google.com/marketplace/product/google/cloudaicompanion.googleapis.com">Duet AI Marketplace</a> to enable the API. You can also run the following command in the Cloud Shell terminal:</li></ol><pre>gcloud services enable cloudaicompanion.googleapis.com –project PROJECT_ID</pre>5. Enable necessary APIs for this implementation if you haven’t already.Alternative to the gcloud command is through the console using this <a href="https://console.cloud.google.com/apis/enableflow?apiid=bigquery.googleapis.com,bigqueryconnection.googleapis.com,aiplatform.googleapis.com">link</a>.<h3>Java Cloud Function</h3><ol><li>Open Cloud Shell Terminal and navigate to the root directory or your default workspace path.</li><li>Click the Cloud Code Sign In icon from the bottom left corner of the status bar and select the active Google Cloud Project that you want to create the Cloud Functions in.</li><li>Click the same icon again and this time select the option to create a new application.</li><li>In the Create New Application pop-up, select Cloud Functions application:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*n53iiiF7ZuyG9J4U" /><figcaption>Create New Application page 1</figcaption></figure>5. Select Java: Hello World option from the next pop-up:<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jRH5u3oXSLqx-a_k" /><figcaption>Create New Application page 2</figcaption></figure>6. Provide a name for the project in the project path. In this case, “GeminiFunctionCalling”.7. You should see the project structure opened up in a new Cloud Shell Editor view:<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*z7PMp8qsGOUik2j1" /><figcaption>Cloud Shell Editor showing the project structure with pom.xml open</figcaption></figure>8. Now go ahead and add the necessary dependencies within the &lt;dependencies&gt;… &lt;/dependencies&gt; tag in the pom.xml file. You can access the entire <a href="https://github.com/AbiramiSukumaran/GeminiFunctionCalling/blob/main/pom.xml">pom.xml</a> from this project’s github <a href="https://github.com/AbiramiSukumaran/GeminiFunctionCalling">repository</a>.<pre>&lt;dependency&gt; &lt;groupId&gt;com.google.cloud&lt;/groupId&gt; &lt;artifactId&gt;google-cloud-vertexai&lt;/artifactId&gt; &lt;/dependency&gt; &lt;dependency&gt; &lt;groupId&gt;com.google.code.gson&lt;/groupId&gt; &lt;artifactId&gt;gson&lt;/artifactId&gt; &lt;version&gt;2.10&lt;/version&gt; &lt;/dependency&gt;</pre>9. You can access the entire HelloWorld.java (or whatever you changed it to) class from the github <a href="https://github.com/AbiramiSukumaran/GeminiFunctionCalling/blob/main/src/main/java/cloudcode/helloworld/HelloWorld.java">link</a>. Let’s understand Function Calling by breaking down this class:Prompt Input: In this example, this is what the input prompt looks like:<pre>“What&#39;s the address for the latlong value 40.714224,-73.961452”</pre>You can find the below code snippet relevant to the input prompt in the file:<pre>String promptText = &quot;What&#39;s the address for the latlong value &#39;&quot; + latlngString + &quot;&#39;?&quot;; //40.714224,-73.961452</pre>API Specification / Signature Definition: We decided to use the <a href="https://developers.google.com/maps/documentation/geocoding/requests-reverse-geocoding">Reverse Geocoding API</a>. In this example, this is what the API spec looks like:<pre>/* Declare the function for the API that we want to invoke (Geo coding API) */ FunctionDeclaration functionDeclaration = FunctionDeclaration.newBuilder() .setName(&quot;getAddress&quot;) .setDescription(&quot;Get the address for the given latitude and longitude value.&quot;) .setParameters( Schema.newBuilder() .setType(Type.OBJECT) .putProperties(&quot;latlng&quot;, Schema.newBuilder() .setType(Type.STRING) .setDescription(&quot;This must be a string of latitude and longitude coordinates separated by comma&quot;) .build()) .addRequired(&quot;latlng&quot;) .build()) .build();</pre>Gemini to orchestrate the prompt with the API specification: This is the part where we send the prompt input and the API spec to Gemini:<pre>// Add the function to a &quot;tool&quot; Tool tool = Tool.newBuilder() .addFunctionDeclarations(functionDeclaration) .build(); // Invoke the Gemini model with the use of the tool to generate the API parameters from the prompt input. GenerativeModel model = GenerativeModel.newBuilder() .setModelName(modelName) .setVertexAi(vertexAI) .setTools(Arrays.asList(tool)) .build(); GenerateContentResponse response = model.generateContent(promptText); Content responseJSONCnt = response.getCandidates(0).getContent();</pre>The response from this is the orchestrated parameters JSON to the API. Output from this step would look like below:<pre>role: &quot;model&quot; parts { function_call { name: &quot;getAddress&quot; args { fields { key: &quot;latlng&quot; value { string_value: &quot;40.714224,-73.961452&quot; } } } } }</pre>The parameter that needs to be passed to the Reverse Geocoding API is this:“latlng=40.714224,-73.961452”From the Content object, you can get the Part, call the hasFunctionCall() method to know if it has a function call request that’s returned by the LLM. Call getFunctionCall() to get a FunctionCall object. Use the hasArgs() method to check if there are arguments, and then a getArgs() method to get the actual arguments. It’s a protobuf Struct object. Match the orchestrated result to the format “latlng=VALUE”. Refer to the full code <a href="https://github.com/AbiramiSukumaran/GeminiFunctionCalling/blob/main/src/main/java/cloudcode/helloworld/HelloWorld.java">here</a>.Invoke the API: At this point you have everything you need to invoke the API. The part of the code that does it is below:<pre>// Create a request String url = API_STRING + &quot;?key=&quot; + API_KEY + params; java.net.http.HttpRequest request = java.net.http.HttpRequest.newBuilder() .uri(URI.create(url)) .GET() .build(); // Send the request and get the response java.net.http.HttpResponse&lt;String&gt; httpresponse = client.send(request, java.net.http.HttpResponse.BodyHandlers.ofString()); // Save the response String jsonResult = httpresponse.body().toString();</pre>The string jsonResult holds the stringified response from the reverse Geocoding API. It looks something like this: (This is a formatted version of the output. Please note the result is truncated as well).<pre>“...277 Bedford Ave, Brooklyn, NY 11211, USA; 279 Bedford Ave, Brooklyn, NY 11211, USA; 277 Bedford Ave, Brooklyn, NY 11211, USA;...”</pre>Process the API response and prepare the prompt: The below code processes the response from the API and prepares the prompt with instructions on how to process the response:<pre>// Provide an answer to the model so that it knows what the result // of a &quot;function call&quot; is. String promptString = &quot;You are an AI address standardizer for assisting with standardizing addresses accurately. Your job is to give the accurate address in the standard format as a JSON object containing the fields DOOR_NUMBER, STREET_ADDRESS, AREA, CITY, TOWN, COUNTY, STATE, COUNTRY, ZIPCODE, LANDMARK by leveraging the address string that follows in the end. Remember the response cannot be empty or null. &quot;; Content content = ContentMaker.fromMultiModalData( PartMaker.fromFunctionResponse( &quot;getAddress&quot;, Collections.singletonMap(&quot;address&quot;, formattedAddress))); String contentString = content.toString(); String address = contentString.substring(contentString.indexOf(&quot;string_value: \&quot;&quot;) + &quot;string_value: \&quot;&quot;.length(), contentString.indexOf(&#39;&quot;&#39;, contentString.indexOf(&quot;string_value: \&quot;&quot;) + &quot;string_value: \&quot;&quot;.length())); List&lt;SafetySetting&gt; safetySettings = Arrays.asList( SafetySetting.newBuilder() .setCategory(HarmCategory.HARM_CATEGORY_HATE_SPEECH) .setThreshold(SafetySetting.HarmBlockThreshold.BLOCK_ONLY_HIGH) .build(), SafetySetting.newBuilder() .setCategory(HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT) .setThreshold(SafetySetting.HarmBlockThreshold.BLOCK_ONLY_HIGH) .build() );</pre>Invoke Gemini and return the standardized address : The below code passes the processed output from the above step as prompt to Gemini:<pre>GenerativeModel modelForFinalResponse = GenerativeModel.newBuilder() .setModelName(modelName) .setVertexAi(vertexAI) .build(); GenerateContentResponse finalResponse = modelForFinalResponse.generateContent(promptString + &quot;: &quot; + address, safetySettings); System.out.println(&quot;promptString + content: &quot; + promptString + &quot;: &quot; + address); // See what the model replies now System.out.println(&quot;Print response: &quot;); System.out.println(finalResponse.toString()); String finalAnswer = ResponseHandler.getText(finalResponse); System.out.println(finalAnswer);</pre>The finalAnswer variable has the standardized address in JSON format. Sample output below:<pre>{&quot;replies&quot;:[&quot;{ \&quot;DOOR_NUMBER\&quot;: null, \&quot;STREET_ADDRESS\&quot;: \&quot;277 Bedford Ave\&quot;, \&quot;AREA\&quot;: \&quot;Brooklyn\&quot;, \&quot;CITY\&quot;: \&quot;New York\&quot;, \&quot;TOWN\&quot;: null, \&quot;COUNTY\&quot;: null, \&quot;STATE\&quot;: \&quot;NY\&quot;, \&quot;COUNTRY\&quot;: \&quot;USA\&quot;, \&quot;ZIPCODE\&quot;: \&quot;11211\&quot;, \&quot;LANDMARK\&quot;: null} null}&quot;]}</pre>Now that you have understood how Gemini Function Calling works with the address standardization use case, go ahead and deploy the Cloud Function.10. Now that you have understood how Gemini Function Calling works with the address standardization use case, go ahead and deploy the Cloud Function.<pre>gcloud functions deploy gemini-fn-calling --gen2 --region=us-central1 --runtime=java11 --source=. --entry-point=cloudcode.helloworld.HelloWorld --trigger-http</pre>The result for this deploy command would be a REST URL in the format as below :<a href="https://us-central1-YOUR_PROJECT_ID.cloudfunctions.net/gemini-fn-calling">https://us-central1-YOUR_PROJECT_ID.cloudfunctions.net/gemini-fn-calling</a>11. Test this Cloud Function by running the following command from the terminal:<pre>gcloud functions call gemini-fn-calling --region=us-central1 --gen2 --data &#39;{&quot;calls&quot;:[[&quot;40.714224,-73.961452&quot;]]}&#39;</pre>Response for a random sample prompt:<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*L2ME8WPaWun8XBIX" /><figcaption>Cloud Shell Terminal response for the Cloud Function call</figcaption></figure><pre> &#39;{&quot;replies&quot;:[&quot;{ \&quot;DOOR_NUMBER\&quot;: \&quot;277\&quot;, \&quot;STREET_ADDRESS\&quot;: \&quot;Bedford Ave\&quot;, \&quot;AREA\&quot;: null, \&quot;CITY\&quot;: \&quot;Brooklyn\&quot;, \&quot;TOWN\&quot;: null, \&quot;COUNTY\&quot;: \&quot;Kings County\&quot;, \&quot;STATE\&quot;: \&quot;NY\&quot;, \&quot;COUNTRY\&quot;: \&quot;USA\&quot;, \&quot;ZIPCODE\&quot;: \&quot;11211\&quot;, \&quot;LANDMARK\&quot;: null}}```&quot;]}&#39;</pre>The request and response parameters of this Cloud Function are implemented in a way that is compatible with BigQuery’s remote function invocation. It can be directly consumed from BigQuery data in-place. It means that if your data input (lat and long data) lives in BigQuery then you can call the remote function on the data and get the function response which can be stored or processed within BigQuery directly. To learn how to leverage this Cloud Function in performing in-place LLM insights on your database, refer to this <a href="https://medium.com/google-cloud/in-place-llm-insights-bigquery-gemini-for-structured-unstructured-data-analytics-fdfac0421626">blog</a>.<h3>Conclusion</h3>This project has demonstrated the power of Gemini Function Calling, transforming a generative AI task into a deterministic, reliable process. If you work with generative AI, don’t let its sometimes-unpredictable nature hold you back. Use the power of Gemini 1.0 Pro Function Calling feature and create applications that are as innovative as they are dependable. Start exploring how you can incorporate this feature into your own work! Are there datasets you could validate, information gaps you could fill, or tasks that could be automated with structured calls embedded within your generative AI responses? Here is the link to the <a href="https://github.com/AbiramiSukumaran/GeminiFunctionCalling">repo</a> and for further reading, refer to the documentation for <a href="https://cloud.google.com/vertex-ai">Vertex AI</a>, <a href="https://cloud.google.com/bigquery/docs/remote-functions">BigQuery Remote Functions</a>, and <a href="https://cloud.google.com/functions">Cloud Functions</a> for more in-depth guidance in these areas.<img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c86a5ab80a9" width="1" height="1" alt=""><hr><a href="https://medium.com/google-cloud/using-gemini-function-calling-in-java-for-deterministic-generative-ai-responses-4c86a5ab80a9">Deterministic Generative AI with Gemini Function Calling in Java</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Author

Link

Published date

Image url

Feed url

Guid

Hidden blurb

--- !ruby/object:Feedjira::Parser::RSSEntry
title: Deterministic Generative AI with Gemini Function Calling in Java
url: https://medium.com/google-cloud/using-gemini-function-calling-in-java-for-deterministic-generative-ai-responses-4c86a5ab80a9?source=rss----e52cf94d98af---4
author: Abirami Sukumaran
categories:
- calling-function
- google-cloud-platform
- gcp-app-dev
- gemini
- generative-ai
published: 2024-04-03 04:48:45.000000000 Z
entry_id: !ruby/object:Feedjira::Parser::GloballyUniqueIdentifier
 is_perma_link: 'false'
 guid: https://medium.com/p/4c86a5ab80a9
carlessian_info:
 news_filer_version: 2
 newspaper: Google Cloud - Medium
 macro_region: Blogs
rss_fields:
- title
- url
- author
- categories
- published
- entry_id
- content
content: '<figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*cwsJqTsvzudgNKgCZqg8ow.png"
 /><figcaption>Navigate better with deterministic Generative AI &amp; Reverse Geocoding
 API</figcaption></figure>Generative AI models are remarkable at understanding
 and responding to natural language. But what if you need precise, predictable outputs
 for critical tasks like address standardization? Traditional generative models can
 sometimes provide different responses at different times for the same prompts, potentially
 leading to inconsistencies. That’s where Gemini’s Function Calling capability shines,
 allowing you to deterministically control elements of the AI’s response.In
 this blog, we’ll illustrate this concept with the address completion and standardization
 use case. For this we will be building a Java Cloud Function that:<ol><li>Takes
 latitude and longitude coordinates</li><li>Calls the Google Maps
 Geocoding API to get corresponding addresses</li><li>Uses Gemini
 1.o Pro Function Calling feature to deterministically standardize and summarize
 those addresses in a specific format that we need</li></ol>Let’s dive in!<h3>Gemini
 Function Calling</h3>Gemini Function Calling stands out in the Generative AI
 era because it lets you blend the flexibility of generative language models with
 the precision of traditional programming. Here’s how it works:Defining
 Functions: You describe functions as if you were explaining them to a coworker.
 These descriptions include:<ol><li>The function’s name (e.g., “getAddress”)</li><li>The
 parameters it expects (e.g., “latlng” as a string)</li><li>The type of data it returns
 (e.g., a list of address strings)</li></ol>“Tools” for Gemini:
 You package function descriptions in the form of API specification into “Tools”.
 Think of a tool as a specialized toolbox Gemini can use to understand the functionality
 of the API.Gemini as API Orchestrator: When you send a prompt
 to Gemini, it can analyze your request and recognize where it can use the tools
 you’ve provided. Gemini then acts as a smart orchestrator:<ol><li>Generates
 API Parameters: It produces the necessary parameters to call your defined functions.</li><li>Calls
 External APIs: Gemini doesn’t call the API on your behalf. You call the API based
 on the parameters and signature that Gemini function calling has generated for you.</li><li>Processes
 Results: Gemini feeds the results from your API calls back into its generation,
 letting it incorporate structured information into its final response which you
 can process in any way you desire for your application.</li></ol><h3>High Level
 Flow Diagram</h3>This diagram represents the flow of data and steps involved
 in the implementation. Please note that the owner for the respective step is mentioned
 in the text underneath.<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2-HYu2sGPsNwe46UdJuVdA.png"
 /><figcaption>High level flow diagram of the implementation</figcaption></figure><h3>Industry
 Use cases and why it matters</h3>Below are some of the examples and industry
 specific use cases for function calling with Gemini.<ol><li>Geocoding
 (Our Use Case): You’ve seen how to define a “getAddress” function and use
 it within the context of address standardization.</li><li>Data Validation:
 Imagine a function called “validateEmail” that takes a string and checks it against
 an email validation service. Gemini can help you formulate the parameters string
 so you can call the email validation API to ensure the quality of generated responses.
 Remember, Gemini does not make the API call for you.</li><li>Fact-Checking:
 Define a “lookupFact” function. Gemini could use this to consult a trusted knowledge
 base, making its responses more reliable within specific domains.</li></ol>Why
 Function Calling MattersBridging Two Worlds: LLMs
 can’t know everything (especially private information, customer details, news that
 are more recent than its knowledge cut-off date), and by integrating function calling,
 it’s able to extend its knowledge and capabilities to bridge the gap between open-ended
 generative AI and the deterministic execution of traditional code.Controlled
 Creativity: Function Calling injects structured, predictable elements into
 Gemini’s creative process, ideal for critical use cases or maintaining consistency.Building
 Agent: Chain together multiple function calls and Gemini processing steps,
 enabling multi-stage generative AI workflows.<h3>Create the Java Cloud Function</h3>This
 is the Gen 2 Cloud Function implementation where we will invoke the Gemini model
 to orchestrate the input for function Calling, invoke the API and then process the
 response in another Gemini call and deploy it to a REST endpoint.<h3>Setup</h3><ol><li>In
 the <a href="https://console.cloud.google.com/">Google Cloud Console</a>, on the
 project selector page, select or create a Google Cloud <a href="https://cloud.google.com/resource-manager/docs/creating-managing-projects">project</a>.</li><li>Make
 sure that billing is enabled for your Cloud project. Learn how to <a href="https://cloud.google.com/billing/docs/how-to/verify-billing-enabled">check
 if billing is enabled on a project</a>.</li><li>You will use Cloud Shell, a command-line
 environment running in Google Cloud that comes preloaded with bq. From the Cloud
 Console, click Activate Cloud Shell on the top right corner.</li><li>Just for support
 with building and delivering the app, let’s enable Duet AI. Navigate to <a href="https://console.cloud.google.com/marketplace/product/google/cloudaicompanion.googleapis.com">Duet
 AI Marketplace</a> to enable the API. You can also run the following command in
 the Cloud Shell terminal:</li></ol><pre>gcloud services enable cloudaicompanion.googleapis.com
 –project PROJECT_ID</pre>5. Enable necessary APIs for this implementation if
 you haven’t already.Alternative to the gcloud command is through the console
 using this <a href="https://console.cloud.google.com/apis/enableflow?apiid=bigquery.googleapis.com,bigqueryconnection.googleapis.com,aiplatform.googleapis.com">link</a>.<h3>Java
 Cloud Function</h3><ol><li>Open Cloud Shell Terminal and navigate to the root directory
 or your default workspace path.</li><li>Click the Cloud Code Sign In icon from the
 bottom left corner of the status bar and select the active Google Cloud Project
 that you want to create the Cloud Functions in.</li><li>Click the same icon again
 and this time select the option to create a new application.</li><li>In the Create
 New Application pop-up, select Cloud Functions application:</li></ol><figure><img
 alt="" src="https://cdn-images-1.medium.com/max/1024/0*n53iiiF7ZuyG9J4U" /><figcaption>Create
 New Application page 1</figcaption></figure>5. Select Java: Hello World option
 from the next pop-up:<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jRH5u3oXSLqx-a_k"
 /><figcaption>Create New Application page 2</figcaption></figure>6. Provide a
 name for the project in the project path. In this case, “GeminiFunctionCalling”.7.
 You should see the project structure opened up in a new Cloud Shell Editor view:<figure><img
 alt="" src="https://cdn-images-1.medium.com/max/1024/0*z7PMp8qsGOUik2j1" /><figcaption>Cloud
 Shell Editor showing the project structure with pom.xml open</figcaption></figure>8.
 Now go ahead and add the necessary dependencies within the &lt;dependencies&gt;…
 &lt;/dependencies&gt; tag in the pom.xml file. You can access the entire <a href="https://github.com/AbiramiSukumaran/GeminiFunctionCalling/blob/main/pom.xml">pom.xml</a>
 from this project’s github <a href="https://github.com/AbiramiSukumaran/GeminiFunctionCalling">repository</a>.<pre>&lt;dependency&gt; &lt;groupId&gt;com.google.cloud&lt;/groupId&gt; &lt;artifactId&gt;google-cloud-vertexai&lt;/artifactId&gt; &lt;/dependency&gt; &lt;dependency&gt; &lt;groupId&gt;com.google.code.gson&lt;/groupId&gt; &lt;artifactId&gt;gson&lt;/artifactId&gt; &lt;version&gt;2.10&lt;/version&gt; &lt;/dependency&gt;</pre>9.
 You can access the entire HelloWorld.java (or whatever you changed it to) class
 from the github <a href="https://github.com/AbiramiSukumaran/GeminiFunctionCalling/blob/main/src/main/java/cloudcode/helloworld/HelloWorld.java">link</a>.
 Let’s understand Function Calling by breaking down this class:Prompt
 Input: In this example, this is what the input prompt looks like:<pre>“What&#39;s
 the address for the latlong value 40.714224,-73.961452”</pre>You can find the
 below code snippet relevant to the input prompt in the file:<pre>String promptText
 = &quot;What&#39;s the address for the latlong value &#39;&quot; + latlngString
 + &quot;&#39;?&quot;; //40.714224,-73.961452</pre>API Specification /
 Signature Definition: We decided to use the <a href="https://developers.google.com/maps/documentation/geocoding/requests-reverse-geocoding">Reverse
 Geocoding API</a>. In this example, this is what the API spec looks like:<pre>/*
 Declare the function for the API that we want to invoke (Geo coding API) */ FunctionDeclaration
 functionDeclaration = FunctionDeclaration.newBuilder() .setName(&quot;getAddress&quot;) .setDescription(&quot;Get
 the address for the given latitude and longitude value.&quot;) .setParameters( Schema.newBuilder() .setType(Type.OBJECT) .putProperties(&quot;latlng&quot;,
 Schema.newBuilder() .setType(Type.STRING) .setDescription(&quot;This
 must be a string of latitude and longitude coordinates separated by comma&quot;) .build()) .addRequired(&quot;latlng&quot;) .build()) .build();</pre>Gemini
 to orchestrate the prompt with the API specification: This is the part
 where we send the prompt input and the API spec to Gemini:<pre>// Add the function
 to a &quot;tool&quot; Tool tool = Tool.newBuilder() .addFunctionDeclarations(functionDeclaration) 
 .build(); // Invoke the Gemini model with the use of the tool to generate
 the API parameters from the prompt input. GenerativeModel model = GenerativeModel.newBuilder() 
 .setModelName(modelName) .setVertexAi(vertexAI) .setTools(Arrays.asList(tool)) 
 .build(); GenerateContentResponse response = model.generateContent(promptText); Content
 responseJSONCnt = response.getCandidates(0).getContent();</pre>The response from
 this is the orchestrated parameters JSON to the API. Output from this step would
 look like below:<pre>role: &quot;model&quot; parts { function_call { name:
 &quot;getAddress&quot; args { fields { key: &quot;latlng&quot; value
 { string_value: &quot;40.714224,-73.961452&quot; } } } } }</pre>The
 parameter that needs to be passed to the Reverse Geocoding API is this:“latlng=40.714224,-73.961452”From
 the Content object, you can get the Part, call the hasFunctionCall() method to know
 if it has a function call request that’s returned by the LLM. Call getFunctionCall()
 to get a FunctionCall object. Use the hasArgs() method to check if there are arguments,
 and then a getArgs() method to get the actual arguments. It’s a protobuf Struct
 object. Match the orchestrated result to the format “latlng=VALUE”. Refer to the
 full code <a href="https://github.com/AbiramiSukumaran/GeminiFunctionCalling/blob/main/src/main/java/cloudcode/helloworld/HelloWorld.java">here</a>.Invoke
 the API: At this point you have everything you need to invoke the API.
 The part of the code that does it is below:<pre>// Create a request String
 url = API_STRING + &quot;?key=&quot; + API_KEY + params; java.net.http.HttpRequest
 request = java.net.http.HttpRequest.newBuilder() .uri(URI.create(url)) .GET() .build(); //
 Send the request and get the response java.net.http.HttpResponse&lt;String&gt;
 httpresponse = client.send(request, java.net.http.HttpResponse.BodyHandlers.ofString()); //
 Save the response String jsonResult = httpresponse.body().toString();</pre>The
 string jsonResult holds the stringified response from the reverse Geocoding API.
 It looks something like this: (This is a formatted version of the output. Please
 note the result is truncated as well).<pre>“...277 Bedford Ave, Brooklyn, NY
 11211, USA; 279 Bedford Ave, Brooklyn, NY 11211, USA; 277 Bedford Ave, Brooklyn,
 NY 11211, USA;...”</pre>Process the API response and prepare the prompt: The
 below code processes the response from the API and prepares the prompt with instructions
 on how to process the response:<pre>// Provide an answer to the model so that
 it knows what the result // of a &quot;function call&quot; is. String
 promptString = &quot;You are an AI address standardizer for assisting with
 standardizing addresses accurately. Your job is to give the accurate address in
 the standard format as a JSON object containing the fields DOOR_NUMBER, STREET_ADDRESS,
 AREA, CITY, TOWN, COUNTY, STATE, COUNTRY, ZIPCODE, LANDMARK by leveraging the address
 string that follows in the end. Remember the response cannot be empty or null. &quot;; Content
 content = ContentMaker.fromMultiModalData( PartMaker.fromFunctionResponse( &quot;getAddress&quot;, Collections.singletonMap(&quot;address&quot;,
 formattedAddress))); String contentString = content.toString(); String
 address = contentString.substring(contentString.indexOf(&quot;string_value: \&quot;&quot;)
 + &quot;string_value: \&quot;&quot;.length(), contentString.indexOf(&#39;&quot;&#39;,
 contentString.indexOf(&quot;string_value: \&quot;&quot;) + &quot;string_value: \&quot;&quot;.length())); List&lt;SafetySetting&gt;
 safetySettings = Arrays.asList( SafetySetting.newBuilder() .setCategory(HarmCategory.HARM_CATEGORY_HATE_SPEECH) .setThreshold(SafetySetting.HarmBlockThreshold.BLOCK_ONLY_HIGH) .build(), SafetySetting.newBuilder() .setCategory(HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT) .setThreshold(SafetySetting.HarmBlockThreshold.BLOCK_ONLY_HIGH) .build() );</pre>Invoke
 Gemini and return the standardized address : The below code passes the
 processed output from the above step as prompt to Gemini:<pre>GenerativeModel
 modelForFinalResponse = GenerativeModel.newBuilder() .setModelName(modelName) .setVertexAi(vertexAI) .build(); GenerateContentResponse
 finalResponse = modelForFinalResponse.generateContent(promptString + &quot;: &quot;
 + address, safetySettings); System.out.println(&quot;promptString + content:
 &quot; + promptString + &quot;: &quot; + address); // See what the model
 replies now System.out.println(&quot;Print response: &quot;); System.out.println(finalResponse.toString()); String
 finalAnswer = ResponseHandler.getText(finalResponse); System.out.println(finalAnswer);</pre>The
 finalAnswer variable has the standardized address in JSON format. Sample output below:<pre>{&quot;replies&quot;:[&quot;{
 \&quot;DOOR_NUMBER\&quot;: null, \&quot;STREET_ADDRESS\&quot;: \&quot;277 Bedford
 Ave\&quot;, \&quot;AREA\&quot;: \&quot;Brooklyn\&quot;, \&quot;CITY\&quot;: \&quot;New
 York\&quot;, \&quot;TOWN\&quot;: null, \&quot;COUNTY\&quot;: null, \&quot;STATE\&quot;:
 \&quot;NY\&quot;, \&quot;COUNTRY\&quot;: \&quot;USA\&quot;, \&quot;ZIPCODE\&quot;:
 \&quot;11211\&quot;, \&quot;LANDMARK\&quot;: null} null}&quot;]}</pre>Now that
 you have understood how Gemini Function Calling works with the address standardization
 use case, go ahead and deploy the Cloud Function.10. Now that you have understood
 how Gemini Function Calling works with the address standardization use case, go
 ahead and deploy the Cloud Function.<pre>gcloud functions deploy gemini-fn-calling
 --gen2 --region=us-central1 --runtime=java11 --source=. --entry-point=cloudcode.helloworld.HelloWorld
 --trigger-http</pre>The result for this deploy command would be a REST URL in
 the format as below :<a href="https://us-central1-YOUR_PROJECT_ID.cloudfunctions.net/gemini-fn-calling">https://us-central1-YOUR_PROJECT_ID.cloudfunctions.net/gemini-fn-calling</a>11.
 Test this Cloud Function by running the following command from the terminal:<pre>gcloud
 functions call gemini-fn-calling --region=us-central1 --gen2 --data &#39;{&quot;calls&quot;:[[&quot;40.714224,-73.961452&quot;]]}&#39;</pre>Response
 for a random sample prompt:<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*L2ME8WPaWun8XBIX"
 /><figcaption>Cloud Shell Terminal response for the Cloud Function call</figcaption></figure><pre> &#39;{&quot;replies&quot;:[&quot;{
 \&quot;DOOR_NUMBER\&quot;: \&quot;277\&quot;, \&quot;STREET_ADDRESS\&quot;: \&quot;Bedford
 Ave\&quot;, \&quot;AREA\&quot;: null, \&quot;CITY\&quot;: \&quot;Brooklyn\&quot;,
 \&quot;TOWN\&quot;: null, \&quot;COUNTY\&quot;: \&quot;Kings County\&quot;, \&quot;STATE\&quot;: \&quot;NY\&quot;,
 \&quot;COUNTRY\&quot;: \&quot;USA\&quot;, \&quot;ZIPCODE\&quot;: \&quot;11211\&quot;,
 \&quot;LANDMARK\&quot;: null}}```&quot;]}&#39;</pre>The request and response
 parameters of this Cloud Function are implemented in a way that is compatible with
 BigQuery’s remote function invocation. It can be directly consumed from BigQuery
 data in-place. It means that if your data input (lat and long data) lives in BigQuery
 then you can call the remote function on the data and get the function response
 which can be stored or processed within BigQuery directly. To learn how to leverage
 this Cloud Function in performing in-place LLM insights on your database, refer
 to this <a href="https://medium.com/google-cloud/in-place-llm-insights-bigquery-gemini-for-structured-unstructured-data-analytics-fdfac0421626">blog</a>.<h3>Conclusion</h3>This
 project has demonstrated the power of Gemini Function Calling, transforming a generative
 AI task into a deterministic, reliable process. If you work with generative AI,
 don’t let its sometimes-unpredictable nature hold you back. Use the power of Gemini
 1.0 Pro Function Calling feature and create applications that are as innovative
 as they are dependable. Start exploring how you can incorporate this feature into
 your own work! Are there datasets you could validate, information gaps you could
 fill, or tasks that could be automated with structured calls embedded within your
 generative AI responses? Here is the link to the <a href="https://github.com/AbiramiSukumaran/GeminiFunctionCalling">repo</a>
 and for further reading, refer to the documentation for <a href="https://cloud.google.com/vertex-ai">Vertex
 AI</a>, <a href="https://cloud.google.com/bigquery/docs/remote-functions">BigQuery
 Remote Functions</a>, and <a href="https://cloud.google.com/functions">Cloud Functions</a>
 for more in-depth guidance in these areas.<img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c86a5ab80a9"
 width="1" height="1" alt=""><hr><a href="https://medium.com/google-cloud/using-gemini-function-calling-in-java-for-deterministic-generative-ai-responses-4c86a5ab80a9">Deterministic
 Generative AI with Gemini Function Calling in Java</a> was originally published
 in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium,
 where people are continuing the conversation by highlighting and responding to this
 story.'

Language

Active

Ricc internal notes

Ricc source

Show this article Back to articles