Integrate LLM Function Calling to your API Service - A Practical Guide¶
Function Calling is a very powerful tool in LLM. It extracts structured data (function arguments) from unstructured input (natural texts).
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the users location.",
},
},
"required": ["location", "format"],
},
}
}
]
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."},
{"role": "user", "content": "What's the weather like today in Los Angeles?"}
],
tools=tools,
tool_choice="auto",
)
response.choices[0].message
ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_tyZbrnlA2uSJb0i1C9RRfcAB', function=Function(arguments='{"location":"Los Angeles","format":"celsius"}', name='get_current_weather'), type='function')])
The Problem¶
This is great for a small function with minimal set of arguments. But what if your API is a behemoth 100+ parameters of an endpoint?
URL¶
POST /api/submit_user_info
Description¶
This endpoint accepts a JSON object containing user information collected from a form.
Request Format¶
The request must be sent as a JSON object in the body of the POST request.
Request Headers¶
Content-Type: application/json
Request Body¶
{
"Name": "",
"Email": "",
"Password": "",
"Age": "",
"Phone": "",
"Address": "",
"City": "",
"Country": "",
"Zip Code": "",
"Gender": "",
"Date of Birth": "",
"Occupation": "",
"Education": "",
"Interests": "",
"Username": "",
"Website": "",
"Bio": "",
"Favorite Color": "",
"Subscribe to Newsletter": "",
"Agree to Terms": ""
}
Parameters¶
- Name: (string) The full name of the user.
- Email: (string) The email address of the user.
- Password: (string) The password chosen by the user.
- Age: (number) The age of the user.
- Phone: (string) The phone number of the user.
- Address: (string) The residential address of the user.
- City: (string) The city of residence of the user.
- Country: (string) The country of residence of the user.
- Zip Code: (string) The postal code of the user's address.
- Gender: (string) The gender of the user. Possible values are "male", "female", and "other".
- Date of Birth: (string) The birth date of the user in YYYY-MM-DD format.
- Occupation: (string) The occupation of the user.
- Education: (string) The educational background of the user.
- Interests: (string) The interests or hobbies of the user.
- Username: (string) The chosen username of the user.
- Website: (string) The personal website URL of the user.
- Bio: (string) A short biography of the user.
- Favorite Color: (string) The favorite color of the user, represented in a hex code.
- Subscribe to Newsletter: (boolean) Indicates if the user wants to subscribe to the newsletter. Possible values are
true
orfalse
. - Agree to Terms: (boolean) Indicates if the user agrees to the terms and conditions. Possible values are
true
orfalse
.
Response Format¶
The response will be a JSON object indicating the success or failure of the request.
Success Response¶
- Status Code: 200 OK
- Body:
{
"status": "success",
"message": "User information submitted successfully."
}
Error Response¶
- Status Code: 400 Bad Request
- Body:
{
"status": "error",
"message": "Invalid input data."
}
Example Request¶
POST /api/submit_user_info HTTP/1.1
Host: example.com
Content-Type: application/json
{
"Name": "John Doe",
"Email": "john.doe@example.com",
"Password": "securepassword123",
"Age": 30,
"Phone": "123-456-7890",
"Address": "123 Main St",
"City": "Springfield",
"Country": "USA",
"Zip Code": "12345",
"Gender": "male",
"Date of Birth": "1990-01-01",
"Occupation": "Engineer",
"Education": "Master's Degree",
"Interests": "Reading, Hiking",
"Username": "johndoe",
"Website": "https://johndoe.com",
"Bio": "An engineer with a passion for technology.",
"Favorite Color": "#0000ff",
"Subscribe to Newsletter": true,
"Agree to Terms": true
}
Example Response¶
{
"status": "success",
"message": "User information submitted successfully."
}
Error Handling¶
If the request contains invalid data (e.g., missing required fields, incorrect data types), the server will respond with an appropriate error message and a 400 status code.
Ensure all required fields are provided and are of the correct data type before making the request to avoid errors.
{
"userId": 1,
"firstName": "John",
"lastName": "Doe",
"email": "john.doe@example.com",
"phoneNumber": "+1234567890",
"addressLine1": "123 Main St",
"addressLine2": "Apt 4B",
"city": "Metropolis",
"state": "NY",
"zipCode": "10001",
"country": "USA",
"dateOfBirth": "1990-01-01",
"gender": "Male",
"profilePictureUrl": "http://example.com/profile.jpg",
"bio": "Software developer and tech enthusiast.",
"website": "http://johndoe.com",
"twitterHandle": "@john_doe",
"linkedinProfile": "http://linkedin.com/in/johndoe",
"githubProfile": "http://github.com/johndoe",
"facebookProfile": "http://facebook.com/johndoe",
"instagramHandle": "@johndoe",
"occupation": "Software Developer",
"company": "Tech Corp",
"skills": ["Python", "JavaScript", "React"],
"hobbies": ["Cycling", "Reading", "Traveling"],
"language": "English",
"timezone": "America/New_York",
"notificationPreferences": {
"email": true,
"sms": false,
"push": true
},
"subscriptionStatus": "Active",
"accountType": "Premium",
"membershipLevel": "Gold",
"pointsBalance": 1500,
"lastLogin": "2024-05-01T12:00:00Z",
"accountCreated": "2015-06-15T08:00:00Z",
"emailVerified": true,
"phoneVerified": true,
"twoFactorAuthEnabled": false,
"preferredLanguage": "en",
"marketingOptIn": true,
"darkModeEnabled": false,
"sessionTimeout": 30,
"privacySettings": {
"profileVisibility": "Public",
"searchEngineIndexing": true
},
"favoriteCategories": ["Technology", "Science", "Art"],
"recentActivities": ["Logged in", "Updated profile", "Posted a comment"],
"theme": "Light",
"fontSize": "Medium",
"backupEmail": "backup@example.com",
"alternatePhone": "+1098765432",
"emergencyContact": {
"name": "Jane Doe",
"relation": "Sister",
"phone": "+1987654321"
},
"addressHistory": [
{
"address": "456 Old St",
"city": "Gotham",
"state": "NJ",
"zipCode": "07001",
"country": "USA"
},
{
"address": "789 New Ave",
"city": "Star City",
"state": "CA",
"zipCode": "90001",
"country": "USA"
}
],
"preferredContactMethod": "Email",
"communicationPreferences": {
"newsletter": true,
"promotions": false,
"updates": true
},
"profileCompletion": 85,
"lastPasswordChange": "2024-03-20T10:00:00Z",
"trustedDevices": ["iPhone", "MacBook"],
"securityQuestions": [
{
"question": "What is your mother's maiden name?",
"answer": "Smith"
},
{
"question": "What was your first pet's name?",
"answer": "Buddy"
}
],
"recentOrders": [
{
"orderId": 12345,
"orderDate": "2024-04-10",
"status": "Delivered"
},
{
"orderId": 67890,
"orderDate": "2024-04-20",
"status": "Shipped"
}
],
"wishList": ["Book 1", "Gadget 2", "Clothing Item 3"],
"cartItems": [
{
"itemId": 111,
"itemName": "Laptop",
"quantity": 1,
"price": 999.99
},
{
"itemId": 222,
"itemName": "Headphones",
"quantity": 2,
"price": 199.99
}
],
"subscriptionRenewalDate": "2025-06-15",
"preferredPaymentMethod": "Credit Card",
"paymentMethods": [
{
"type": "Credit Card",
"last4": "1234",
"expiryDate": "2026-01"
},
{
"type": "PayPal",
"email": "paypal@example.com"
}
],
"billingAddress": {
"addressLine1": "123 Billing St",
"addressLine2": "Suite 500",
"city": "Finance City",
"state": "TX",
"zipCode": "75001",
"country": "USA"
},
"shippingAddress": {
"addressLine1": "456 Shipping Ave",
"addressLine2": "Warehouse 7",
"city": "Logistics Town",
"state": "FL",
"zipCode": "33001",
"country": "USA"
},
"orderHistory": [
{
"orderId": 34567,
"orderDate": "2023-12-25",
"items": [
{
"itemId": 333,
"itemName": "Tablet",
"quantity": 1,
"price": 499.99
}
],
"totalAmount": 499.99,
"status": "Delivered"
}
],
"subscriptionPlans": ["Basic", "Premium", "Enterprise"],
"discountCodes": ["WELCOME10", "SPRINGSALE"],
"giftCards": [
{
"code": "GIFT2024",
"balance": 50.00
}
],
"loyaltyProgram": {
"memberId": "LP123456",
"status": "Gold",
"points": 1500
},
"appVersion": "1.2.3",
"deviceInfo": {
"deviceType": "Smartphone",
"os": "iOS",
"osVersion": "14.4"
},
"supportTickets": [
{
"ticketId": 7890,
"issue": "Login issue",
"status": "Resolved"
}
],
"favorites": ["Item 1", "Item 2", "Item 3"],
"savedSearches": ["Search 1", "Search 2"],
"preferredDeliveryTime": "Morning",
"deliveryInstructions": "Leave at the front door",
"surveyResponses": [
{
"surveyId": 101,
"answers": [5, 4, 3]
}
],
"feedback": "Great service!",
"productReviews": [
{
"productId": 2222,
"review": "Excellent product!",
"rating": 5
}
],
"browserHistory": ["Page 1", "Page 2"],
"notifications": [
{
"type": "Message",
"content": "You have a new message",
"timestamp": "2024-05-15T14:00:00Z"
}
],
"location": {
"latitude": 40.7128,
"longitude": -74.0060
},
"activityLog": [
"Logged in",
"Updated profile",
"Changed password"
]
}
Filling tools metadata¶
You need to populate the tools
list with the endpoint definition. Depending on the nature of your codebase, this could be a trivial task, or it could be a nightmare.
Not only that, as we all aware, API can change. Every time there's an update in your service, you need to reflect that changes with the LLM as well.
I ran into this issue when working with a team to integrate LLM with their business. There was no docstring to draw information from. The only piece of information we have was a Postman page and the client web UI.
If only there was a way to automate away the metadata generation that also keeps up to date with the API changes.
So I look into the web UI. My hypothesis is: Since the web UI is the primary way that a user interact with the service, it should already contain sufficient information describing what the API functionality is.
the UI itself is an HTML string. Something like this:
Example HTML Form¶
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@4.3.1/dist/css/bootstrap.min.css"
integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">
</head>
<body>
<div class="container col-6">
<form class="form" action="#" method="post">
<div class="form-group">
<label for="name">Name:</label>
<input type="text" class="form-control" id="name" name="name">
</div><br>
<div class="form-group">
<label for="email">Email:</label>
<input type="email" class="form-control" id="email" name="email">
</div><br>
<div class="form-group">
<label for="password">Password:</label>
<input type="password" class="form-control" id="password" name="password"
autocomplete="current-password">
</div><br>
<div class="form-group">
<label for="age">Age:</label>
<input type="number" class="form-control" id="age" name="age">
</div><br>
<div class="form-group">
<label for="phone">Phone:</label>
<input type="tel" class="form-control" id="phone" name="phone">
</div><br>
<div class="form-group">
<label for="address">Address:</label>
<input type="text" class="form-control" id="address" name="address">
</div><br>
<div class="form-group">
<label for="city">City:</label>
<input type="text" class="form-control" id="city" name="city">
</div><br>
<div class="form-group">
<label for="country">Country:</label>
<input type="text" class="form-control" id="country" name="country">
</div><br>
<div class="form-group">
<label for="zipcode">Zip Code:</label>
<input type="text" class="form-control" id="zipcode" name="zipcode">
</div><br>
<div class="form-group">
<label for="gender">Gender:</label>
<select class="form-control" id="gender" name="gender">
<option value="male">Male</option>
<option value="female">Female</option>
<option value="other">Other</option>
</select>
</div><br>
<div class="form-group">
<label for="dob">Date of Birth:</label>
<input type="date" class="form-control" id="dob" name="dob">
</div><br>
<div class="form-group">
<label for="occupation">Occupation:</label>
<input type="text" class="form-control" id="occupation" name="occupation">
</div><br>
<div class="form-group">
<label for="education">Education:</label>
<input type="text" class="form-control" id="education" name="education">
</div><br>
<div class="form-group">
<label for="interests">Interests:</label>
<input type="text" class="form-control" id="interests" name="interests">
</div><br>
<div class="form-group">
<label for="username">Username:</label>
<input type="text" class="form-control" id="username" name="username">
</div><br>
<div class="form-group">
<label for="website">Website:</label>
<input type="url" class="form-control" id="website" name="website">
</div><br>
<div class="form-group">
<label for="bio">Bio:</label>
<textarea class="form-control" id="bio" name="bio"></textarea>
</div><br>
<div class="form-group">
<label for="favorite_color">Favorite Color:</label>
<input type="color" class="form-control" id="favorite_color" name="favorite_color">
</div><br>
<div class="form-group form-check">
<input type="checkbox" class="form-check-input" id="subscribe" name="subscribe">
<label class="form-check-label" for="subscribe">Subscribe to Newsletter</label>
</div><br>
<div class="form-group form-check">
<input type="checkbox" class="form-check-input" id="agree" name="agree">
<label class="form-check-label" for="agree">Agree to Terms</label>
</div><br>
<button type="submit" class="btn btn-primary">Submit</button>
</form>
</div>
</body>
</html>
Deriving metadata from the already existing source¶
Each fields have a coresponding label. So I use an LLM to extract a JSON object from HTML string
import requests
import json
url = "http://127.0.0.1:8080/form_example.html"
html_form = requests.get(url).text
prompt = """
You're given the following HTML Form with several HTML <input> elements:
---
[HTML_FORM]
---
your task is to extract each <input> fields into a JSON object with each key equal to the label of each field. And the value is a dictionary containing:
1. type - JSON Schema basic types,
2. required - <input> has attribute "required",
3. description
For example, the following input field:
---
<input type="text" required name="field1"/>
---
should return this object:
---
{
"field1": {
"type": "string",
"required": False,
"description": "field1"
}
}
---
Respond only in JSON
"""
content = prompt.replace('[HTML_FORM]', html_form)
completion = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{
"role": "user", "content": content
}]
)
arguments = json.loads(completion.choices[0].message.content)
arguments
{'Name': {'type': 'string', 'required': False, 'description': 'Name'}, 'Email': {'type': 'string', 'required': False, 'description': 'Email'}, 'Password': {'type': 'string', 'required': False, 'description': 'Password'}, 'Age': {'type': 'integer', 'required': False, 'description': 'Age'}, 'Phone': {'type': 'string', 'required': False, 'description': 'Phone'}, 'Address': {'type': 'string', 'required': False, 'description': 'Address'}, 'City': {'type': 'string', 'required': False, 'description': 'City'}, 'Country': {'type': 'string', 'required': False, 'description': 'Country'}, 'Zip Code': {'type': 'string', 'required': False, 'description': 'Zip Code'}, 'Gender': {'type': 'string', 'required': False, 'description': 'Gender'}, 'Date of Birth': {'type': 'string', 'required': False, 'description': 'Date of Birth'}, 'Occupation': {'type': 'string', 'required': False, 'description': 'Occupation'}, 'Education': {'type': 'string', 'required': False, 'description': 'Education'}, 'Interests': {'type': 'string', 'required': False, 'description': 'Interests'}, 'Username': {'type': 'string', 'required': False, 'description': 'Username'}, 'Website': {'type': 'string', 'required': False, 'description': 'Website'}, 'Bio': {'type': 'string', 'required': False, 'description': 'Bio'}, 'Favorite Color': {'type': 'string', 'required': False, 'description': 'Favorite Color'}, 'Subscribe to Newsletter': {'type': 'string', 'required': False, 'description': 'Subscribe to Newsletter'}, 'Agree to Terms': {'type': 'string', 'required': False, 'description': 'Agree to Terms'}}
From this information, we can generate the tools
object with the help of LLM
properties = {}
required = []
for key in arguments.keys():
property = arguments[key]
properties[key] = {
"type": property['type'],
"description": property['description']
}
if (property.get('required') == True):
required.append(key)
required = list(set(required))
print(properties)
print(required)
{'Name': {'type': 'string', 'description': 'Name'}, 'Email': {'type': 'string', 'description': 'Email'}, 'Password': {'type': 'string', 'description': 'Password'}, 'Age': {'type': 'integer', 'description': 'Age'}, 'Phone': {'type': 'string', 'description': 'Phone'}, 'Address': {'type': 'string', 'description': 'Address'}, 'City': {'type': 'string', 'description': 'City'}, 'Country': {'type': 'string', 'description': 'Country'}, 'Zip Code': {'type': 'string', 'description': 'Zip Code'}, 'Gender': {'type': 'string', 'description': 'Gender'}, 'Date of Birth': {'type': 'string', 'description': 'Date of Birth'}, 'Occupation': {'type': 'string', 'description': 'Occupation'}, 'Education': {'type': 'string', 'description': 'Education'}, 'Interests': {'type': 'string', 'description': 'Interests'}, 'Username': {'type': 'string', 'description': 'Username'}, 'Website': {'type': 'string', 'description': 'Website'}, 'Bio': {'type': 'string', 'description': 'Bio'}, 'Favorite Color': {'type': 'string', 'description': 'Favorite Color'}, 'Subscribe to Newsletter': {'type': 'string', 'description': 'Subscribe to Newsletter'}, 'Agree to Terms': {'type': 'string', 'description': 'Agree to Terms'}} []
tools = [
{
"type": "function",
"function": {
"name": "submit_user_info",
"description": "This endpoint accepts a JSON object containing user information collected from a form.",
"parameters": {
"type": "object",
"properties": properties,
"required": required
}
}
}
]
tools
[{'type': 'function', 'function': {'name': 'submit_user_info', 'description': 'This endpoint accepts a JSON object containing user information collected from a form.', 'parameters': {'type': 'object', 'properties': {'Name': {'type': 'string', 'description': 'Name'}, 'Email': {'type': 'string', 'description': 'Email'}, 'Password': {'type': 'string', 'description': 'Password'}, 'Age': {'type': 'integer', 'description': 'Age'}, 'Phone': {'type': 'string', 'description': 'Phone'}, 'Address': {'type': 'string', 'description': 'Address'}, 'City': {'type': 'string', 'description': 'City'}, 'Country': {'type': 'string', 'description': 'Country'}, 'Zip Code': {'type': 'string', 'description': 'Zip Code'}, 'Gender': {'type': 'string', 'description': 'Gender'}, 'Date of Birth': {'type': 'string', 'description': 'Date of Birth'}, 'Occupation': {'type': 'string', 'description': 'Occupation'}, 'Education': {'type': 'string', 'description': 'Education'}, 'Interests': {'type': 'string', 'description': 'Interests'}, 'Username': {'type': 'string', 'description': 'Username'}, 'Website': {'type': 'string', 'description': 'Website'}, 'Bio': {'type': 'string', 'description': 'Bio'}, 'Favorite Color': {'type': 'string', 'description': 'Favorite Color'}, 'Subscribe to Newsletter': {'type': 'string', 'description': 'Subscribe to Newsletter'}, 'Agree to Terms': {'type': 'string', 'description': 'Agree to Terms'}}, 'required': []}}}]
Finishing Touch¶
Last step is to use this to accept user's natural text and invoke Function Calling
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{
"role": "system",
"content": """
You have received a message containing various pieces of information. \
Your task is to extract the relevant data from the message and \
call the appropriate tool if necessary.
"""
}, {
"role": "user",
"content": "Hello, my name is Emily Rodriguez, and you can reach me at emily.rodriguez@email.com or (123) 456-7890. My mailing address is 123 Main Street, Anytown, USA. I identify as female, and I wholeheartedly accept the terms of service. Looking forward to engaging with you!"
}],
tools=tools,
tool_choice="auto"
)
response.choices[0].message
ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_znwdxoPwNk2h9asShFjSU30n', function=Function(arguments='{"Name": "Emily Rodriguez", "Email": "emily.rodriguez@email.com", "Phone": "(123) 456-7890", "Address": "123 Main Street", "City": "Anytown", "Country": "USA", "Gender": "female", "Agree to Terms": "true"}', name='submit_user_info'), type='function'), ChatCompletionMessageToolCall(id='call_ZzUJtpqb5R8QYrreJaXXneo8', function=Function(arguments='{"Subscribe to Newsletter": "true"}', name='submit_user_info'), type='function')])
Conclusion¶
This guide should give some ideas into how we can leverage LLM with less efforts. There's still room for improvement. Feel free to use it to inspire your next project when working with LLM.
You can find the full source code here: [https://github.com/tholapz/automatic-guide-api.git]