Unlocking Function Calling in LLMs (And Why It’s a Big Deal)

Large language models can generate impressive text, answer questions, and engage in conversation. But until recently, they existed in isolation—unable to interact with external systems, retrieve real-time data, or perform actions beyond generating text. Function calling changed that fundamental limitation, transforming LLMs from sophisticated text generators into agents capable of using tools, accessing databases, and orchestrating complex workflows.

If you’ve only used LLMs for straightforward text generation, you’re missing one of their most powerful capabilities. Function calling is the bridge between language models and the real world, enabling applications that would be impossible with text generation alone.

What Is Function Calling?

Function calling allows an LLM to recognize when it needs external information or capabilities, specify which function should be called with what parameters, and then incorporate the results into its response. The model doesn’t actually execute the function—your code does—but the model understands when a function is needed and how to invoke it correctly.

Here’s a simple example. A user asks: “What’s the weather like in Tokyo right now?” A traditional LLM would either hallucinate an answer or admit it doesn’t have real-time data. With function calling, the model recognizes it needs weather data, generates a structured function call like get_weather(location=”Tokyo”), your code executes that function and returns actual weather data, and the model incorporates that real information into a natural language response.

The key innovation: the model produces structured output specifying function calls in a predictable format, allowing your application to programmatically execute those functions and feed results back to the model.

How Function Calling Works

The process involves several steps:

1. Define Available Functions: You provide the model with descriptions of functions it can call—function names, what they do, and what parameters they accept. These are formatted as JSON schemas that specify each parameter’s type, whether it’s required, and what it represents.
2. Send the Query: The user’s message goes to the model along with the function definitions and conversation history.
3. Model Decides: The model analyzes the query and determines whether it needs to call a function. If the query can be answered directly, it responds normally. If external data or actions are needed, it generates a function call.
4. Execute Functions: Your code receives the function call specification, executes the actual function (calling an API, querying a database, performing a calculation), and gets results.
5. Return Results: You send the function results back to the model as a new message in the conversation.
6. Generate Final Response: The model incorporates the function results into a natural language response to the user.

This back-and-forth can happen multiple times in a single query. The model might call several functions, use results from one function to determine what other functions to call, and orchestrate complex multi-step workflows.

Real-World Applications

Real-Time Information Retrieval: The most obvious use case is accessing current data. Weather, stock prices, news, sports scores, traffic conditions—anything that changes frequently and can’t be in the model’s training data. Function calling lets the model retrieve this information on-demand and incorporate it naturally into responses.

A travel assistant can check real-time flight status, current hotel availability, and weather forecasts for a destination—all in response to a single user query about travel plans.

Database Queries: Rather than exposing your entire database to the model or trying to encode everything in prompts, function calling lets the model query databases strategically. A customer service bot can look up order status, check inventory, or retrieve account information as needed.

The model translates natural language questions into appropriate database queries through function calls, retrieves specific relevant data, and presents it conversationally. This is vastly more efficient than trying to include all possible data in context.

Action Execution: Function calling isn’t just for reading data—it enables actions. A scheduling assistant can actually create calendar events. An email assistant can send messages. A smart home interface can control devices. The model understands user intent, structures the appropriate function call, and your code executes the action.

This transforms LLMs from advisory tools into operational agents that accomplish tasks, not just suggest them.

Multi-Step Workflows: Complex tasks often require multiple steps with dependencies. Function calling enables the model to orchestrate these workflows intelligently.

For example, booking a restaurant reservation might involve: checking available times, verifying the user’s calendar for conflicts, making the reservation, sending a confirmation email, and adding it to the calendar. The model can chain these functions together, using outputs from earlier steps to inform later ones.

Calculation and Data Processing: LLMs are not calculators. They approximate mathematical operations through pattern matching, leading to errors. Function calling solves this by delegating calculations to actual code.

A financial advisor application can use function calls for precise calculations—loan amortization schedules, investment projections, tax computations—ensuring accuracy while the model handles explanation and presentation.

External API Integration: Function calling is the natural bridge to external services. A model can call weather APIs, payment processors, shipping carriers, translation services, or any other API. This makes LLMs the orchestration layer for complex integrations that would otherwise require extensive custom code.

Technical Implementation

Let’s look at how function calling works in practice with OpenAI-compatible APIs:

Defining Functions:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "City and state, e.g. San Francisco, CA"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    }
]

Making a Request:

from openai import OpenAI

client = OpenAI(
    api_key="your-api-key",
    base_url="https://api.deepinfra.com/v1/openai"
)

messages = [
    {"role": "user", "content": "What's the weather in Boston?"}
]

response = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-70B-Instruct",
    messages=messages,
    tools=tools
)

Handling Function Calls:

if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    function_name = tool_call.function.name
    arguments = json.loads(tool_call.function.arguments)
    
    # Execute the actual function
    if function_name == "get_current_weather":
        weather_data = get_current_weather(
            location=arguments["location"],
            unit=arguments.get("unit", "fahrenheit")
        )
        
        # Add function result to conversation
        messages.append(response.choices[0].message)
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(weather_data)
        })
        
        # Get final response
        final_response = client.chat.completions.create(
            model="meta-llama/Meta-Llama-3.1-70B-Instruct",
            messages=messages
        )
        
        print(final_response.choices[0].message.content)

The model returns structured function calls, your code executes them and adds results to the conversation, and the model generates a natural response incorporating that data.

Best Practices

Clear Function Descriptions: The model relies on your function descriptions to understand when and how to call them. Be specific. Instead of “gets data,” write “retrieves current weather conditions including temperature, humidity, and conditions for a specified location.”

Validate Parameters: Don’t blindly trust function parameters from the model. Validate inputs before executing functions, especially for actions that modify data or external systems. Check that dates are valid, amounts are reasonable, and required fields are present.

Handle Errors Gracefully: Functions can fail—APIs go down, databases timeout, parameters are invalid. Return error information to the model so it can explain the problem to users rather than producing nonsensical responses based on missing data.

Limit Function Scope: Each function should do one thing well. Granular functions give the model more control to compose complex workflows. Instead of a “book_travel” function that does everything, provide separate functions for searching flights, booking flights, finding hotels, etc.

Use Enums for Limited Options: When parameters have a restricted set of valid values, define them explicitly in the schema using enums. This reduces errors and helps the model choose correctly.

Consider Security: Function calling can execute arbitrary code paths in your application. Implement proper authorization checks. Just because the model requests an action doesn’t mean the user should be allowed to perform it.

Model Support and Compatibility

Function calling requires model-level support—the model must be trained or fine-tuned to understand function schemas and generate appropriate calls. Not all models support this feature equally.

OpenAI’s GPT-3.5 and GPT-4 have robust function calling support. Among open-source models, several now offer excellent function calling capabilities:

• Llama 3.1 series (especially 70B and larger) includes strong function calling support with reliable adherence to schemas
• Mistral models offer function calling through their “tools” parameter
• Qwen series includes function calling in newer versions
• Specialized models fine-tuned for tool use provide even better reliability

When choosing an inference platform, verify that it supports function calling with your selected models. Platforms like DeepInfra offer function calling support across compatible models using the standard OpenAI-compatible API format, making implementation straightforward regardless of which underlying model you choose.

Function Calling vs. Other Approaches

Traditional Prompt Engineering: Before function calling, developers tried to get models to output structured data through careful prompting—”return your answer as JSON with these fields.” This was unreliable. Models would vary formats, include extra text, or misunderstand schemas. Function calling provides standardized, reliable structured output.

Agents and ReAct Patterns: Function calling is the foundation for agentic workflows. Frameworks like LangChain’s agents use function calling under the hood to let models decide which tools to use and when. Function calling is the primitive that enables these higher-level patterns.

Direct API Integration: You could build systems where your code determines when to call APIs and the model only processes text. Function calling inverts this—the model decides what data it needs, making systems more flexible and intelligent.

Why Function Calling Is Transformative

Function calling fundamentally expands what’s possible with LLMs. Before, you could build chatbots that discuss topics within the model’s training data. Now you can build agents that:

• Access any external data source in real-time
• Perform calculations and data processing accurately
• Execute actions across systems and services
• Orchestrate complex multi-step workflows
• Adapt to user needs by choosing appropriate tools dynamically

This isn’t a minor feature addition—it’s the difference between a sophisticated text generator and a capable AI agent that can actually get things done.

Applications that seemed impossible—”build me a personal assistant that can check my calendar, book appointments, send emails, and order food”—become straightforward to implement. The model handles understanding user intent and orchestrating actions; you provide the functions that interface with real systems.

Getting Started

If you haven’t experimented with function calling yet, start simple:

1. Pick a straightforward use case—maybe a chatbot that can look up real-time information
2. Define one or two simple functions with clear parameters
3. Implement the actual functions in your code
4. Use an OpenAI-compatible API with a model supporting function calling
5. Test with various queries and observe how the model decides when to call functions

Once you see it working, the possibilities become obvious. You’ll start recognizing everywhere that function calling could enhance your applications—adding capabilities, improving accuracy, and enabling workflows that pure text generation can’t handle.

Function calling is the feature that transforms LLMs from impressive text generators into practical tools for building real applications. If you’re building with LLMs and not using function calling, you’re working with one hand tied behind your back.