From Chatbot to Agent: The Role of Tool Use

A base large language model (LLM) is a text-in, text-out system. It generates responses based on training data with a knowledge cutoff — it can't look up today's weather, query your database, or send an email. Tool use changes that. By exposing structured function definitions to an LLM, developers can give models the ability to invoke real-world capabilities and receive structured results.

This is the architectural foundation of modern AI agents, copilots, and automation pipelines.

How Tool Use Works: The Core Loop

The general mechanism, regardless of provider, follows a pattern:

  1. Define tools: The developer provides a list of available functions — their names, descriptions, and input schemas (usually as JSON Schema).
  2. Model decides to call: When the model determines that a tool would help answer the user's request, it outputs a structured "tool call" rather than a plain text response.
  3. Application executes: Your code intercepts the tool call, executes the actual function (an API request, database query, etc.), and returns the result.
  4. Model continues: The result is fed back into the context, and the model generates its final response incorporating the data.

This loop can repeat multiple times in a single conversation turn for complex multi-step tasks.

OpenAI Function Calling: A Concrete Example

Here's a simplified example of a tool definition in OpenAI's API format:

{
  "type": "function",
  "function": {
    "name": "get_current_weather",
    "description": "Get the current weather for a city",
    "parameters": {
      "type": "object",
      "properties": {
        "city": { "type": "string", "description": "City name" },
        "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
      },
      "required": ["city"]
    }
  }
}

The model receives this definition alongside the user's message. If the user asks "What's the weather in Berlin?", the model returns a structured call to get_current_weather with {"city": "Berlin"} rather than guessing an answer.

The OpenAI Plugin Specification (Legacy Context)

OpenAI's original plugin system (now superseded by the Assistants API and GPT Actions) introduced several conventions that influenced the broader ecosystem:

  • ai-plugin.json: A manifest file hosted at /.well-known/ai-plugin.json describing the plugin's name, description, authentication method, and API spec URL.
  • OpenAPI spec: The plugin's capabilities were described using a standard OpenAPI 3.x document, making them both human-readable and machine-parseable.
  • Natural language descriptions: Each endpoint included rich descriptions specifically written to help the model decide when to call it.

Emerging Standards: Model Context Protocol (MCP)

Anthropic's Model Context Protocol (MCP), open-sourced in late 2024, aims to standardize how LLMs connect to external data sources and tools. Rather than each model and platform defining its own integration format, MCP proposes a unified client-server protocol where:

  • MCP Servers expose tools, resources (files, database records), and prompts.
  • MCP Clients (AI applications) connect to servers and make their capabilities available to the model.

This mirrors how the Language Server Protocol (LSP) standardized IDE integrations — one protocol, many compatible servers and clients.

Security Considerations for AI Tool Use

Giving an LLM the ability to call APIs introduces real risks:

  • Prompt injection: Malicious content in tool results can try to hijack the model's behavior.
  • Over-privileged tools: Apply the principle of least privilege — only expose the capabilities the model genuinely needs.
  • Confirmation gates: For destructive or sensitive actions (deleting records, sending emails), require explicit human approval before execution.
  • Output validation: Always validate and sanitize tool call arguments before executing them.

Tool use transforms LLMs from text generators into action-taking agents. Designed carefully, it unlocks enormous productivity gains. Designed carelessly, it creates new attack surfaces.