Create agent skills for on-device generative AI (I/O Connect ‘26)

Introduction: The New Era of Edge AI

As generative artificial intelligence matures, the industry is witnessing a significant shift from centralized cloud-hosted servers to local, on-device execution. At Google I/O Connect, Aaron Walsh (Developer Relations Engineer, Google AI Edge) and Reza Heydari (Software Engineer, Lite-TLM) unveiled a powerful paradigm for expanding the capabilities of lightweight local models: Agent Skills.

By executing Large Language Models (LLMs) directly on user devices—such as smartphones, browsers, and single-board computers—developers can deliver highly responsive, private, and cost-effective experiences. In this article, we’ll break down the Google AI Edge stack, explore the architecture of Agent Skills, and walk through how you can build dynamic, on-device capabilities for your own applications.

Why On-Device Generative AI Matters

Running models locally on edge devices provides a distinct set of advantages over traditional cloud APIs:

• Ultra-Low Latency: Eliminating network roundtrips enables immediate model responses, which is critical for highly interactive or real-time user experiences.
• Enhanced Privacy: User data stays entirely on-device, satisfying strict compliance guidelines and protecting sensitive user information.
• Offline Availability: Your application continues to function perfectly without an active internet connection—whether the user is on an airplane or in a remote area.
• Eliminated Server Costs: Offloading inference workloads to client hardware drastically reduces developer infrastructure and hosting costs.

The Google AI Edge Developer Stack

Google AI Edge is an end-to-end suite designed to take a raw model from training to highly optimized on-device deployment across platforms like Android, iOS, web, and IoT. The pipeline is split into three main phases: preparation, execution, and testing.

1. Preparation: Conversion and Quantization

Before a model can run on mobile hardware, it needs to be adapted. LiteRTorch seamlessly converts model architectures into the optimized runtime formats required by mobile processors. To minimize memory footprint without sacrificing accuracy, the Edge Quantizer compresses weights (e.g., down to 4-bit integers), making large architectures run within tight mobile RAM constraints.

2. Execution: LiteRT and Lite-TLM

At the base of the runtime stack is LiteRT (the evolved framework powering lightweight AI execution), which leverages hardware acceleration across CPUs, GPUs, and NPUs. Above this layer sits Lite-TLM (Task Language Model API). Lite-TLM provides a high-level developer abstraction designed specifically for generative LLMs, handling complex tasks like context window management, key-value (KV) caching, and text generation loops out of the box.

3. Testing: Edge Portal and Model Explorer

To ensure robust performance, developers can utilize the Edge Portal to run, benchmark, and test models across a fleet of physical mobile devices. If any bottlenecks appear, Model Explorer provides a visual debugger to inspect model graphs, evaluate execution paths, and optimize architecture layers before public release.

Demystifying Agent Skills

While on-device models are highly efficient, smaller models (such as 2 Billion parameter variants) often lack the broad domain knowledge and complex instruction-following capabilities of their cloud-based counterpoints. This is where Agent Skills come in.

An Agent Skill is an open-directory format that dynamically injects domain expertise, code, and UI elements into an LLM. It operates like an onboarding guide for a new team member, structuring capabilities into three core vectors:

• Repetitive Prompt Reduction: Saves complex system instructions locally so you don’t have to repeatedly prepend extensive instructions to every user query.
• Dynamic Capability Discovery: Enables the model to register new functional capabilities in real-time, depending on user contexts.
• Domain Knowledge Injection: Provides the model with deterministic logic and reference materials to answer questions it wasn’t natively pre-trained on.

The Structure of an Agent Skill

At its core, a skill is simply a structured local directory loaded at runtime:

[skill_directory]/
├── skill.md (Metadata, description, instructions, and schemas)
├── script.js (Deterministic business logic/API wrappers)
└── assets/ (HTML, CSS, or JS for rendering UI components)

The skill.md file outlines the skill’s identity (the metadata name and description used by the LLM to understand when to invoke it) and lists the function call schema. When a user asks a query that matches the skill description, the LLM generates a structured JSON output mapping to the defined function call, which then executes the local JavaScript code and updates the user interface.

Step-by-Step: The Task Planner & Timer Demo

To demonstrate this in action, Google built a custom Task Planner & Timer skill. Small LLMs natively struggle with an abstract concept of time. By equipping a model with a skill directory, developers can enable it to construct visual, interactive widgets directly within the app.

How It Works:

1. The user provides a complex text prompt: “Help me plan my evening. I need to cook dinner, watch game highlights, and review presentation slides.”
2. The on-device model (e.g., Gemma 2B or Gemini Nano) recognizes that this planning request maps directly to the Task Planner skill description.
3. The model generates a structured JSON payload detailing tasks, durations, and sequences based on the skill’s system instructions.
4. The system parses the JSON, calls the deterministic JavaScript code, and loads the HTML timer asset inside an app WebView, rendering operational timers directly in the chat interface.

Getting Started and Building Your Own Skill

Ready to start building? Google has made this workflow highly accessible through an interactive Google Colab workspace and their open-source AI Edge Gallery application.

Developing with AI Edge Gallery:

• Download the App: Install the AI Edge Gallery developer app on your Android or iOS device.
• Download Your Base Model: Initialize the model locally (such as Gemma 2B) within the developer sandbox.
• Import Your Custom Skill: Use the interactive Colab to bundle your skill.md, scripting logic, and HTML UI. Then, export the zip package and upload it directly into your device’s skills panel.
• Utilize AI to Build Skills: You can even prompt cloud-based models (like Gemini) to output the boilerplate code, schemas, and asset code for new skills based on a simple description of your target feature.

Conclusion: Bridging Models and Applications

On-device LLMs are scaling down in footprint while simultaneously growing in utility. By pairing lightweight, hardware-accelerated runtimes like Lite-TLM with structured, modular Agent Skills, you can turn a basic mobile language model into an interactive application engine. Check out the open-source Google AI Edge GitHub repository, run the Colab, and begin building the next generation of fast, private, and highly capable offline software!