Disclosure: I created this blog post for the purposes of entering the Gemini Live Agent Challenge hackathon. All code, architecture decisions, and findings described here are from my own project, built during the hackathon period. When sharing on social media, please use the hashtag #GeminiLiveAgentChallenge.

Every student has been there. It's late, there's a maths exam tomorrow, and the only AI tool available just hands you the answer. You copy it, submit the homework, and learn absolutely nothing. The test comes and you're exposed.

That frustration was the starting point for Math Tutor AI Agent — a multimodal AI tutoring system built entirely on Google AI and Google Cloud during the Gemini Live Agent Challenge hackathon.

"The tutor that teaches. Not the AI that answers."

The live app runs at ai-agent-frontend-1087118236338.us-central1.run.app. The repositories are open source at github.com/tay4real/ai-agent-backend and github.com/tay4real/ai-agent-frontend.

The problem this solves

Access to quality mathematics tutoring is one of the most unequally distributed educational resources in the world. Private tutors are expensive. Classroom teachers are stretched. And every AI tool available today shares the same fundamental flaw: they give students the answer instead of building their understanding.

The students who succeed in maths almost universally had access to someone patient enough to work through problems with them — explaining the reasoning, not just the result. A student in Lagos or Dhaka has the same intellectual potential as a student in London. What they often lack is that access.

Math Tutor AI Agent is built to close that gap. It is not a calculator. It is a tutor.

Why Gemini 2.5 Flash was the right model

The first architectural decision was the AI model. I chose Google Gemini 2.5 Flash for three specific reasons.

1. Native multimodality in a single API call

Math Tutor AI Agent accepts problems via text, image, and voice. With Gemini 2.5 Flash, all three modalities flow through a single API call — no separate OCR service, no separate audio transcription model.

const response = await geminiModel.generateContentStream({
  contents: [{
    role: "user",
    parts: [
      { text: sessionContext },
      {
        inlineData: {
          mimeType: "image/jpeg",
          data: base64ImageData   // handwritten homework photo
        }
      },
      { text: "Guide me through this step by step." }
    ]
  }]
});

2. Function calling for verified computation

Gemini's function calling API lets the agent autonomously invoke search() for concept lookups and calculator() for symbolic computation — without any explicit routing logic in application code.

3. Cost and speed profile

For an always-available tutoring system, response latency and cost per token matter. Gemini 2.5 Flash is the fastest model in the Gemini family and the most cost-efficient for this use case.

System architecture

The system is two repositories: a Node.js backend and a React frontend, communicating via REST and WebSocket.

The AgentService and session memory

The core of the backend is an AgentService class that owns all interaction with the Gemini API. Every request — text, image, or voice — flows through it.

class AgentService {
  async processMessage(sessionId, userInput, inputType) {
    const history = sessionStore.get(sessionId) || [];
    const systemPrompt = buildTutorSystemPrompt();
    const contents = buildContents(history, userInput, inputType);

    const stream = await geminiModel.generateContentStream({
      systemInstruction: systemPrompt,
      contents,
      tools: availableTools
    });

    let fullResponse = '';
    for await (const chunk of stream) {
      const token = chunk.text();
      this.emit('token', token);
      fullResponse += token;
    }

    sessionStore.save(sessionId, history, userInput, fullResponse);
  }
}

Session memory is held in-process with a 30-minute TTL. The full conversation history is passed back to Gemini on every request, giving the agent complete context of everything discussed.

WebSocket for real-time streaming

I migrated from REST to WebSocket mid-project when I realised REST couldn't support conversational continuity. Tokens stream from Gemini directly to the client as they are generated, and the persistent connection means a student's follow-up question mid-explanation arrives on the same channel with full session context preserved.

wss.on('connection', (ws) => {
  ws.on('message', async (data) => {
    const { type, text, sessionId, imageData } = JSON.parse(data);

    const agent = new AgentService();
    agent.on('token', (token) => {
      ws.send(JSON.stringify({ type: 'token', content: token }));
    });

    await agent.processMessage(sessionId, text, type, imageData);
    ws.send(JSON.stringify({ type: 'done' }));
  });
});

The feature that makes it a tutor: interruptible sessions

The feature I spent the most time on is what I call interruptible tutoring. At any point during a step-by-step explanation, a student can send a follow-up question. The agent addresses it completely and then resumes the original explanation from the right place.

This works because of two things:

• Session memory — the full conversation history, including the interrupted explanation, is passed back to Gemini with the follow-up. The model has complete context.

• System prompt design — the agent is explicitly instructed to address follow-up questions fully and then resume. The prompt frames this as a natural part of good tutoring.

The key prompt insight: "Do not give the answer" produced stilted behaviour. "Your goal is for the student to reach understanding themselves — your job is to create the conditions for that discovery" produced natural, genuinely helpful tutoring. The framing of the goal, not just the constraint, was what mattered.

Multimodal in practice: image and voice

Image input — handwritten homework

The uploaded file is converted to base64 and passed as an inline data part in the Gemini API call. The agent states what it believes the problem to be before beginning the explanation — a verification step that lets students correct any misreading and eliminates a whole class of frustrating interactions.

Voice input — spoken equations

Voice input is sent as a base64-encoded audio file. Gemini handles transcription and interpretation natively, correctly processing spoken mathematical notation like "x squared plus five x" into x² + 5x, without any additional processing layer.

// Backend: /api/audio endpoint
app.post('/api/audio', async (req, res) => {
  const { audioData, sessionId } = req.body;   // base64 MP3

  const contents = [{
    role: "user",
    parts: [
      { inlineData: { mimeType: "audio/mp3", data: audioData } },
      { text: "Please transcribe and then help me solve this." }
    ]
  }];

  const agent = new AgentService();
  await agent.processWithContents(sessionId, contents, res);
});

Deploying to Google Cloud Run — staying within the free tier

Region selection is critical

Cloud Run's always-free tier is only available in us-central1, us-east1, and us-west1. Deploying anywhere else — including European or Asian regions — is billable from the first request. The deploy script is locked to us-central1.

Scale-to-zero is the most important cost control

With --min-instances=0, the container shuts down completely when idle. You burn zero CPU-seconds during quiet periods.

gcloud run deploy math-tutor-ai-backend \
  --region="us-central1" \           # free tier region
  --min-instances="0" \              # scale to zero = zero idle cost
  --max-instances="3" \              # hard cap prevents runaway billing
  --timeout="300s" \                 # supports WebSocket sessions
  --set-secrets="GEMINI_API_KEY=gemini-api-key:latest"  # Secret Manager

Free tier numbers

Cloud Run's always-free tier in us-central1: 180,000 vCPU-seconds, 360,000 GiB-seconds of memory, and 2 million requests per month — resetting every month. For a hackathon project with scale-to-zero, this is effectively unlimited.

Tip: Set a billing budget alert at $1 in the Google Cloud console before deploying. Takes 90 seconds and guarantees you'll never be surprised by a charge.

KaTeX: making maths look like maths

Gemini returns explanations with LaTeX notation naturally. The frontend renders it with KaTeX, a client-side LaTeX renderer. Equations appear properly typeset — not as ugly plain-text approximations. This detail signals to students that the tool takes mathematics seriously.

What I learned

Prompt engineering is product design. The most impactful work was writing the system prompt. The framing of the goal — not just the constraints — determines the entire character of the experience.

Gemini's multimodal capability genuinely changes what you can build. Handling text, image, and audio in a single model call simplifies the architecture and reduces failure points in ways that matter for real-world applications.

Serverless + AI APIs have eliminated the cost barrier for educational technology. A production-quality AI tutoring system can now be deployed and run at zero cost until it reaches meaningful scale. This is significant beyond this one project.

Real-time streaming changes the psychology of the interaction. Watching an explanation build token by token feels like watching someone think. That quality — the sense of a live presence — matters enormously for an application whose value is that it feels like a real tutor.

What's next

Persistent learning profiles. Adaptive difficulty. Expansion into physics and chemistry. Interactive graphing. A lightweight offline mode for low-bandwidth environments.

The goal has not changed from the first line of code to the last: every student, everywhere, deserves a patient and knowledgeable tutor.

This post was created for the purposes of entering the Gemini Live Agent Challenge hackathon.

#GeminiLiveAgentChallenge #GoogleCloud #GeminiAI #BuildWithGoogle #EdTech #AI #NodeJS #React #CloudRun

Links

• Live app: https://ai-agent-frontend-1087118236338.us-central1.run.app

• Backend repo: https://github.com/tay4real/ai-agent-backend

• Frontend repo: https://github.com/tay4real/ai-agent-frontend

React Native, for the uninitiated, is a game-changing framework developed by Facebook. It enables developers to create natively-rendered mobile applications for both iOS and Android platforms using a single JavaScript codebase.

What is React Native?

In the simplest terms, React Native bridges the gap between web development and native app creation. It leverages the same design as the popular React framework but directs these components to native end-points.

Why Use React Native?

Opting for React Native isn't just a fad; it's a strategic move. The framework saves significant development time, reduces cost, and offers high performance. Moreover, its hot-reloading feature ensures that you see the results of the latest change immediately, without losing your current application state.

Installation Guide For React Native

Embarking on your React Native journey involves setting up your environment correctly. Let's dive in!

Prerequisites

Before getting hands-on with React Native, you'll need:

• Operating System: Windows, macOS, or Linux.

• Code Editor: Visual Studio Code, Atom, etc.

• Terminal: Command Prompt, Terminal, or iTerm.

Setting Up Your Environment

1. Ensure that you have a JDK (Java Development Kit) installed. Check by typing java -version in your terminal.

2. Install Android Studio for Android development. This will provide you with the Android SDK and emulators.

3. For iOS development, you'll need a Mac with Xcode installed.

Installing Node.js and the React Native CLI

Node.js is pivotal to React Native development. To install:

1. Head to Node.js official website and download the recommended version.

2. Once installed, open your terminal and type npm install -g react-native-cli. This installs the React Native command-line interface.

Android Development Environment

Ensure that you have the Android Studio SDK, an Android Device or emulator, and the necessary build tools.

iOS Development Environment

For iOS, you'd need a Mac with Xcode and an iOS simulator or device. Ensure that you've also set up CocoaPods, an application-level dependency manager for Swift and Objective-C.

Creating a New React Native Project

Once everything's set up, create a new project with react-native init YourProjectName.

Building with React Native

Components and Syntax

Building in React Native is all about components. These reusable pieces define how UI renders and functions.

State Management and Redux

Redux plays a vital role in state management in React Native applications, ensuring smooth data flow throughout the app.

Testing Your React Native App

Tools for Testing

There's a plethora of tools out there, like Jest and Mocha. Choosing one depends on your project's needs.

Writing and Running Tests

It's crucial to maintain a testing regimen, ensuring your app's robustness and reliability.

Deployment and Distribution

Deploying to App Stores

Deploying your app requires adhering to guidelines set by Apple and Google. Always keep them in mind!

Over-the-Air Updates with CodePush

Microsoft's CodePush allows developers to push code updates to devices instantly.

Best Practices

Performance Optimization

To ensure your app runs buttery smooth, consider techniques like lazy loading.

Common Pitfalls to Avoid

Steer clear from making common mistakes like neglecting platform-specific design guidelines.