Android Bench | Android Developers

https://developer.android.com/bench • 322 KB fetched
Open original page
Android Bench | Android Developers

Skip to main content

Essentials

* Build AI experiences

* Build AI-powered Android apps with Gemini APIs and more.

*

Get started

* Get started

* Create your first app. Use Compose for UI and adaptive app development to support multiple device form factors.

*

Hello, world!

*

Developer Verification

*

Adaptive apps

*

Compose for UI

*

AI-powered IDE

*

Training

*

Monetization with Play ↗️

* Optimize by form factor

* Build apps that optimize the user experience on phones, tablets, foldables, cars, XR, and more.

*

Phones, tablets, foldables

*

Android for Cars

*

Android TV

*

Android XR

*

Googlebook

*

ChromeOS

*

Wear OS

* Build by category

* Learn to build for your use case by following Google's prescriptive and opinionated guidance.

*

Games

*

Camera & media

*

Social & messaging

*

Health & fitness

*

Productivity

*

Enterprise apps

* Get the latest

* Stay in touch with the latest releases throughout the year, join our preview programs, and give us your feedback.

*

Latest updates

*

Experimental updates

*

Android Studio preview

*

Jetpack & Compose libraries

*

Wear OS releases

*

Privacy Sandbox ↗️

Design & Plan

* Excellent Experiences

* Build the best experiences for your best users.

*

Learn more

* UI Design

* Design a beautiful user interface using Android best practices.

*

Design for Android

*

Mobile

*

Desktop experiences

*

XR Headsets & XR Glasses

*

AI Glasses

*

Widgets

*

Wear OS

*

Android TV

*

Android for Cars

* Architecture

* Design robust, testable, and maintainable app logic and services.

*

Introduction

*

Libraries

*

Navigation

*

Modularization

*

Testing

*

Kotlin Multiplatform

* Quality

* Plan for app quality and align with Play store guidelines.

*

Overview

*

Core value

*

User experience

*

Accessibility

*

Technical quality

*

Excellent Experiences

* Security

* Safeguard users against threats and ensure a secure Android experience.

*

Overview

*

Privacy

*

Permissions

*

Identity

*

Fraud prevention

Develop

* Gemini in Android Studio

* Your AI development companion for Android development.

*

Learn more

*

Get Android Studio

* Core areas

* Get the samples and docs for the features you need.

*

Samples

*

Multidevice support

*

User interfaces

*

Background work

*

Data and files

*

Connectivity

*

All core areas ⤵️

* Tools and workflow

* Use the IDE to write and build your app, or create your own pipeline.

*

Write and debug code

*

Build projects

*

Test your app

*

Performance

*

Command-line tools

*

Gradle plugin API

*

Android Bench

* Device tech

* Write code for form factors. Connect devices and share data.

*

Phones, tablets, foldables

*

ChromeOS

*

Android for Cars

*

Android TV

*

Android XR

*

Wear OS

*

Android Health

*

Better Together

*

All devices ⤵️

* Libraries

* Browse API reference documentation with all the details.

*

Android platform

*

Jetpack libraries

*

Compose libraries

*

Google Play services ↗️

*

Google Play SDK index ↗️

Google Play

* Play Console

* Publish your app or game and grow your business on Google Play.

*

Go to Play Console

*

Learn more ↗️

* Fundamentals

* Learn how to engage users, monitize, and secure your app.

*

Play Monetization

*

Play Integrity

*

Play Policies

*

Play Programs ↗️

* Games Dev Center

* Develop and deliver games. Get tools, downloads, and samples.

*

Overview

*

Play Asset Delivery

*

Play Games Services

*

Play Games on PC

*

Level Up guidelines

*

All Play guides ⤵️

* Libraries

* Browse API reference documentation with all the details.

*

Play Feature Delivery

*

Play In-app Updates

*

Play In-app Reviews

*

Play Install Referrer

*

Google Play services ↗️

*

Google Play SDK index ↗️

*

All Play libraries ⤵️

* Tools & resources

* Tools for publishing, promoting, and managing your app.

*

Android App Bundles

*

Brand & marketing

*

Play Console APIs ↗️

Blog

* the Android Developer's Blog

*

Read the latest

*

Explore the authors →

* Explore categories

*

Product News

Stay in the loop with the latest updates, feature releases, and behind-the-scenes looks at how we’re evolving.

*

Community

Discover stories from our users, spotlight features, and the collaborative projects that bring us all together.

*

How-tos

From quick tips to deep-dive tutorials, these step-by-step guides are designed to help you work smarter and faster.

*

Case Studies

Real-world results from real-world users. Explore how different teams use our tools to achieve their goals.

*

Events & Programs

Get the details on upcoming webinars, workshops, meetups, and special initiatives designed to help you grow.

*

Documentation

Access detailed manuals, API references, and technical specifications to help you build and integrate with confidence.

/

*
English

*
Deutsch

*
Español – América Latina

*
Français

*
Indonesia

*
Polski

*
Português – Brasil

*
Tiếng Việt

*
中文 – 简体

*
日本語

*
한국어

Android Studio

Sign in

*

Android Bench

Leaderboard

Our methodology

Leaderboard archive

Changelog

*

Essentials

*

More

*

Design & Plan

*

More

*

Develop

*

More

*

Leaderboard

*

Our methodology

*

Leaderboard archive

*

Changelog

*

Google Play

*

More

*

Blog

*

More

*

Android Studio

*

Build AI experiences

*

Get started

*

Get started

*

Hello, world!

*

Developer Verification

*

Adaptive apps

*

Compose for UI

*

AI-powered IDE

*

Training

*

Monetization with Play ↗️

*

Optimize by form factor

*

Phones, tablets, foldables

*

Android for Cars

*

Android TV

*

Android XR

*

Googlebook

*

ChromeOS

*

Wear OS

*

Build by category

*

Games

*

Camera & media

*

Social & messaging

*

Health & fitness

*

Productivity

*

Enterprise apps

*

Get the latest

*

Latest updates

*

Experimental updates

*

Android Studio preview

*

Jetpack & Compose libraries

*

Wear OS releases

*

Privacy Sandbox ↗️

*

Excellent Experiences

*

Learn more

*

UI Design

*

Design for Android

*

Mobile

*

Desktop experiences

*

XR Headsets & XR Glasses

*

AI Glasses

*

Widgets

*

Wear OS

*

Android TV

*

Android for Cars

*

Architecture

*

Introduction

*

Libraries

*

Navigation

*

Modularization

*

Testing

*

Kotlin Multiplatform

*

Quality

*

Overview

*

Core value

*

User experience

*

Accessibility

*

Technical quality

*

Excellent Experiences

*

Security

*

Overview

*

Privacy

*

Permissions

*

Identity

*

Fraud prevention

*

Gemini in Android Studio

*

Learn more

*

Get Android Studio

*

Core areas

*

Samples

*

Multidevice support

*

User interfaces

*

Background work

*

Data and files

*

Connectivity

*

All core areas ⤵️

*

Tools and workflow

*

Write and debug code

*

Build projects

*

Test your app

*

Performance

*

Command-line tools

*

Gradle plugin API

*

Android Bench

*

Device tech

*

Phones, tablets, foldables

*

ChromeOS

*

Android for Cars

*

Android TV

*

Android XR

*

Wear OS

*

Android Health

*

Better Together

*

All devices ⤵️

*

Libraries

*

Android platform

*

Jetpack libraries

*

Compose libraries

*

Google Play services ↗️

*

Google Play SDK index ↗️

*

Play Console

*

Go to Play Console

*

Learn more ↗️

*

Fundamentals

*

Play Monetization

*

Play Integrity

*

Play Policies

*

Play Programs ↗️

*

Games Dev Center

*

Overview

*

Play Asset Delivery

*

Play Games Services

*

Play Games on PC

*

Level Up guidelines

*

All Play guides ⤵️

*

Libraries

*

Play Feature Delivery

*

Play In-app Updates

*

Play In-app Reviews

*

Play Install Referrer

*

Google Play services ↗️

*

Google Play SDK index ↗️

*

All Play libraries ⤵️

*

Tools & resources

*

Android App Bundles

*

Brand & marketing

*

Play Console APIs ↗️

*

the Android Developer's Blog

*

Read the latest

*

Explore the authors →

*

Explore categories

*

Product News

*

Community

*

How-tos

*

Case Studies

*

Events & Programs

*

Documentation

*

Android Developers

*

Develop

*

Android Bench

Stay organized with collections

Save and categorize content based on your preferences.

Android Bench

AI-assisted software engineering has seen the emergence of several benchmarks to measure the capabilities of LLMs. Android developers face specific challenges that aren't covered by existing benchmarks, so we created one that focuses on a north star of high quality Android development.

All models
Open weight models
Closed weight models

Model

Score (%)
info

Average percentage of 100 test cases successfully resolved across 5 runs for each model

arrow_range
Cl range (%)
info

Expected performance range, reflecting the results' statistical reliability (p-value

Avg latency (h)
info

Average time taken to solve 100 tasks across 5 runs

Avg cost ($)
info

Average cost per full benchmark run

Claude Opus 5

91.8

87.4 — 95.7

13.2

$175.4

Claude Fable 5

90.9

86.2 — 95.4

8.7

$144.3

GPT 5.6 Sol

90.8

85.6 — 95.6

14.2

$150.0

Kimi K3

90.2

85.0 — 94.6

29.6

$112.9

GPT 5.6 Luna

87.6

81.8 — 92.3

10.6

$7.2

Qwen3.8 Max

87.0

82.2 — 91.6

41.2

$196.7

GPT 5.6 Terra

86.8

81.7 — 91.2

7.6

$48.0

GPT 5.5

80.2

73.0 — 86.6

11.4

$138.3

Claude Sonnet 5

76.2

68.8 — 82.5

12.3

$99.9

Gemini 3.6 Flash

75.6

67.5 — 82.1

10.2

$135.5

Gemini 3.1 Pro Preview

74.3

66.8 — 81.5

10.5

$86.6

GPT 5.4

74.1

66.6 — 81.2

8.4

$83.4

Claude Opus 4.8

72.4

65.1 — 79.2

6.7

$88.0

GLM 5.2

72.2

65.3 — 79.1

38.9

$117.0

Gemini 3.5 Flash

71.1

63.8 — 78.5

28.3

$165.6

Kimi K2.7 Code

70.4

63.4 — 77.1

31.8

$48.1

Claude Opus 4.7

68.7

60.9 — 76.4

7.0

$96.5

Kimi K2.6

67.6

59.8 — 74.8

57.2

$49.4

Claude Sonnet 4.6

67.0

58.5 — 75.1

16.9

$127.6

Minimax M3

63.6

56.1 — 70.7

26.0

$41.7

GLM 5.1

63.2

55.3 — 70.2

17.6

$53.5

Gemini 3 Flash Preview

62.5

54.7 — 69.7

13.1

$30.1

Mimo V2.5 Pro

60.8

53.2 — 68.6

13.6

$9.2

Deepseek V4 Pro

59.5

51.4 — 67.9

9.0

$3.7

Qwen3.7 Plus

57.7

50.2 — 65.6

18.5

$18.6

Deepseek V4 Flash

54.7

46.9 — 62.8

8.9

$1.5

Qwen3.7 Max

54.2

46.9 — 62.1

14.2

$58.3

Gemini 3.5 Flash-Lite

50.6

42.3 — 58.4

5.4

$34.1

Qwen3.6 27b

45.1

37.6 — 52.8

25.8

$97.3

MiniMax M2.7

41.6

34.7 — 49.4

18.2

$14.9

Gemma 4 31b IT

37.1

29.9 — 44.5

36.3

$10.4

Qwen3.6 35b A3b

37.0

29.1 — 44.3

16.3

$17.8

Latest results as of

August 11th.

View archived leaderboards and check back periodically for updates.

Latest Updates

Track the latest AI model benchmarks, newly introduced agent architectures, and continuous performance evaluations on the platform. Stay updated with our routine methodology updates and release logs.

View all updates

*

circle

New models •

Aug 11th

Claude Opus 5

*

circle

New models •

Aug 11th

GPT 5.6 Sol, GPT 5.6 Luna, GPT 5.6 Terra

*

circle

New models •

Aug 11th

Kimi K3

*

circle

New models •

Aug 11th

Qwen3.8 Max

*

circle

New models •

Aug 11th

Gemini 3.6 Flash, Gemini 3.5 Flash Lite

*

circle

Archived models •

Aug 11th

Gemma 4 26B A4B IT

*

circle

New updates •

Jul 13th

Share results on the community repository

*

circle

New updates •

Jul 13th

Contribute new tasks to our community dataset on GitHub

*

circle

New updates •

Jul 8th

Dataset available on Harbor

*

circle

New models •

Jul 8th

Claude Fable 5, Claude Sonnet 5, Claude Opus 4.8

*

circle

New models •

Jul 8th

Qwen 3.7 Max, Qwen 3.7 Plus

*

circle

New models •

Jul 8th

GLM 5.2

*

circle

New models •

Jul 8th

Kimi K2.7 Code

*

circle

New models •

Jul 8th

MiniMax M3

*

circle

Archived models •

Jul 8th

Claude Opus 4.6, Claude Sonnet 4.5

*

circle

Archived models •

Jul 8th

GPT OSS 120B, GPT OSS 20B

*

circle

Archived models •

Jul 8th

Qwen 3.5 9B, Qwen 3.6 Max Preview

*

circle

New updates •

Jul 8th

We have migrated our benchmark framework to Harbor

*

circle

New updates •

Jul 8th

We've updated Android Bench

Learn more about Android Bench

Our methodology

Learn more about how we created a set of common Android developer tasks.

Read more

Android best practices

Many of the tasks are based on how we define high quality Android development, which is detailed in our developer documentation.

Read more

Harbor dataset

See the full dataset on Harbor dashboard.

View more

*

X

Follow @AndroidDev on X

*

YouTube

Check out Android Developers on YouTube

*

LinkedIn

Connect with the Android Developers community on LinkedIn

*
More Android

*

Android

*

Android for Enterprise

*

Security

*

Source

*

News

*

Blog

*

Podcasts

*
Discover

*

Gaming

*

Machine Learning

*

Health & Fitness

*

Camera & Media

*

Privacy

*

5G

*
Android Devices

*

Large screens

*

Wear OS

*

ChromeOS devices

*

Android for cars

*

Android TV

*
Releases

*

Android 17

*

Android 16

*

Android 15

*

Android 14

*

Android 13

*

Android 12

*

Android 11

*
Documentation and Downloads

*

Android Studio guide

*

Developers guides

*

API reference

*

Download Studio

*

Android NDK

*
Support

*

Report platform bug

*

Report documentation bug

*

Google Play support

*

Join research studies

*

Android

*

Chrome

*

Firebase

*

Google Cloud Platform

*

All products

*

Privacy

*

License

*

Brand guidelines

*

Manage cookies

*

Get news and tips by email

Subscribe

*
English

*
Deutsch

*
Español – América Latina

*
Français

*
Indonesia

*
Polski

*
Português – Brasil

*
Tiếng Việt

*
中文 – 简体

*
日本語

*
한국어
Android Bench | Android Developers

Links found on this page
