Debugging Doc: Slow Android Lint Execution Time #6169
Replies: 1 comment 3 replies
-
|
I don't have many specific suggestions yet, but the write-up so far is excellent and I like the direction this is going. Note that the environment variable override will be harder than it seems because we directly call to the linter rather than running it in a separate process. I tried doing this earlier in the week by copying what we do in I think broadly moving toward a very fast incremental run model for local runs and a full run model on CI would be a massive improvement over the current situation. Assuming that incremental rarely misses anything, the CI run is a good backup in case incremental fails but incremental encourages checking changes before uploading (which is broadly a good practice since CI can take a long time to run these days). Not sure if this will actually be feasible, but I'm keen to see the investigation results once more info is uncovered. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Context
Issue: #5886
PR: #6098
Debugging doc: https://docs.google.com/document/d/1RGcf2j_c8Nfh_emBfQOE7ZrlVQyhAuxb-mbPQeaJ_gI/edit?tab=t.0
Issue Details
Describe the problem
The Android Lint script (
bazel run //scripts:android_lint_check) takes ~4.5 minutes locally (M2 Mac) and >8 minutes on CI (GitHub Actions Ubuntu 2-core x86_64). The original issue reported 10–20 minute execution times. The goal is to optimize execution without sacrificing correctness.Screenshots + videos
Not Applicable (script-only, no UI)
Repro steps
developbazel run //scripts:android_lint_check -- $(pwd)--timerflag (measures script-only time, excluding Bazel build phase)Observed behaviour:
Full lint check takes ~4m 33s (script-only, M2 Mac) or >8m on CI. Individual checks each take ~2 minutes due to fixed project model construction overhead.
Expected behaviour:
Faster execution, ideally under 2 minutes for common workflows.
Error logs
Not applicable (no errors, this is a performance optimization issue)
Where did the problem occur?
How often does it happen?
Every time I perform the repro steps.
Link to GitHub issue
Feature Request: Optimize execution time for Android Lint Script
Your Initial Investigation
Online resources
"android lint slow performance large project"
TopDownAnalyzerFacadeForJVM). No built-in cross-run cache exists for standalone invocations."android lint incremental analysis bazel"
rules_android_lintsupports persistent JVM workers for Bazel."android-lint-performance-probe"
"android lint --analyze-only --report-only"
"large android projects lint optimization"
Code analysis
https://github.com/oppia/oppia-android/blob/develop/scripts/src/java/org/oppia/android/scripts/lint/AndroidLintRunner.kt#L253-L278
runAnalysis()generates a project description XML and then runs lint against it:The project description is built by
LintProjectDescription, which walks every.kt/.javafile in all 5 layers (app, domain, testing, utility, data) and writes them as<src>entries:https://github.com/oppia/oppia-android/blob/develop/scripts/src/java/org/oppia/android/scripts/lint/LintProjectDescription.kt#L491-L527
Android Lint then has to parse ALL of these files via UAST (
TopDownAnalyzerFacadeForJVM) before any checks run. This is the ~3.5 minute bottleneck — it happens even for checks that only look at XML/resources.The lint invocation uses
-Wall(all ~400+ checks). 10 checks are suppressed:https://github.com/oppia/oppia-android/blob/develop/scripts/src/java/org/oppia/android/scripts/lint/AndroidLintRunner.kt#L230-L247
Also noticed: the script creates a per-run cache directory, but the Lint CLI may still internally access
~/.android/cache, which violates Bazel hermeticity.https://github.com/oppia/oppia-android/blob/develop/scripts/src/java/org/oppia/android/scripts/lint/LintProjectDescription.kt#L252
Hypothesis Testing
UAST initialization is the bottleneck — source file parsing dominates even for XML-only checks
Confirmed
Ran
--checks=HardcodedText(an XML-only check) with and without .kt/.java source files in the project description.Same pattern for
IconDensities:Re-ran with
--timerto isolate script time from Bazel build overhead.That's a ~4.7x speedup; ~92% of runtime is spent on source parsing for checks that don't need it.
The UAST module (
TopDownAnalyzerFacadeForJVM) parses every.kt/.javafile before any detectors run, regardless of which checks are enabled.This only helps when running source-free checks though; checks like
NewApineed sources.Some individual detectors are disproportionately expensive
Ruled out
Profiled 22 representative checks individually using
--checks=X --timer(mix of source-only, XML, icon, manifest checks).Running all 150 checks together:
So ~2 minutes is fixed overhead (project model construction) and all 150 detectors combined add only ~2.5 more minutes.
No outlier. Disabling individual checks won't help.
Running too many irrelevant checks via
-WallUnder investigation
Enumerated all 183 active checks using
--list.Breakdown:
Of the 17 checks currently firing on Oppia:
Ben asked (Mar 19) for the full grouped list of all ~400 checks (useful vs not-useful), then incremental vs holistic breakdown. Also wants incremental perf for a single expensive check. Still working on this.
~/.android/cachehermeticity violationUntested
Ben flagged that the Lint CLI may access
~/.android/cacheoutside our per-run cache dir, violating Bazel hermeticity.Potential fixes:
ANDROID_LINT_CACHEenv var$HOMESolution
(Not yet solved. Next steps: full check categorization, incremental benchmarking)
Beta Was this translation helpful? Give feedback.
All reactions