CPU profiling 也有类似边界。perf_sample、cpu_profile_stack_sample 是采样事实,不是执行全量记录;采样频率、采样窗口、unwind、symbolization 和 lost sample 统计都会影响结论。没有有效采样记录时,应写“未获得可用 CPU profile”,再区分未采集、未命中目标、权限或解析问题,不能直接写“没有 CPU 热点”。
SELECT name, idx, severity, source, value, CASE WHEN severity ='data_loss'ANDLOWER(name) LIKE'%ftrace%'THEN'sched_or_ftrace_degraded' WHENLOWER(name) LIKE'%frame_timeline%'THEN'frame_timeline_degraded' WHENLOWER(name) LIKE'%track_event%'THEN'track_event_degraded' WHEN severity IN ('error', 'data_loss') THEN'metric_dependency_check_required' ELSE'configuration_or_parser_context' ENDAS metric_impact_hint FROM stats WHEREvalue!=0 AND ( severity IN ('error', 'data_loss') ORLOWER(name) LIKE'%loss%' ORLOWER(name) LIKE'%drop%' ORLOWER(name) LIKE'%parse%' OR name IN ('ftrace_setup_errors', 'traced_buf_chunks_overwritten', 'traced_buf_chunks_discarded', 'traced_flushes_failed', 'traced_final_flush_failed') ) ORDERBY severity, source, name, idx;
如果结果里出现 data_loss、parser error、buffer overrun、packet loss 相关内容,后面的判断都要降低语气。ftrace loss 会直接影响 sched/thread_state/wakeup 判断,但不一定让 App Track Event 全部不可用;FrameTimeline 缺失会影响帧指标,但不一定影响 Binder 或内存查询。报告里要保留 stats 明细,再由每个指标决定复抓、剔除或降级。
需要做采集配置优化时,再放宽到 value != 0 查看完整 stats,因为 buffer 大小、写入量、ftrace 计数这类 info 项也能解释为什么丢。第 12 篇会专门讲完整的数据丢失分类;这一篇先把它放进自动化入口。
SELECT'actual'AS table_name, COUNT(*) AS row_count FROM actual_frame_timeline_slice UNIONALL SELECT'expected'AS table_name, COUNT(*) AS row_count FROM expected_frame_timeline_slice;
INCLUDE PERFETTO MODULE slices.with_context; INCLUDE PERFETTO MODULE time.conversion;
SELECT 'app_marker'AS window_source, name AS window_name, upid, utid, ts AS start_ts, ts + dur AS end_ts, time_to_ms(ts) AS start_ms, time_to_ms(ts + dur) AS end_ms, ROUND(dur /1e6, 3) AS duration_ms, process_name, thread_name FROM thread_slice WHERE process_name ='com.example.app' AND name ='feed_scroll' AND dur >0 ORDERBY ts LIMIT 10;
WITH target_process(upid) AS (VALUES (NULL)) SELECT 'frame_timeline'AS window_source, name AS window_name, id AS frame_slice_id, upid, ts AS start_ts, ts + dur AS end_ts, time_to_ms(ts) AS start_ms, time_to_ms(ts + dur) AS end_ms, ROUND(dur /1e6, 3) AS duration_ms FROM actual_frame_timeline_slice WHERE upid = (SELECT upid FROM target_process) AND dur >0 ORDERBY dur DESC LIMIT 10;
WITH candidate_processes AS ( SELECT p.name AS process_name, p.upid, p.pid, COUNT(th.utid) AS thread_count, MAX(th.is_main_thread) AS has_main_thread, time_to_ms(MIN(th.start_ts)) AS first_thread_start_ms FROM process p JOIN thread th USING (upid) WHERE (p.name ='com.example.app'OR p.name GLOB 'com.example.app:*') GROUPBY p.name, p.upid, p.pid ) SELECT cp.process_name, cp.upid, cp.pid, cp.thread_count, cp.has_main_thread, cp.first_thread_start_ms, th.name AS thread_name, th.utid, th.tid, th.is_main_thread FROM thread th JOIN candidate_processes cp USING (upid) ORDERBY cp.process_name, th.is_main_thread DESC, th.name;
INCLUDE PERFETTO MODULE slices.with_context; INCLUDE PERFETTO MODULE time.conversion;
WITH target_window(start_ts, end_ts) AS ( VALUES (123000000000, 123500000000) ), target_process(upid) AS (VALUES (NULL)), target_slices AS ( SELECT s.ts, s.dur, s.upid, s.utid, s.name, s.thread_name, s.process_name, MAX(0, MIN(s.ts + s.dur, w.end_ts) -MAX(s.ts, w.start_ts)) AS overlap_dur FROM thread_slice s JOIN target_window w WHERE s.upid = (SELECT upid FROM target_process) AND (s.is_main_thread =1OR s.thread_name ='RenderThread') AND s.dur >0 AND s.ts < w.end_ts AND s.ts + s.dur > w.start_ts ) SELECT upid, utid, time_to_ms(ts) AS ts_ms, ROUND(overlap_dur /1e6, 3) AS overlap_ms, ROUND(dur /1e6, 3) AS original_dur_ms, name, thread_name, process_name FROM target_slices WHERE overlap_dur > time_from_ms(5) ORDERBY overlap_ms DESC;
WITH target_window(start_ts, end_ts) AS ( VALUES (123000000000, 123500000000) ), target_process(upid) AS (VALUES (NULL)), target_threads AS ( SELECT th.utid FROM thread th JOIN process p USING (upid) WHERE p.upid = (SELECT upid FROM target_process) AND (th.is_main_thread =1OR th.name ='RenderThread') ) SELECT'thread_state_rows'AS check_name, COUNT(*) AS row_count FROM thread_state ts JOIN target_threads t USING (utid) JOIN target_window w WHERE ts.dur >0 AND ts.ts < w.end_ts AND ts.ts + ts.dur > w.start_ts UNIONALL SELECT'sched_rows'AS check_name, COUNT(*) AS row_count FROM sched s JOIN target_threads t USING (utid) JOIN target_window w WHERE s.dur >0 AND s.ts < w.end_ts AND s.ts + s.dur > w.start_ts;
WITH target_window(start_ts, end_ts) AS ( VALUES (123000000000, 123500000000) ), target_process(upid) AS (VALUES (NULL)), target_threads AS ( SELECT th.utid, p.name AS process_name, th.name AS thread_name FROM thread th JOIN process p USING (upid) WHERE p.upid = (SELECT upid FROM target_process) AND (th.is_main_thread =1OR th.name ='RenderThread') ), state_in_window AS ( SELECT t.utid, t.process_name, t.thread_name, CASE WHEN ts.state ='Running'THEN'running' WHEN ts.state GLOB 'R*'THEN'runnable' WHEN ts.state ='S'THEN'interruptible_sleep' WHEN ts.state GLOB 'D*'THEN'uninterruptible_blocked' ELSE'other_'||COALESCE(ts.state, 'unknown') ENDAS state_bucket, MAX(0, MIN(ts.ts + ts.dur, w.end_ts) -MAX(ts.ts, w.start_ts)) AS overlap_dur FROM thread_state ts JOIN target_threads t USING (utid) JOIN target_window w WHERE ts.dur >0 AND ts.ts < w.end_ts AND ts.ts + ts.dur > w.start_ts ) SELECT utid, process_name, thread_name, state_bucket, ROUND(SUM(overlap_dur) /1e6, 3) AS overlap_ms FROM state_in_window GROUPBY utid, process_name, thread_name, state_bucket ORDERBY thread_name, overlap_ms DESC;
S 常见于正常 sleep 或 futex 等待,D 及附加标志(如 D|K)属于不可中断等待类别;需要定责时再结合 thread_state 的 blocked reason、io_wait、waker 字段和 schema 版本确认。点击响应、滑动首帧、启动首帧这类场景里,出现长 Runnable 说明 CPU 竞争或调度延迟也要一起查,但它还不能单独定责:线程 running time 是线程维度,CPU busy 和 cluster 解释要另查 sched.cpu、CPU idle、CPU frequency、capacity,并限制在同一窗口内。
import csv import json import os from importlib.metadata import version from pathlib import Path
from perfetto.trace_processor import TraceProcessor, TraceProcessorConfig
TP_BIN = Path(os.environ["TRACE_PROCESSOR_BIN"]) ifnot TP_BIN.is_absolute() ornot TP_BIN.is_file(): raise ValueError("TRACE_PROCESSOR_BIN must be an existing absolute file path") TP_CONFIG = TraceProcessorConfig(bin_path=str(TP_BIN)) PYTHON_PACKAGE_VERSION = version("perfetto")
QUALITY_DETAIL_QUERY = """ SELECT name, idx, severity, source, value FROM stats WHERE value != 0 AND ( severity IN ('error', 'data_loss') OR LOWER(name) LIKE '%loss%' OR LOWER(name) LIKE '%drop%' OR LOWER(name) LIKE '%parse%' OR name IN ('ftrace_setup_errors', 'traced_buf_chunks_overwritten', 'traced_buf_chunks_discarded', 'traced_flushes_failed', 'traced_final_flush_failed') ) ORDER BY severity, source, name, idx; """
FRAME_TIMELINE_QUERY = """ SELECT (SELECT COUNT(*) FROM actual_frame_timeline_slice) AS actual_count, (SELECT COUNT(*) FROM expected_frame_timeline_slice) AS expected_count; """
SLOW_SLICE_QUERY_TEMPLATE = """ INCLUDE PERFETTO MODULE slices.with_context; INCLUDE PERFETTO MODULE time.conversion; WITH target_window(start_ts, end_ts) AS ( VALUES ({start_ts}, {end_ts}) ), target_slices AS ( SELECT s.process_name, s.upid, s.thread_name, s.utid, s.name AS slice_name, MAX(0, MIN(s.ts + s.dur, w.end_ts) - MAX(s.ts, w.start_ts)) AS overlap_dur FROM thread_slice s JOIN target_window w WHERE s.upid = {target_upid} AND (s.is_main_thread = 1 OR s.thread_name = 'RenderThread') AND s.dur > 0 AND s.ts < w.end_ts AND s.ts + s.dur > w.start_ts ) SELECT process_name, upid, thread_name, utid, slice_name, COUNT(*) AS slice_count, ROUND(SUM(overlap_dur) / 1e6, 3) AS overlap_total_ms, ROUND(MAX(overlap_dur) / 1e6, 3) AS max_overlap_ms FROM target_slices WHERE overlap_dur > time_from_ms(5) GROUP BY process_name, upid, thread_name, utid, slice_name ORDER BY overlap_total_ms DESC, upid, utid, slice_name LIMIT 20; """
STATE_QUERY_TEMPLATE = """ WITH target_window(start_ts, end_ts) AS ( VALUES ({start_ts}, {end_ts}) ), target_threads AS ( SELECT th.utid, p.name AS process_name, th.name AS thread_name FROM thread th JOIN process p USING (upid) WHERE p.upid = {target_upid} AND (th.is_main_thread = 1 OR th.name = 'RenderThread') ), state_in_window AS ( SELECT t.process_name, t.thread_name, ts.utid, CASE WHEN ts.state = 'Running' THEN 'running_ns' WHEN ts.state GLOB 'R*' THEN 'runnable_ns' WHEN ts.state = 'S' THEN 'sleeping_ns' WHEN ts.state GLOB 'D*' THEN 'uninterruptible_blocked_ns' WHEN ts.state IS NULL THEN 'unknown_state_ns' ELSE 'other_state_ns' END AS state_bucket, MAX(0, MIN(ts.ts + ts.dur, w.end_ts) - MAX(ts.ts, w.start_ts)) AS overlap_dur FROM thread_state ts JOIN target_threads t USING (utid) JOIN target_window w WHERE ts.dur > 0 AND ts.ts < w.end_ts AND ts.ts + ts.dur > w.start_ts ) SELECT process_name, thread_name, utid, state_bucket, SUM(overlap_dur) AS overlap_ns FROM state_in_window GROUP BY process_name, thread_name, utid, state_bucket; """
defrequired_int(metadata, key): value = metadata[key] iftype(value) isnotintornot0 <= value < 2**63: raise ValueError(f"{key} must be a nonnegative signed-64-bit JSON integer") return value
defquality_summary(rows, frame_row): data_loss_sources = [] ftrace_issue_rows = 0 for row in rows: name = row.name or"" severity = row.severity or"" if severity == "data_loss": data_loss_sources.append(f"{row.source}:{name}[{row.idx}]") if"ftrace"in name.lower() and ( severity in ("error", "data_loss") or"loss"in name.lower() or"drop"in name.lower() ): ftrace_issue_rows += 1
with TraceProcessor(trace=str(trace_path), config=TP_CONFIG) as tp: bounds = next(iter(tp.query("SELECT start_ts, end_ts FROM trace_bounds"))) if start_ts < bounds.start_ts or end_ts > bounds.end_ts: raise ValueError(f"Window outside trace bounds: {trace_path}") identity = list(tp.query( f"SELECT upid FROM process WHERE upid = {target_upid} " f"AND name = {sql_string(process_name)}" )) iflen(identity) != 1: raise ValueError(f"Process identity mismatch: {trace_path}") scope_row = next(iter(tp.query(f""" INCLUDE PERFETTO MODULE slices.with_context; WITH target_threads AS ( SELECT utid FROM thread WHERE upid = {target_upid} AND (is_main_thread = 1 OR name = 'RenderThread') ) SELECT (SELECT COUNT(*) FROM target_threads) AS thread_count, (SELECT COUNT(*) FROM thread_slice WHERE utid IN (SELECT utid FROM target_threads) AND dur > 0 AND ts < {end_ts} AND ts + dur > {start_ts}) AS observed_slice_count; """))) quality_rows = list(tp.query(QUALITY_DETAIL_QUERY)) frame_row = next(iter(tp.query(FRAME_TIMELINE_QUERY))) slow_rows = list(tp.query(slow_slice_query)) state_rows = list(tp.query(state_query))
(quality_dir / f"{trace_path.stem}.stats.json").write_text(json.dumps( [{key: getattr(row, key) for key in ("name", "idx", "severity", "source", "value")} for row in quality_rows], ensure_ascii=False, indent=2), encoding="utf-8") quality = quality_summary(quality_rows, frame_row) state_by_thread = {} for row in state_rows: state_by_thread.setdefault((row.utid, row.thread_name), {})[row.state_bucket] = row.overlap_ns
for row in slow_rows: states = state_by_thread.get((row.utid, row.thread_name), {}) if row else {} # thread_state intervals for one utid are disjoint; sum exact ns. covered_ns = sum(states.values()) full_coverage = bool(states) and covered_ns == end_ts - start_ts sched_coverage = "full"if full_coverage else ("partial"if states else"missing") fill_zero = full_coverage andnot quality_rows andnot states.get("unknown_state_ns", 0) defstate_value(key): value = states.get(key, 0if fill_zero elseNone) returnround(value / 1e6, 3) if value isnotNoneelse""
import glob import os from pathlib import Path import pandas as pd from perfetto.batch_trace_processor.api import BatchTraceProcessor, BatchTraceProcessorConfig from perfetto.trace_processor import TraceProcessorConfig
QUERY = """ INCLUDE PERFETTO MODULE slices.with_context; INCLUDE PERFETTO MODULE time.conversion; WITH candidate_windows AS ( SELECT upid, ts AS start_ts, ts + dur AS end_ts FROM thread_slice WHERE process_name = 'com.example.app' AND is_main_thread = 1 AND name = 'feed_scroll' AND dur > 0 ), target_window AS ( SELECT * FROM candidate_windows WHERE (SELECT COUNT(*) FROM candidate_windows) = 1 ), target_slices AS ( SELECT s.process_name, s.upid, thread_name, utid, name AS slice_name, MAX(0, MIN(ts + dur, w.end_ts) - MAX(ts, w.start_ts)) AS overlap_dur FROM thread_slice s JOIN target_window w ON s.upid = w.upid WHERE dur > 0 AND (is_main_thread = 1 OR thread_name = 'RenderThread') AND ts < w.end_ts AND ts + dur > w.start_ts ), grouped AS ( SELECT process_name, upid, thread_name, utid, slice_name, COUNT(*) AS slice_count, ROUND(SUM(overlap_dur) / 1e6, 3) AS overlap_total_ms FROM target_slices WHERE overlap_dur > time_from_ms(5) GROUP BY process_name, upid, thread_name, utid, slice_name ORDER BY overlap_total_ms DESC, upid, utid, slice_name LIMIT 20 ) SELECT 'observed_top20_slow_slice_groups' AS metric_status, * FROM grouped UNION ALL SELECT CASE (SELECT COUNT(*) FROM candidate_windows) WHEN 0 THEN 'missing_marker' WHEN 1 THEN 'no_slow_slices_in_observed_data' ELSE 'ambiguous_marker' END, NULL, NULL, NULL, NULL, NULL, NULL, NULL WHERE NOT EXISTS (SELECT 1 FROM grouped); """
files = sorted(glob.glob("traces/*.perfetto-trace")) ifnot files: raise ValueError("No input traces") tp_bin = Path(os.environ["TRACE_PROCESSOR_BIN"]) ifnot tp_bin.is_absolute() ornot tp_bin.is_file(): raise ValueError("TRACE_PROCESSOR_BIN must be an existing absolute file path") config = BatchTraceProcessorConfig( tp_config=TraceProcessorConfig(bin_path=str(tp_bin)) ) outputs = [] for offset inrange(0, len(files), 4): with BatchTraceProcessor(files[offset:offset + 4], config=config) as btp: outputs.append(btp.query_and_flatten(QUERY)) result = pd.concat(outputs, ignore_index=True) print(result) # The filesystem resolver adds _path to identify each trace.
还有一条容易被忽略:指标要写出失败时的解释口径。比如“5ms 以上主线程 slice 累计持续时间”只能回答切片里有多少长任务,它不能独立证明卡顿;“Runnable 总耗时”只能说明线程有等待 CPU 的迹象,它也不能替代 CPU 频率、负载和调度器证据。FrameTimeline 缺失时,报告应写 metric_status=unavailable、fallback_used=false、degrade_reason=missing_frame_timeline;ftrace loss 命中时,调度类指标应写 sched_evidence_grade=weak。
嵌套 slice 的持续时间会重叠;SUM(overlap_dur) 是切片累计量,不能当成互不重叠的墙钟时间或 CPU 时间,也不能跨父子切片直接相加来计算线程占用率。
同一目标窗口里的 CPU frequency、CPU idle、thermal、cluster/CPU placement 更适合放在报告的“解释变量”里。它们通常不单独定责,但能解释为什么同样的业务切片在某次 Trace 里变慢。
多 Trace 才能回答回归问题
单份 Trace 适合定位一个现场,多份 Trace 才能回答“优化有没有稳定变好”。很多性能报告的问题不在 Trace 本身,而在证据组织:截图很多,结论下得重,但没有固定配置、没有样本数、没有 data loss 检查、没有 before/after 统计口径。