perfetto.common.exceptions.PerfettoException: Trace processor failed to start. Unable to run a simple query in python

[hijaz-siemens]

I have this simple code in python:

from perfetto.trace_processor import TraceProcessor
tp = TraceProcessor(trace='out.perfetto-trace')

query_text = """
SELECT
  id,
  ts,
  dur,
  CASE WHEN state IN ('S', 'I') THEN 0 ELSE 1 END AS layer,
  ucpu,
  utid,
  sched_state_io_to_human_readable_string(state, io_wait) AS state,
  0 AS depth
FROM (thread_state)
WHERE utid = 38'
"""
qr_it = tp.query(query_text)
qr_df = qr_it.as_pandas_dataframe()
print(qr_df.to_string())

I keep getting:

python3 test.py 
Traceback (most recent call last):
  File "/home/hassanijaz/Downloads/test.py", line 2, in <module>
    tp = TraceProcessor(trace='out.perfetto-trace')
  File "/usr/local/lib/python3.10/dist-packages/perfetto/trace_processor/api.py", line 159, in __init__
    self.http = self._create_tp_http(addr)
  File "/usr/local/lib/python3.10/dist-packages/perfetto/trace_processor/api.py", line 280, in _create_tp_http
    url, self.subprocess = load_shell(
  File "/usr/local/lib/python3.10/dist-packages/perfetto/trace_processor/shell.py", line 104, in load_shell
    raise PerfettoException("Trace processor failed to start.\n"
perfetto.common.exceptions.PerfettoException: Trace processor failed to start.
stdout: 
############################################                              62.4%   

[primiano]

@LalitMaganti feels like the code in python/perfetto/trace_processor/shell.py has an optimistic timeout of 3 seconds to wait for trace processor to start responding.

That's okay (maybe a bit too optimistic still) in the case where the binary is already in the system.
But in this case, on the first run, I think this shell.py doesn't realize that we might need to download the prebuilt, which takes more than 3 seconds for sure.

From the stdout feels like they are hitting the path where we curl the prebuilt. I am not too familiar with that code.

Although if my analysis is correct, how come we have never hit this before?

[LalitMaganti]

Although if my analysis is correct, how come we have never hit this before?

I'm not sure honestly.

@hijaz-siemens can you please set load_timeout [1] in TraceProcessorConfig to 10s or similar to confirm if Primiano's point is the problem?

[1] https://github.com/google/perfetto/blob/main/python/perfetto/trace_processor/api.py#L86C3-L86C15