[Feature] greenlet profiler for python agent

[jaychoww]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

now the python agent(0.8.0) has a profiler for thread model and it works well when application runing in thread mode, but lots of services are running with gevent(greenlet), the thread profiler can't worker correct with it. an implement of greenlet profiler would give skywalking more applicability

Use case

the application run with gevent can be profiled

Related issues

no

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[wu-sheng]

Hi, do you have any proposals to discuss or submit? Typically, a new idea should be posted on the discussion panel. The feature issue is for tracking tasks having owners.

[jaychoww]

sorry, and thanks for your help to move the issue to here.

[Superskyyy]

@westarest Hello, the idea is great and indeed the current profiler won't work for greenlet, are you going to submit a PR since you checked the option?

[jaychoww]

Yes, I'm going to do this. But I need to discuss this issue:
According to this document , when we profile to the greenlet, can we snapshot the context at the time when the greenlet we profiling be switched in/out?

If it is ok, then we would make a different process flow from thread model(the thread model get snapshots in period time), and it will not be very compactable with the original flow.would it make the oap raise some error when it handle the snapshot from the new flow?

@Superskyyy

[wu-sheng]

I think generally, OAP doesn't care where is the snapshot from. The agent has full control over how to profile.
The key points of the backend required are

1. The snapshots are still in the same task ID. https://github.com/apache/skywalking-data-collect-protocol/blob/3dc973f859e4dab9521bac7f5d2ede2c4f57c524/profile/Profile.proto#L58
2. Segment ID should follow your tracing context, https://github.com/apache/skywalking-data-collect-protocol/blob/3dc973f859e4dab9521bac7f5d2ede2c4f57c524/profile/Profile.proto#L60
3. Only report the finished status you are sure it is done, and the durations of span and the profiling time should match.

The OAP doesn't care your thread model, but timestamp and segment ID matter.

[jaychoww]

hi, I have almost finished the coding work, and I can't find any test code or units about profiling(all of the test cases are about tracing and loging) when I want to add some test cases, did I miss any things about it? Or we do not have a profiling tests framework? @Superskyyy