Re: [RFC] Adding Support for Firefox's Gecko Profile Format

[Anup Sharma]

On Fri, Jun 02, 2023 at 02:19:30AM +0530, Anup Sharma wrote:
> On Sat, May 27, 2023 at 01:19:47AM +0530, Anup Sharma wrote:
> > On Fri, May 26, 2023 at 08:17:44AM -0700, Ian Rogers wrote:
> > > On Thu, May 25, 2023 at 1:53 PM Namhyung Kim <namhyung@xxxxxxxxxx> wrote:
> > > >
> > > > Hi Anup,
> > > >
> > > > On Wed, May 24, 2023 at 12:25 PM Anup Sharma <anupnewsmail@xxxxxxxxx> wrote:
> > > > >
> > > > > Hello everyone,
> > > > >
> > > > > I'm happy to share that I'll be working on adding support for Firefox's Gecko profile format.
> > > > > This format is utilized by the Firefox profiler, which is a powerful tool for performance analysis
> > > > > and debugging. By enhancing the perf data command to generate perf.data files in the Gecko
> > > > > profile format, it will allow us to leverage the capabilities of the Firefox profiler for visualizing
> > > > > and analyzing the performance data.
> > > > >
> > > > > As a starter task, I have created a test for testing the perf data JSON converter command.
> > > > > I'm also looking for a few more starter tasks related to this project. I would greatly appreciate
> > > > > your advice and guidance.
> > > > >
> > > > > In my effort to identify any existing bugs, I have enabled flags like 'fsanitize=address' to detect
> > > > > potential issues but have not found any :). Additionally, I am running perf data commands to ensure
> > > > > that all use cases are handled properly.
> > > >
> > > > Great, good to know it works well with asan for the basic use cases at least.
> > > >
> > > > >
> > > > > I have one question regarding the installation process. Typically, I navigate to the perf directory,
> > > > > run 'make', and then cp 'perf' to '/usr/bin'. However, I noticed that by default, perf is installed in
> > > > > the '~/bin/perf' directory. Could someone please clarify why this is the case? Furthermore, I would
> > > > > like to know how all of you compile the perf tree.
> > > >
> > > > I guess $HOME is the default prefix unless you set it to other, then
> > > > make install will put
> > > > the binary there.  You can make sure if your PATH contains the ~/bin and use it.
> > > >
> > > > But it's also possible you can run the perf without installing.  I
> > > > have a symlink in
> > > > my tmp directory to point to the recent build of the binary and use it
> > > > always. :)
> > > > To build, you can either 'cd tools/perf; make' or 'make -C tools/perf'
> > > > in the top
> > > > level linux source tree.  I also pass "BUILD_BPF_SKEL=1" to enable BPF.
> > > >
> > > > Thanks,
> > > > Namhyung
> > > 
> > > I quite often test with address sanitizer, I do this by passing to make:
> > > DEBUG=1 EXTRA_CFLAGS="-O0 -g -fno-omit-frame- pointer
> > > -fsanitize=address" NO_LIBTRACEEVENT=1
> > > 
> > > The libtraceevent exclusion is to avoid false address sanitizer
> > > warnings in libtraceevent (it wasn't compiled with address sanitizer).
> > > The other flags are to make the code easier to debug. A good place to
> > > start for a description of the build flags is Makefile.perf:
> > > https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/Makefile.perf?h=perf-tools-next
> > > 
> > > There's also some description here:
> > > https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/Documentation/Build.txt?h=perf-tools-next
> > > 
> > > Perhaps you can suggest improvements :-)
> > >
> > 
> > Thanks Namhyung and Ian for your suggestions. I'll try them out and let you know if I have any questions.
> >
> 
> Hello all,
> 
> Thank you for participating in today's office hours and helping me clarify my doubts. 
> One topic that emerged from our discussion is the choice of programming language 
> for writing the converter. I am considering using Python as it offers convenient
> libraries for JSON manipulation. However, I need to investigate whether Python is
> enabled by default in most of the Linux distributions that ship with perf, as its
> absence could pose a potential obstacle.
> 
> Additionally, two profile formats are available: Gecko profile format and Processed profile format[1].
> Upon discussing on Firefox Profiler matrix channel, they recommended opting for the Processed format,
> as it will be supported in future releases as well. Therefore, I intend to begin by working with the
> Processed format and evaluate the results. If any of you have suggestions regarding the choice of 
> format, I would greatly appreciate your input.
> 
> Later on, I will address concerns I have regarding the file-loading process.
> Once again, thank you all for your time.

I wanted to provide an update on my exploration of various tools
and methods to generate a profiler format. In my experimentation, I 
considered following command as a reference. 

perf record -F 99 -g -- perf test -w noploop

The command generated perf.data file has been used with three
different tools to generate the required format and uploaded them to 
profiler.firefox.com. Interestingly, I noticed distinct call trees in
each of the three cases. I find myself in a state of confusion regarding
which result to consider as a reference. Here are the outcomes:

    1. Result obtained using perf script: https://share.firefox.dev/3qxEt7F
    2. Result obtained using samply:https://share.firefox.dev/3OZsha2
    3. Result obtained using simpleperf gecko_profile_generator: https://share.firefox.dev/45Q2BTe

I would greatly appreciate any guidance or suggestions in this matter.

> Thanks, 
> Anup 
> 
> [1] https://github.com/firefox-devtools/profiler/blob/main/docs-developer/custom-importer.md
> 
> > > Thanks,
> > > Ian
> > > 
> > > > >
> > > > > Thank you for your support and I'm looking forward to collaborating with you on this project!