Re: [PATCH v1] dynamic_debug: add support for logs destination

[jim . cromie]

On Mon, Sep 18, 2023 at 1:20 AM Łukasz Bartosik <lb@xxxxxxxxxxxx> wrote:
>
> pt., 15 wrz 2023 o 20:02 <jim.cromie@xxxxxxxxx> napisał(a):
> >
> > On Fri, Sep 15, 2023 at 9:49 AM Łukasz Bartosik <lb@xxxxxxxxxxxx> wrote:
> > >

hi Lukasz,

sorry my kernel-time has been in my own trees.

What I dont understand is why +T is insufficient.

IIUC, tracefs is intended for production use.
thats why each event can be enabled / disabled
- to select and minimize whats traced, and not impact the system

and +T  can forward all pr_debugs to trace,
(by 1-few trace events defined similarly to others)
or very few, giving yet another selection mechanism
to choose or eliminate specific pr-debugs and reduce traffic to
interesting stuff.

Once your debug is in the trace-buf,
shouldnt user-space be deciding what to do with it ?
a smart daemon could leverage tracefs to good effect.

IMO the main value of +T is that it allows feeding existing pr_debugs
into the place where other trace-data is already integrated and managed.

At this point, I dont see any extra destination handling as prudent.

Jim





> > > Add support for selection of dynamic debug logs destination.
> > > When enabled it allows to configure destination of each callsite
> > > individually. The option adds a framework (described by struct
> > > ddebug_dst_ops) which provides flexible means to add a new destination
> > > for debug logs. On top of the framework support for trace instance as
> > > destination is added. By default destination of debug logs is syslog.
> > > Syslog can be set explicitly by using dst keyword or is selected by
> > > default when dst keyword is omitted, for example:
> >
> > A while back,
> > Cc: Vincent Whitchurch <vincent.whitchurch@xxxxxxxx>
> > proposed these patches:
> >       https://lore.kernel.org/lkml/20200825153338.17061-1-vincent.whitchurch@xxxxxxxx/
> >
> > his approach adds a single flag, +x or (my pref) +T
> > which sends the pr_debug to tracefs, independent of +p.
> > Steve Rostedt offered feedback on one of the versions,
> > and one could read that as provisional acceptance.
> >
> > Ive worked his patchset since, it is here:
> > https://github.com/jimc/linux/tree/dd-trc-11
> > (it has some unrelated bits too)
> >
> > but it sits atop another patchset:
> > https://lore.kernel.org/lkml/20230911230838.14461-1-jim.cromie@xxxxxxxxx/
> > or for git remote add
> > https://github.com/jimc/linux/tree/dd-fix-5i
> >
>
> I looked through the patches you pointed me to. The solution with
> +T/+x is elegant and tailored to specific needs.
> Our rationale to write pr_debug/dev_dbg logs to tracefs is similar to
> Vincent's to aid us in debugging.
> Unfortunately the solution with +T/+x does not cover our use case.
>
> Our use case is different. We experience issues reported by Chromebook
> users which are difficult to reproduce
> that's why we would like to enable debug logs to be written to tracefs
> on productions systems.
> When users experiences an issues and sends us a feedback report the
> debug logs written to tracefs would be
> attached to the feedback report. We would like to write debug logs to
> a separate trace instances based on
> the subsystem/driver. Our initial areas of interest/issues are usbcore
> and thunderbolt drivers/subsystems.
> More may come in the future.
>
> With my proposal this could be achieved for the thunderbolt subsystem
> with the following steps:
> * enable kernel configuration option CONFIG_DYNAMIC_DEBUG_CORE,
> * enable kernel configuration option DYNAMIC_DEBUG_DST and set its
>   operation mode to Static.
> * add the option to the thunderbolt's drivers/thunderbolt/Makefile
> in order to compile in debug logs:
>         CFLAGS_nhi.o := -DDYNAMIC_DEBUG_MODULE
> * compile the kernel and update it on a target device,
> * finally append the entry to the kernel boot command line:
>         thunderbolt.dyndbg="dst trace:thunderbolt =p"
>
> Both the solutions (I mean +x/+T and mine proposal) have pros and cons.
> Maybe both could coexist together, +x/+T by default and this patch as
> configurable option
> which adds more flexibility ?
>
> >
> > syslog and/or tracefs seems sufficient, do you have a 3rd destination ?
> >
>
> I don't have a use case for the third destination, but for example
> adding socket destination to my proposal would be relatively fast and
> easy (that would be for limited use only after a network is up and
> running).
>
> Thanks,
> Lukasz
>
> > thanks