Re: [PATCH 4/4] kvm: x86: export TSC offset to user-space

From: Masami Hiramatsu
Date: Sat Sep 03 2016 - 00:11:38 EST


On Fri, 2 Sep 2016 21:23:11 -0300
Marcelo Tosatti <mtosatti@xxxxxxxxxx> wrote:

> On Fri, Sep 02, 2016 at 10:15:41AM -0400, Steven Rostedt wrote:
> > On Fri, 2 Sep 2016 09:43:01 -0400
> > Stefan Hajnoczi <stefanha@xxxxxxxxx> wrote:
> >
> > > Can TSC offset changes occur at runtime?
>
> Yes, but Linux guests don't write to the TSC offset
> after booting and unless user does manual TSC writes.
>
> > > One example is vcpu hotplug where the tracing tool would need to fetch
> > > the new vcpu's TSC offset after tracing has already started.
> > >
> > > Another example is if QEMU or the guest change the TSC offset while
> > > running. If the tracing tool doesn't notice this then trace events will have
> > > incorrect timestamps.
>
> So what happens is this:
>
> HostTSC (a variable).
> GuestTSC (variable) = GuestTSCOffset (fixed) + HostTSC (variable)

The same idea has been done by Yoshihiro
http://events.linuxfoundation.org/sites/events/files/cojp13_yunomae.pdf


> Then the algorithm processes the trace as follows:
> line = for each line(guest_trace)
> line = line - GuestTSCOffset (only the timestamp of course)
>
> So from the moment the guest writes a new TSC offset, the host
> should use the new TSC offset to subtract from the trace entries.
> The trace entries are in fact:
>
> HostTSC + GuestTSCOffset
>
> So the guest trace should contain entries for "USE NEW TSC OFFSET,
> VALUE: xxx", which can be done (hum not sure if guest entries
> or host entries).
>
> However, correct me if i am wrong, the usecase seems to be:
>
> 1) Boot guest.
> 2) run trace-cmd
> 3) run workload
> 4) read traces on host.

IIRC, previous (current?) method is to run trace-cmd at first (before
boot the guest) so that it can get tsc-offset event and
can wait on a special unix domain socket.

For above usecase, we have to have an interface to get the current
tsc offset like Luis suggested.

>
> Another option is to have notifications as follows: record on a buffer
> the following:
>
> [ EVENT: TSC offset write, VAL: [ host tsc value, guest tsc offset ] ]
> [ EVENT: TSC offset write, VAL: [ host tsc value, guest tsc offset ] ]
>
> Then when merging the trace entries, you do:
>
> line = for each line(host trace)
> write_to_merged_trace(line)
> if (contains_tsc_offset_event(line)) {
> GuestTSCOffset = line.GuestTSCOffset
> if (!guest_tsc_offset_initialized) {
> process_all_guest_lines(
> line = line - GuestTSCOffset (only the timestamp of course)
> }
> }
>
> Aha, fail: the traces on the host are not sufficient to know when
> to use which offset to subtract on the guest trace.
>
> So the only possibility is to have the guest inform the occurrence
> of the events: however the guest does not have access to the TSC offset.
>
> So the host needs to inform the new tsc offset value and the guest needs
> to inform when the event occurred on its side. So the algorithm can use
> information on both traces to know which value to subtract on the
> algorithm above.
>
> Is this necessary? Or people do:
> 1) Boot guest.
> 2) run trace-cmd
> 3) run workload
> 4) read traces on host.
>
> > I believe there are tracepoints for these events. They would obviously
> > need to be enabled for the tracer to catch them.

Yes, Yoshihiro introduced a tracepoint for that.
http://lkml.iu.edu/hypermail/linux/kernel/1306.1/01741.html

So, we have to trace this event in host side.

> >
> > I would also recommend that they go into their own instance to make
> > sure other events do not overwrite them.
> >
> > -- Steve

Thanks,

--
Masami Hiramatsu <mhiramat@xxxxxxxxxx>