Re: [PATCH v28 21/22] x86/vdso: Implement a vDSO for Intel SGX enclave call

[Nathaniel McCallum]

On Tue, Mar 17, 2020 at 6:23 PM Xing, Cedric <cedric.xing@xxxxxxxxx> wrote:
>
> On 3/17/2020 9:50 AM, Nathaniel McCallum wrote:
> > On Mon, Mar 16, 2020 at 8:18 PM Xing, Cedric <cedric.xing@xxxxxxxxx> wrote:
> >>
> >> On 3/16/2020 4:59 PM, Sean Christopherson wrote:
> >>> On Mon, Mar 16, 2020 at 04:50:26PM -0700, Xing, Cedric wrote:
> >>>> On 3/16/2020 3:53 PM, Sean Christopherson wrote:
> >>>>> On Mon, Mar 16, 2020 at 11:38:24PM +0200, Jarkko Sakkinen wrote:
> >>>>>>> My suggestions explicitly maintained robustness, and in fact increased
> >>>>>>> it. If you think we've lost capability, please speak with specificity
> >>>>>>> rather than in vague generalities. Under my suggestions we can:
> >>>>>>> 1. call the vDSO from C
> >>>>>>> 2. pass context to the handler
> >>>>>>> 3. have additional stack manipulation options in the handler
> >>>>>>>
> >>>>>>> The cost for this is a net 2 additional instructions. No existing
> >>>>>>> capability is lost.
> >>>>>>
> >>>>>> My vague generality in this case is just that the whole design
> >>>>>> approach so far has been to minimize the amount of wrapping to
> >>>>>> EENTER.
> >>>>>
> >>>>> Yes and no.   If we wanted to minimize the amount of wrapping around the
> >>>>> vDSO's ENCLU then we wouldn't have the exit handler shenanigans in the
> >>>>> first place.  The whole process has been about balancing the wants of each
> >>>>> use case against the overall quality of the API and code.
> >>>>>
> >>>> The design of this vDSO API was NOT to minimize wrapping, but to allow
> >>>> maximal flexibility. More specifically, we strove not to restrict how info
> >>>> was exchanged between the enclave and its host process. After all, calling
> >>>> convention is compiler specific - i.e. the enclave could be built by a
> >>>> different compiler (e.g. MSVC) that doesn't share the same list of CSRs as
> >>>> the host process. Therefore, the API has been implemented to pass through
> >>>> virtually all registers except those used by EENTER itself. Similarly, all
> >>>> registers are passed back from enclave to the caller (or the exit handler)
> >>>> except those used by EEXIT. %rbp is an exception because the vDSO API has to
> >>>> anchor the stack, using either %rsp or %rbp. We picked %rbp to allow the
> >>>> enclave to allocate space on the stack.
> >>>
> >>> And unless I'm missing something, using %rcx to pass @leaf would still
> >>> satisfy the above, correct?  Ditto for saving/restoring %rbx.
> >>>
> >>> I.e. a runtime that's designed to work with enclave's using a different
> >>> calling convention wouldn't be able to take advantage of being able to call
> >>> the vDSO from C, but neither would it take on any meaningful burden.
> >>>
> >> Not exactly.
> >>
> >> If called directly from C code, the caller would expect CSRs to be
> >> preserved.
> >
> > Correct. This requires collaboration between the caller of the vDSO
> > and the enclave.
> >
> >> Then who should preserve CSRs?
> >
> > The enclave.
> >
> >> It can't be the enclave
> >> because it may not follow the same calling convention.
> >
> > This is incorrect. You are presuming there is not tight integration
> > between the caller of the vDSO and the enclave. In my case, the
> > integration is total and complete. We have working code today that
> > does this.
> >
> >> Moreover, the
> >> enclave may run into an exception, in which case it doesn't have the
> >> ability to restore CSRs.
> >
> > There are two solutions to this:
> > 1. Write the handler in assembly and don't return to C on AEX.
> > 2. The caller can simply preserve the registers. Nothing stops that.
> >
> > We have implemented #1.
> >
> What if the enclave cannot proceed due to an unhandled exception so the
> execution has to get back to the C caller of the vDSO API?

mov $60, %rax
mov $1, %rdi
syscall

We exit in all such cases.

> It seems to me the caller has to preserve CSRs by itself, otherwise it
> cannot continue execution after any enclave exception. Passing @leaf in
> %ecx will allow saving/restoring CSRs in C by setjmp()/longjmp(), with
> the help of an exit handler. But if the C caller has already preserved
> CSRs, why preserve CSRs again inside the enclave? It looks to me things
> can be simplified only if the host process handles no enclave exceptions
> (or exceptions inside the enclave will crash the calling thread). Thus
> the only case of enclave EEXIT'ing back to its caller is considered
> valid, hence the enclave will always be able to restore CSRs, so that
> neither vDSO nor its caller has to preserve CSRs.
>
> Is my understanding correct?
>