Re: New subsystem for acceleration devices

[Oded Gabbay]

On Mon, Aug 1, 2022 at 5:35 AM <yuji2.ishikawa@xxxxxxxxxxxxx> wrote:
>
> > -----Original Message-----
> > From: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
> > Sent: Monday, August 1, 2022 12:38 AM
> > To: Oded Gabbay <oded.gabbay@xxxxxxxxx>
> > Cc: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> > <yuji2.ishikawa@xxxxxxxxxxxxx>; Jiho Chu <jiho.chu@xxxxxxxxxxx>; Arnd
> > Bergmann <arnd@xxxxxxxx>; Linux-Kernel@Vger. Kernel. Org
> > <linux-kernel@xxxxxxxxxxxxxxx>
> > Subject: Re: New subsystem for acceleration devices
> >
> > On Sun, Jul 31, 2022 at 02:45:34PM +0300, Oded Gabbay wrote:
> > > Hi,
> > > Greg and I talked a couple of months ago about preparing a new accel
> > > subsystem for compute/acceleration devices that are not GPUs and I
> > > think your drivers that you are now trying to upstream fit it as well.
> > >
> > > Would you be open/interested in migrating your drivers to this new
> > subsystem ?
> > >
> > > Because there were no outstanding candidates, I have done so far only
> > > a basic and partial implementation of the infrastructure for this
> > > subsystem, but if you are willing to join I believe I can finish it
> > > rather quickly.
> > >
> > > At start, the new subsystem will provide only a common device
> > > character (e.g. /dev/acX) so everyone will do open/close/ioctl on the
> > > same device character. Also sysfs/debugfs entries will be under that
> > > device and maybe an IOCTL to retrieve information.
> > >
> > > In the future I plan to move some of habanalabs driver's code into the
> > > subsystem itself, for common tasks such as memory management, dma
> > > memory allocation, etc.
> > >
> > > Of course, you will be able to add your own IOCTLs as you see fit.
> > > There will be a range of IOCTLs which are device-specific (similar to
> > > drm).
> > >
> > > wdyt ?
> >
> > That sounds reasonable to me as a sane plan forward, thanks for working on
> > this.
> >
> > greg k-h
>
> Hi,
> Thank you for your suggestion.
> I'm really interested in the idea to have a dedicated subsystem for accelerators.
> Let me challenge if the Visconti DNN driver can be re-written to the framework.
> As Visconti SoC has several accelerators as well as DNN,
> having a general/common interface will be a big benefit for us to maintain drivers for a long time.
>
> I've heard that the framework has some basic implementation.
> Do you have some sample code or enumeration of idea to describe the framework?
> Would you share them with me? if that does not bother you.
>
Great to hear that!

I will need a couple of days to complete the code and clean it up
because I stopped working on it a couple of months ago as there were
no other candidates at the time.
Once I have it ready I will put it in my linux repo and send you a branch name.

In the meantime, I'd like to describe at a high level how this
framework is going to work.

I'll start with the main theme of this framework which is allowing
maximum flexibility to the different device drivers. This is because
in my opinion there is and always will be a large variance between
different compute accelerators, which stems from the fact their h/w is
designed for different purposes. I believe that it would be nearly
impossible to create a standard acceleration API that will be
applicable to all compute accelerators.

Having said that, there are many things that can be common. I'll just
name here a few things:

- Exposing devices in a uniform way (same device character files) and
managing the major/minor/open/close (with callbacks to the open/close
of the device driver)

- Defining a standard user API for monitoring applications that
usually run in a data-center. There is a big difference
between the acceleration uAPI and a monitoring uAPI and while the
former is highly specialized, the latter can be standardized.

- Common implementation of a memory manager that will give services of
allocating kernel memory that can be
  mmap'ed by the user.

- For devices with on-chip MMU, common memory manager for allocating
device memory and managing it.

- Integration with dma-buf for interoperability with other drivers.

The initial implementation of the framework will expose two device
character files per device. One will be used for the main/compute
operations and the other one will be used for monitoring applications.
This is done in case there are some limitations on the main device
file (e.g. allowing only a single compute application to be connected
to the device).

Each driver will call a register function during its probe function
(very similar to what is done in the drm subsystem). This will
register the device in the accel framework and expose the device
character files. Every call on those files
(e.g. open, ioctl) will then go through the accel subsystem first and
then will go to the callbacks of the specific device drivers. The
framework will take care of allocating major and minor numbers and
handle these issues in a uniform way.

For now, I plan to define a single ioctl in the common part, which is
an information ioctl, allowing the user to retrieve various
information on the device (fixed or dynamic). I don't want to jump
ahead of myself and define other common ioctls before gaining some
traction. As I wrote above, each driver will be allowed to define its
own custom ioctls.

There will be an infrastructure to add custom sysfs and debugfs
entries. Once we start working, I'm sure we will find some properties
that we can move to the infrastructure itself (e.g. asking to reset
the device, operational status)

I don't know if I want to add the memory manager we prepared in
habanalabs driver at day one, because I would like to minimize the
effort we have to convert our drivers to using this framework. From
the above, I believe you can see the current effort is not large.

That's it from high-level pov. As I wrote at the beginning, I'll get
back to you in a few days with a link to a git repo containing the
initial implementation.

> If you need further information on the current DNN driver, please let me know.
> Also, I can provide some for other accelerator drivers which are currently under cleaning up for contribution.
I looked at the patches you sent and I believe it will be a good match
for this framework.
If you have additional drivers that you think can be a goot match, by
all means, please send links to me.

Thanks,
Oded