Re: [PATCH v2 5/9] arm64: dts: ls1046a: make dma-coherent global to the SoC

[Sean Anderson]

On 9/28/22 7:15 PM, Leo Li wrote:
> 
> 
>> -----Original Message-----
>> From: Sean Anderson <sean.anderson@xxxxxxxx>
>> Sent: Friday, September 23, 2022 11:35 AM
>> To: Leo Li <leoyang.li@xxxxxxx>; shawnguo@xxxxxxxxxx;
>> devicetree@xxxxxxxxxxxxxxx
>> Cc: robh+dt@xxxxxxxxxx; linux-arm-kernel@xxxxxxxxxxxxxxxxxxx; linux-
>> kernel@xxxxxxxxxxxxxxx; Laurentiu Tudor <laurentiu.tudor@xxxxxxx>
>> Subject: Re: [PATCH v2 5/9] arm64: dts: ls1046a: make dma-coherent global
>> to the SoC
>> 
>> 
>> 
>> On 9/23/22 12:26 PM, Leo Li wrote:
>> >
>> >
>> >> -----Original Message-----
>> >> From: Sean Anderson <sean.anderson@xxxxxxxx>
>> >> Sent: Friday, September 23, 2022 11:11 AM
>> >> To: Leo Li <leoyang.li@xxxxxxx>; shawnguo@xxxxxxxxxx;
>> >> devicetree@xxxxxxxxxxxxxxx
>> >> Cc: robh+dt@xxxxxxxxxx; linux-arm-kernel@xxxxxxxxxxxxxxxxxxx; linux-
>> >> kernel@xxxxxxxxxxxxxxx; Laurentiu Tudor <laurentiu.tudor@xxxxxxx>
>> >> Subject: Re: [PATCH v2 5/9] arm64: dts: ls1046a: make dma-coherent
>> >> global to the SoC
>> >>
>> >>
>> >> Hi All,
>> >>
>> >> On 9/15/22 7:34 PM, Li Yang wrote:
>> >> > These SoCs are really completely dma coherent in their entirety so
>> >> > add the dma-coherent property at the soc level in the device tree
>> >> > and drop the instances where it's specifically added to a few select
>> devices.
>> >> >
>> >> > Signed-off-by: Laurentiu Tudor <laurentiu.tudor@xxxxxxx>
>> >> > Signed-off-by: Li Yang <leoyang.li@xxxxxxx>
>> >> > ---
>> >> >  arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 5 +----
>> >> >  1 file changed, 1 insertion(+), 4 deletions(-)
>> >> >
>> >> > diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
>> >> > b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
>> >> > index 27033c558e3e..e406499a26b4 100644
>> >> > --- a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
>> >> > +++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
>> >> > @@ -273,6 +273,7 @@ soc: soc {
>> >> >  		#size-cells = <2>;
>> >> >  		ranges;
>> >> >  		dma-ranges = <0x0 0x0 0x0 0x0 0x10000 0x00000000>;
>> >> > +		dma-coherent;
>> >> >
>> >> >  		ddr: memory-controller@1080000 {
>> >> >  			compatible = "fsl,qoriq-memory-controller"; @@ -
>> >> 355,7 +356,6 @@
>> >> > crypto: crypto@1700000 {
>> >> >  			ranges = <0x0 0x00 0x1700000 0x100000>;
>> >> >  			reg = <0x00 0x1700000 0x0 0x100000>;
>> >> >  			interrupts = <GIC_SPI 75 IRQ_TYPE_LEVEL_HIGH>;
>> >> > -			dma-coherent;
>> >> >
>> >> >  			sec_jr0: jr@10000 {
>> >> >  				compatible = "fsl,sec-v5.4-job-ring", @@ -
>> >> 794,7 +794,6 @@ pcie1:
>> >> > pcie@3400000 {
>> >> >  			#address-cells = <3>;
>> >> >  			#size-cells = <2>;
>> >> >  			device_type = "pci";
>> >> > -			dma-coherent;
>> >> >  			num-viewport = <8>;
>> >> >  			bus-range = <0x0 0xff>;
>> >> >  			ranges = <0x81000000 0x0 0x00000000 0x40
>> >> 0x00010000 0x0 0x00010000   /* downstream I/O */
>> >> > @@ -834,7 +833,6 @@ pcie2: pcie@3500000 {
>> >> >  			#address-cells = <3>;
>> >> >  			#size-cells = <2>;
>> >> >  			device_type = "pci";
>> >> > -			dma-coherent;
>> >> >  			num-viewport = <8>;
>> >> >  			bus-range = <0x0 0xff>;
>> >> >  			ranges = <0x81000000 0x0 0x00000000 0x48
>> >> 0x00010000 0x0 0x00010000   /* downstream I/O */
>> >> > @@ -874,7 +872,6 @@ pcie3: pcie@3600000 {
>> >> >  			#address-cells = <3>;
>> >> >  			#size-cells = <2>;
>> >> >  			device_type = "pci";
>> >> > -			dma-coherent;
>> >> >  			num-viewport = <8>;
>> >> >  			bus-range = <0x0 0xff>;
>> >> >  			ranges = <0x81000000 0x0 0x00000000 0x50
>> >> 0x00010000 0x0 0x00010000   /* downstream I/O */
>> >> >
>> >>
>> >> I'd like to summarize the conclusions of [1] below. This patch breaks
>> >> I2C0, which is the only user of eDMA at the moment. eDMA is
>> >> noncoherent because snooping is not enabled for it. I have submitted
>> >> a patch [2] to U- Boot to enable snooping for eDMA. For now, this
>> >> patch must add dma- noncoherent to the i2c0 node.
>> >
>> > I have sent a V3 yesterday to set dma-noncoherent on edma node.  But are
>> you saying that the dma-noncoherent need to be added to the i2c node to
>> make it work?
>> 
>> I believe dma coherency is a property of the consumer, not the provider. See
>> e.g.
>> really_probe/platform_dma_configure/of_dma_configure/of_dma_is_cohe
>> rent.
> 
> You are probably right.  The provider dma driver only maps the MMIO register while the consumer i2c driver maps the data buffer in memory which needs correct coherency setting.
> 
>> 
>> > For the u-boot patch, I will check with the hardware team to see if it is safe
>> to set the reserved bit for edma snooping.
>> 
>> Thanks. I'm curious as to whether this omission is intentional or not.
> 
> I have asked around but no one seems to know why LS1046a is different.  So from your experiment, changing the reserved bit does make a impact on the final result?

Yes.

> And setting it together with i2c nodes having dma-coherent enabled make it work.  Right?

Yes. But I only tested I2C0.

> If that's the case, probably it is just a documentation issue that we should fix.

Sounds like it.

--Sean