SDM660 FastRPC enablement

[setotau]

Currently I'm trying to enable FastRPC on the SDM660 (and SDM630/636 as well).
My commits regarding this:
sdm660-mainline/linux@8cdb63a
sdm660-mainline/linux@f22b0ff
sdm660-mainline/linux@0cd1afe (not sure about interconnect, vmids and dma-coherent properties)

When I'm trying to run fastrpc_test with the following command:
fastrpc_test -d 3 -U 1 -t linux -a v68
I'm getting errors like that:
https://paste.sr.ht/~setotau/4f0b3adb314a81254aa6a3aa05fb89d2cdeed40e
This also occurs when I'm trying to run hexagonrpcd with the following command:
sudo -u fastrpc hexagonrpcd -f /dev/fastrpc-cdsp -R /usr/share/qcom/sdm660/Xiaomi/clover/ -d cdsp -c /usr/share/qcom/sdm660/Xiaomi/clover/dsp/cdsp/fastrpc_shell_3
(with needed libs copied from the dspso partition on eMMC)
Also I've found out that it seems that both aDSP and cDSP use one memory region simultaneously (as defined in the downstream):
https://github.com/xiaomi-sdm660/android_kernel_xiaomi_sdm660/blob/11-EAS/arch/arm/boot/dts/qcom/sdm660.dtsi#L1691

[setotau]

@ekanshibu Can this series help with this issue since I'm getting SMMU errors or is this intended for new SoC's?

[ekanshibu]

Hi @setotau,

The series might not completely help but it might help getting the DSP logs to get mode details about the remoteproc crash if runtime farf feature is supported on this platform. Can you please help me with the DSP fw version that you're using here?

[setotau]

Hello @ekanshibu, sure

{
    "Image_Build_IDs": {
        "adsp": "ADSP.VT.4.0-00330-SDM660-1.186426.1", 
        "apps": "LA.UM.6.2.r1-08100-sdm660.0-1.158576.0.164567.1", 
        "boot": "BOOT.XF.1.4-00256-S660LZB-1", 
        "btfm": "BTFM.CHE.2.1.1.c2-00147-QCACHROMZ-1", 
        "cdsp": "CDSP.VT.1.0-00222-SDM660-1.186426.1", 
        "common": "SDM660.LA.2.1-21284-STD.PROD-1.186426.3.191687.1", 
        "cpev2": "CPE.TSF.2.0-00006-W9335AAAAAAAZQ-1", 
        "glue": "GLUE.SDM660_LA.2.0-00042-NOOP_TEST-1", 
        "modem": "MPSS.AT.3.1-00729.1-SDM660_GEN_PACK-1.186426.1.191687.1", 
        "rpm": "RPM.BF.1.8-00052-SDM660AAAAANAZR-1", 
        "tz": "TZ.BF.4.0.7-00089-S660AAAAANAZT-2", 
        "video": "VIDEO.VE.4.4-00034-PROD-1", 
        "wdsp": "WDSP.9340.1.0.c1-00008-W9340AAAAAAAZQ-1", 
        "wlan": "WLAN.HL.1.0.1.c2-00392-QCAHLSWMTPLZ-1.158576.1.186426.3"
    }, 
    "Metabuild_Info": {
        "Meta_Build_ID": "SDM660.LA.2.1-21284-STD.PROD-1.186426.3.191687.1", 
        "Product_Flavor": "['asic']", 
        "Time_Stamp": "2019-05-15 12:00:47"
    }, 
    "Version": "1.0"
}

[ekanshibu]

this looks to be very old DSP version with some old design of logger framework. Can you try creating a fastrpc_test.farf file with 0x1f001f as it's content and push it to /usr/lib/rfsa/adsp/ path before running the fastrpc_test app? If the logger is stable, you will see DSP logs in syslog.

[setotau]

I've done this, but I'm getting no DSP logs, only this:

Jan 06 21:12:05 xiaomi-clover [5029]: user ran command fastrpc_test as root from /home/user
Jan 06 21:12:05 xiaomi-clover [5029]: fastrpc_apps_user.c:4204: fastrpc_apps_user_init done. default domain:3 and &fastrpc_trace:0xffffafa237d0
Jan 06 21:12:05 xiaomi-clover [5029]: fastrpc_apps_user.c:4325: multidsplib_env_init: libcdsprpc.so loaded
Jan 06 21:12:05 xiaomi-clover [5029]: fastrpc_apps_user.c:2699: remote_session_control Unsigned PD enable 1 request for domain 3
Jan 06 21:12:05 xiaomi-clover kernel: arm-smmu 5180000.iommu: Unhandled context fault: fsr=0x402, iova=0xfffff000, fsynr=0x1, cbfrsynra=0x5, cb=2
Jan 06 21:12:05 xiaomi-clover kernel: arm-smmu 5180000.iommu: FSR    = 00000402 [Format=2 TF], SID=0x5
Jan 06 21:12:05 xiaomi-clover kernel: arm-smmu 5180000.iommu: FSYNR0 = 00000001 [S1CBNDX=0 PLVL=1]
Jan 06 21:12:05 xiaomi-clover kernel: qcom_q6v5_pas 1a300000.remoteproc: fatal error received: :0:EX:kernel:0:frpck_0_0:77:PC=c0117de0
Jan 06 21:12:05 xiaomi-clover kernel: remoteproc remoteproc2: crash detected in cdsp: type fatal error
Jan 06 21:12:05 xiaomi-clover kernel: remoteproc remoteproc2: handling crash #1 in cdsp
Jan 06 21:12:05 xiaomi-clover kernel: remoteproc remoteproc2: recovering cdsp
Jan 06 21:12:05 xiaomi-clover kernel: (NULL device *): Error: dsp information is incorrect err: -32

Can I collect DSP logs into journald?

[setotau]

I've also enabled DMA heap in the kernel config, but it didn't help

[setotau]

Maybe I need to setup diag for log collection?

[setotau]

Are there required kernel configs for FastRPC? Maybe I've missed something?

[ekanshibu]

I've done this, but I'm getting no DSP logs, only this:

Jan 06 21:12:05 xiaomi-clover [5029]: user ran command fastrpc_test as root from /home/user
Jan 06 21:12:05 xiaomi-clover [5029]: fastrpc_apps_user.c:4204: fastrpc_apps_user_init done. default domain:3 and &fastrpc_trace:0xffffafa237d0
Jan 06 21:12:05 xiaomi-clover [5029]: fastrpc_apps_user.c:4325: multidsplib_env_init: libcdsprpc.so loaded
Jan 06 21:12:05 xiaomi-clover [5029]: fastrpc_apps_user.c:2699: remote_session_control Unsigned PD enable 1 request for domain 3
Jan 06 21:12:05 xiaomi-clover kernel: arm-smmu 5180000.iommu: Unhandled context fault: fsr=0x402, iova=0xfffff000, fsynr=0x1, cbfrsynra=0x5, cb=2
Jan 06 21:12:05 xiaomi-clover kernel: arm-smmu 5180000.iommu: FSR    = 00000402 [Format=2 TF], SID=0x5
Jan 06 21:12:05 xiaomi-clover kernel: arm-smmu 5180000.iommu: FSYNR0 = 00000001 [S1CBNDX=0 PLVL=1]
Jan 06 21:12:05 xiaomi-clover kernel: qcom_q6v5_pas 1a300000.remoteproc: fatal error received: :0:EX:kernel:0:frpck_0_0:77:PC=c0117de0
Jan 06 21:12:05 xiaomi-clover kernel: remoteproc remoteproc2: crash detected in cdsp: type fatal error
Jan 06 21:12:05 xiaomi-clover kernel: remoteproc remoteproc2: handling crash #1 in cdsp
Jan 06 21:12:05 xiaomi-clover kernel: remoteproc remoteproc2: recovering cdsp
Jan 06 21:12:05 xiaomi-clover kernel: (NULL device *): Error: dsp information is incorrect err: -32

Can I collect DSP logs into journald?

Should be possible, can you share full journalctl and syslog for this run?

Maybe I need to setup diag for log collection?

If you can set-up diag to capture QXDM logs, that would also work in further understanding the failure.

Are there required kernel configs for FastRPC? Maybe I've missed something?

Nothing extra is required, things should work with the default configs.

[setotau]

Should be possible, can you share full journalctl and syslog for this run?

Yes. https://paste.sr.ht/~setotau/bddee2a36fc04a9845a0ee8750cdb781dd504257

[ekanshibu]

From logs, it looks like the PD spawn request is causing some SMMU fault coming from SID 0x5.

I'm assuming that you're using this DT for your testing: https://lore.kernel.org/all/20251023-qcom-sdm660-cdsp-adsp-dts-v2-1-895ffe50ab5f@mainlining.org/

Looking at this, the first iommu context bank is being used and this is causing a remoteproc crash in the first call itself.

I'm suspecting there might be some problem with the compute-cb@5 configuration and for this I'm trying to get more details internally.

Meanwhile, can you try running the cdsprpcd daemon process and see if it is also giving the same problem. If not, pls help with journalctl for that as well.

[setotau]

I've ran cdsprpcd and got this:

Jan 08 20:57:45 xiaomi-clover [3286]: user ran command cdsprpcd as root from /home/user
Jan 08 20:57:45 xiaomi-clover [3286]: dsprpcd.c:96:CDSP daemon starting
Jan 08 20:57:45 xiaomi-clover [3286]: fastrpc_apps_user.c:4204: fastrpc_apps_user_init done. default domain:3 and &fastrpc_trace:0xffff8e6867d0
Jan 08 20:57:45 xiaomi-clover [3286]: fastrpc_apps_user.c:4325: multidsplib_env_init: libcdsprpc.so loaded
Jan 08 20:57:45 xiaomi-clover [3286]: dsprpcd.c:103:adsp_default_listener_start called
Jan 08 20:57:45 xiaomi-clover [3286]: fastrpc_apps_user.c:1782: remote_handle_open: Successfully opened handle 0x103 for '":;./\attachguestos on domain 3 (spawn time 0 us, load time 0 us)
Jan 08 20:57:45 xiaomi-clover kernel: arm-smmu 5180000.iommu: Unhandled context fault: fsr=0x402, iova=0xfffff000, fsynr=0x1, cbfrsynra=0x5, cb=2
Jan 08 20:57:45 xiaomi-clover kernel: arm-smmu 5180000.iommu: FSR    = 00000402 [Format=2 TF], SID=0x5
Jan 08 20:57:45 xiaomi-clover kernel: arm-smmu 5180000.iommu: FSYNR0 = 00000001 [S1CBNDX=0 PLVL=1]
Jan 08 20:57:45 xiaomi-clover kernel: qcom_q6v5_pas 1a300000.remoteproc: fatal error received: :0:EX:kernel:0:frpck_0_0:108a:PC=c0117de0
Jan 08 20:57:45 xiaomi-clover kernel: remoteproc remoteproc2: crash detected in cdsp: type fatal error
Jan 08 20:57:45 xiaomi-clover kernel: remoteproc remoteproc2: handling crash #1 in cdsp
Jan 08 20:57:45 xiaomi-clover kernel: remoteproc remoteproc2: recovering cdsp
Jan 08 20:57:45 xiaomi-clover kernel: (NULL device *): Error: dsp information is incorrect err: -32
Jan 08 20:57:45 xiaomi-clover [3286]: fastrpc_cap.c:132: Warning 0xffffffff: fastrpc_get_cap failed to get attribute 131 for domain 3 (errno Broken pipe)
Jan 08 20:57:45 xiaomi-clover [3286]: fastrpc_apps_user.c:3347: Error 0x0: open_device_node failed for domain ID 3, sess ID 0 secure dev : /dev/fastrpc-cdsp-secure, dev : /dev/fastrpc-cdsp>
Jan 08 20:57:45 xiaomi-clover [3286]: fastrpc_apps_user.c:3832: Error 0x72: remote_init failed for domain 3, errno No such file or directory, ioErr 0
Jan 08 20:57:45 xiaomi-clover [3286]: fastrpc_apps_user.c:4043: Error 0x72: domain_init (-1) failed for domain 3 (errno No such file or directory)
Jan 08 20:57:45 xiaomi-clover [3286]: fastrpc_apps_user.c:3347: Error 0x0: open_device_node failed for domain ID 3, sess ID 0 secure dev : /dev/fastrpc-cdsp-secure, dev : /dev/fastrpc-cdsp>
Jan 08 20:57:45 xiaomi-clover [3286]: fastrpc_cap.c:132: Warning 0x72: fastrpc_get_cap failed to get attribute 257 for domain 3 (errno No such file or directory)
Jan 08 20:57:45 xiaomi-clover [3286]: fastrpc_apps_user.c:1778: Error 0x72: remote_handle_open failed for adsp_default_listener (errno No such file or directory)
Jan 08 20:57:45 xiaomi-clover [3286]: adsp_default_listener.c:380::error: 114: 0 == (nErr = remote_handle_open("adsp_default_listener", (remote_handle *)&listener_fd))
Jan 08 20:57:45 xiaomi-clover [3286]: adsp_default_listener.c:446:Error 0x72: adsp_default_listener_start exiting
Jan 08 20:57:45 xiaomi-clover [3286]: dsprpcd.c:114:fastRPC device is not accessible, daemon exiting...
Jan 08 20:57:45 xiaomi-clover [3286]: dsprpcd.c:123:daemon exiting 72
Jan 08 20:57:45 xiaomi-clover [3286]: fastrpc_apps_user.c:4097: fastrpc_apps_user_deinit done
Jan 08 20:57:45 xiaomi-clover kernel: remoteproc remoteproc2: stopped remote processor cdsp
Jan 08 20:57:45 xiaomi-clover kernel: remoteproc remoteproc2: remote processor cdsp is now up
Jan 08 20:57:45 xiaomi-clover kernel: qcom,fastrpc 1a300000.remoteproc:glink-edge.fastrpcglink-apps-dsp.-1.-1: assigned reserved memory node adsp-region@f6000000
Jan 08 20:57:45 xiaomi-clover kernel: platform 1a300000.remoteproc:glink-edge:fastrpc:compute-cb@5: Adding to iommu group 8
Jan 08 20:57:45 xiaomi-clover kernel: platform 1a300000.remoteproc:glink-edge:fastrpc:compute-cb@6: Adding to iommu group 9
Jan 08 20:57:45 xiaomi-clover kernel: platform 1a300000.remoteproc:glink-edge:fastrpc:compute-cb@7: Adding to iommu group 10
Jan 08 20:57:45 xiaomi-clover kernel: platform 1a300000.remoteproc:glink-edge:fastrpc:compute-cb@8: Adding to iommu group 11
Jan 08 20:57:45 xiaomi-clover kernel: platform 1a300000.remoteproc:glink-edge:fastrpc:compute-cb@9: Adding to iommu group 12
Jan 08 20:57:45 xiaomi-clover kernel: platform 1a300000.remoteproc:glink-edge:fastrpc:compute-cb@10: Adding to iommu group 14
Jan 08 20:57:45 xiaomi-clover kernel: platform 1a300000.remoteproc:glink-edge:fastrpc:compute-cb@11: Adding to iommu group 15
Jan 08 20:57:45 xiaomi-clover kernel: platform 1a300000.remoteproc:glink-edge:fastrpc:compute-cb@12: Adding to iommu group 16
Jan 08 20:57:45 xiaomi-clover kernel: platform 1a300000.remoteproc:glink-edge:fastrpc:compute-cb@13: Adding to iommu group 17

Still can't collect logs through diag-router

[setotau]

From logs, it looks like the PD spawn request is causing some SMMU fault coming from SID 0x5.

I'm assuming that you're using this DT for your testing: https://lore.kernel.org/all/20251023-qcom-sdm660-cdsp-adsp-dts-v2-1-895ffe50ab5f@mainlining.org/

I'm using this DT with these changes (with adsp-region assigning to the CDSP's FastRPC and with non-secure-domain property): sdm660-mainline/linux@0cd1afe

[setotau]

I'm suspecting there might be some problem with the compute-cb@5 configuration and for this I'm trying to get more details internally.

context bank numeration is taken from downstream: https://github.com/xiaomi-sdm660/android_kernel_xiaomi_sdm660/blob/11-EAS/arch/arm/boot/dts/qcom/sdm660.dtsi#L1696