Skip to content

System hung when playback failed & copy debugfs to other folder immediately on BYT nocodec. #684

Closed

Description

@stevyan

Summary
System hung when playback failed & copy debugfs to other folder immediately on BYT nocodec.

Precondition
Use the aplay unsupported parameters to ensure the aplay will return failed.

Test step

  1. input following command within one line:
    aplay -Dhw:0,0 -f s24_le -r 48000 -c 2 /dev/zero;sudo su -c "timeout 1 cp -rf /sys/kernel/debug/sof /tmp/"

Test env
sof master: 58e90d9
kernel sof-dev: 65df8ef
tplg: sof-byt-nocodec.tplg

Log Info
Due to the device has hung, no any log output.

Activity

  1. added
    bugSomething isn't working
    BYTApplies to Baytrail platform
    on Mar 1, 2019
  2. plbossart commented on Mar 4, 2019

    @plbossart
    Member

    @xun2z What are you trying to do with the copy of the debugfs folder? What is this test about?

  3. xiulipan commented on Mar 5, 2019

    @xiulipan

    @plbossart
    This is a fail log collection method. We would like to copy the logs and debugfs once the test is failed. But during the test we found that the BYT will have system hang if we try to copy something at some fail case.

  4. plbossart commented on Mar 19, 2019

    @plbossart
    Member

    Is anyone assigned to look into this? @mengdonglin @wenqingfu this hasn't progressed in 2 weeks.

  5. xun2z commented on Mar 20, 2019

    @xun2z

    I tried it on my byt nocodec. Using the linux:95b2fab5, I see the hung whenever the 2nd time I run this command, the 1st is OK.

    To get the dmesg, I just use this command. Basically, It works well in fulfilling our origin purpose

    copy something at some fail case

    aplay -Dhw:0,0 -f s24_le -r 48000 -c 2 /dev/zero; sleep 5 && sudo su -c "timeout 1 cp -rf /sys/kernel/debug/sof /tmp/"

    [   64.784572] sof-audio-acpi 80860F28:01: pcm: open stream 0 dir 0
    [   64.784580] sof-audio-acpi 80860F28:01: period min 192 max 16384 bytes
    [   64.784584] sof-audio-acpi 80860F28:01: period count 2 max 16
    [   64.784589] sof-audio-acpi 80860F28:01: buffer max 65536 bytes
    [   64.785297] sof-audio-acpi 80860F28:01: rate_min: 48000 rate_max: 48000
    [   64.785302] sof-audio-acpi 80860F28:01: channels_min: 2 channels_max: 2
    [   64.785309] sof-audio-acpi 80860F28:01: rate_min: 48000 rate_max: 48000
    [   64.785314] sof-audio-acpi 80860F28:01: channels_min: 2 channels_max: 2
    [   64.785319] sof-audio-acpi 80860F28:01: rate_min: 48000 rate_max: 48000
    [   64.785323] sof-audio-acpi 80860F28:01: channels_min: 2 channels_max: 2
    [   64.785331] sof-audio-acpi 80860F28:01: pcm: hw params stream 0 dir 0
    [   64.785339] sof-audio-acpi 80860F28:01: generating page table for 00000000b490ebfc size 0xfc00 pages 16
    [   64.785345] sof-audio-acpi 80860F28:01: stream_tag 0
    [   64.785357] sof-audio-acpi 80860F28:01: ipc tx: 0x60010000: GLB_STREAM_MSG: PCM_PARAMS
    [   64.786804] sof-audio-acpi 80860F28:01: error: ipc error for 0x60010000 size 0x14
    [   64.786839] sof-audio-acpi 80860F28:01: error: hw params ipc failed for stream 0
    [   64.786857] sof-audio-acpi 80860F28:01: ASoC: 80860F28:01 hw params failed: -22
    [   64.786872]  Passthrough: ASoC: hw_params FE failed -22
    [   64.786902] sof-audio-acpi 80860F28:01: pcm: free stream 0 dir 0
    [   64.786916] sof-audio-acpi 80860F28:01: ipc tx: 0x60030000: GLB_STREAM_MSG: PCM_FREE
    [   64.787016] sof-audio-acpi 80860F28:01: ipc tx succeeded: 0x60030000: GLB_STREAM_MSG: PCM_FREE
    [   64.788393] sof-audio-acpi 80860F28:01: pcm: close stream 0 dir 0
    [   64.982923] sof-audio-acpi 80860F28:01: ipc rx: 0x90020000: GLB_TRACE_MSG
    [   64.982942] sof-audio-acpi 80860F28:01: ipc rx done: 0x90020000: GLB_TRACE_MSG
    [   69.897820] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   69.898390] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   69.899111] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   69.899775] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   69.899928] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   69.900081] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   69.900232] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   69.900381] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   69.900692] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   74.719195] sof-audio-acpi 80860F28:01: pcm: open stream 0 dir 0
    [   74.719203] sof-audio-acpi 80860F28:01: period min 192 max 16384 bytes
    [   74.719207] sof-audio-acpi 80860F28:01: period count 2 max 16
    [   74.719210] sof-audio-acpi 80860F28:01: buffer max 65536 bytes
    [   74.721249] sof-audio-acpi 80860F28:01: rate_min: 48000 rate_max: 48000
    [   74.721264] sof-audio-acpi 80860F28:01: channels_min: 2 channels_max: 2
    [   74.721277] sof-audio-acpi 80860F28:01: rate_min: 48000 rate_max: 48000
    [   74.721287] sof-audio-acpi 80860F28:01: channels_min: 2 channels_max: 2
    [   74.721299] sof-audio-acpi 80860F28:01: rate_min: 48000 rate_max: 48000
    [   74.721309] sof-audio-acpi 80860F28:01: channels_min: 2 channels_max: 2
    [   74.721325] sof-audio-acpi 80860F28:01: pcm: hw params stream 0 dir 0
    [   74.721341] sof-audio-acpi 80860F28:01: generating page table for 00000000b490ebfc size 0xfc00 pages 16
    [   74.721354] sof-audio-acpi 80860F28:01: stream_tag 0
    [   74.721373] sof-audio-acpi 80860F28:01: ipc tx: 0x60010000: GLB_STREAM_MSG: PCM_PARAMS
    [   75.025759] sof-audio-acpi 80860F28:01: error: ipc timed out for 0x60010000 size 0x6c
    [   75.025838] sof-audio-acpi 80860F28:01: error: unexpected fault 0x00000000 trace 0x60010000
    [   75.025852] sof-audio-acpi 80860F28:01: error: waking up any trace sleepers
    [   75.025869] sof-audio-acpi 80860F28:01: error: hw params ipc failed for stream 0
    [   75.025885] sof-audio-acpi 80860F28:01: ASoC: 80860F28:01 hw params failed: -110
    [   75.025900]  Passthrough: ASoC: hw_params FE failed -110
    [   75.025930] sof-audio-acpi 80860F28:01: pcm: free stream 0 dir 0
    [   75.025944] sof-audio-acpi 80860F28:01: ipc tx: 0x60030000: GLB_STREAM_MSG: PCM_FREE
    [   75.329711] sof-audio-acpi 80860F28:01: error: ipc timed out for 0x60030000 size 0xc
    [   75.329793] sof-audio-acpi 80860F28:01: error: unexpected fault 0x00000000 trace 0x60030000
    [   75.329806] sof-audio-acpi 80860F28:01: error: waking up any trace sleepers
    [   75.330755] sof-audio-acpi 80860F28:01: pcm: close stream 0 dir 0
    [   80.399021] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   80.400399] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   80.401109] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   80.401990] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   80.402185] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   80.402368] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   80.402690] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   80.402887] sof-audio-acpi 80860F28:01: Copying cached debugfs data
    [   80.403069] sof-audio-acpi 80860F28:01: Copying cached debugfs data

    So I think we have 2 issues here

    1. IPC will time out for a while rather than return Invalid argument the 2nd time.
    2. IPC timeout will conflict with debugfs copy.
  6. lgirdwood commented on Mar 20, 2019

    @lgirdwood
    Member

    @xun2z can you also switch on LOCK DEBUGGING with the kernel (use kconfig menuconfig) and wait for 2 minutes after the hang. This option should print out debug statements to the console if we are deadlocked with IRQs OFF.

  7. xun2z commented on Mar 21, 2019

    @xun2z

    I tried on the latest commit: ee46ece. This issue is fixed by Ranjani's pr #708.
    The corresponding commit number is 68bbdac
    reverted by 1238f67. The actual fix is 95dea6e

    1. IPC will time out for a while rather than return Invalid argument the 2nd time.

    This is fixed. So I think we can close this issue and reopen it when need.

    1. IPC timeout will conflict with debugfs copy.

    Using the old version, I still cannot return from hang after waiting a few minutes.
    enabled "Debug Lockups and Hangs", I can return to normal.
    dmesg.txt
    lockdep.txt
    lockdep_chains.txt
    lockdep_stats.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

BYTApplies to Baytrail platformbugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions