Repository navigation
Remove DSP after unregistering the machine driver - #863
Conversation
plbossart
left a comment
There was a problem hiding this comment.
I don't get why this pops up now? what are we trying to fix, is this a regression or something that was never tested?
@plbossart I dont understand that either. It is a regression as pointed out in #861 but this should have been caught much earlier. |
|
I'd like to know the last time this test was run. It should be part of the daily validation. |
The test run on ww16.5 shows the result as a PASS. But I tested with the same kernel commit as in the test and it failed. So I am not sure what went wrong |
|
@ranj063 @plbossart @wenqingfu sorry for the test result mistake & confusion, of cause, this case already added to our daily validation, however, there are more dependency relationships for the sof driver modules, we always use 'kmod_scripts/sof_bootloop.sh' to execute the reload case, actually, the script missed the 'Panic' failure checking, hence, it adjusted the test 'PASS' wrongly. I'll correct all the corresponding adjusting condition - 'Panic' in relative test scripts. |
|
This has been clarified as an unfortunate test case failure. The test case has been fixed. Ping for update on patch review. |
|
@plbossart this was a missed failure when the change to make the component driver resource managed was implemented. Prior to that, we used to unregister the component driver before calling snd_sof_dsp_remove(). |
|
ping... @plbossart |
|
we had an explicit code move with commit 6b7db56, and I don't get why we are reverting this, see e.g. the previous comment: Besides, there are still issues, e.g. snd_soc_unregister_component(sdev->dev); is still used in the core while we moved to devm_ functions. In short it's a mess. And I will not merge RFC. |
|
not to mention that 2 approvals are needed as agreed. |
@plbossart this commit was merged some time before we made changes recently in the core to alter the sequence for unloading the SOF module. So, I wont use this against this PR as we dont even have an unregister_component anymore. The component dev is resource managed now and it is unregistered when the sof device is removed. Having said that, I am not even asking for merging this at this point but only for some feedback. The fix works but I do not understand why the codec topology doesnt face the same issue. Like I mentioned in my description, if the DSP is removed before we unload the modules, unloading the pipeline widget should fail. This is what we see with the nocodec topology but not with the codec topology. I'll work with @bardliao on Monday to see if we can come up with an explanation for this. |
|
@plbossart @bardliao I now know the reason it works for the pcm512x topology and doesnt work with the nocodec topology. In the case of pcm512x, hda_dsp_remove() calls the snd_hdac_ext_bus_device_remove() to remove the codec which unregisters the component driver for the hdmi codec. The component device for the hdmi codec is the same as the component device for the SOF pcm component driver. So unregistering hdmi codec ends up unregistering the SOF component driver thereby removing the topology. This happens during: linux/sound/soc/sof/intel/hda.c Line 646 in 1071743 And so the topology is removed before the DSP is removed. And this is followed by unregistering the machine driver which attempts to remove the topology but the topology has already been removed. So all is good. But in the case of nocodec topology, hda_dsp_remove() removes the DSP first. Then unregistering the machine driver ends up removing topology and this crashes because the DSP has been removed already. |
3fae09d to
f8d3fb9
Compare
bardliao
left a comment
There was a problem hiding this comment.
The explanation is clear, and the solution makes sense to me
| if (!IS_ERR_OR_NULL(pdata->pdev_mach)) | ||
| platform_device_unregister(pdata->pdev_mach); | ||
|
|
||
| /* remove DSP */ |
There was a problem hiding this comment.
Please add some comments nearby to show the correct sequence of driver removing. You explanation in PR is very nice. Summarize and add to here will make sure others understand that.
| return 0; | ||
|
|
||
| comp_err: | ||
| snd_soc_unregister_component(sdev->dev); |
There was a problem hiding this comment.
Is this really helpful for fixing #861 or only changing it while you are at here? My understanding is that we can invoke unregister specifically for resource-managed component device.
There was a problem hiding this comment.
yup, just while i'm at it. Not relevant for #861. Only the first commit is needed to fix the issue
There was a problem hiding this comment.
I'm not sure this is a good idea. This is a .probe() "bottom half," it's often performed in a work context after the actual probing has completed and returned success. Usually when you use managed allocations, it's done within .probe(), so that a failure leads to the .probe() failing and then all such allocations are freed. In this case an allocation failure will not lead to .probe() failing, so, the resource will stay allocated. This isn't bad, it will be freed if the modules are unloaded, but it's also not useful to keep it in the error case. So, maybe better to keep that unregister.
There was a problem hiding this comment.
@lyakh i havent tried this yet but what happens if you leave the unregister here, the machine driver registration fails and then you try to unload sof device?
There was a problem hiding this comment.
@ranj063 I haven't tried either. Nothing bad should happen at the resource management level, you have already unregistered the resource, so, no second unregistering will be attempted by the resource management system. But I don't know whether the driver's .release() method would choke because it assumes that the registration has completed successfully.
There was a problem hiding this comment.
@lyakh i just tried it and nothing bad happened. I am OK to leave it here or remove it. removing it makes more sense to me but if you prefer leaving it here i'm OK with it too.
|
|
||
| /* remove DSP */ | ||
| snd_sof_remove(sdev); | ||
|
|
There was a problem hiding this comment.
Good catch, but still one question unclear to me:
In your commit message it is stated "snd_hdac_ext_bus_device_remove() to remove the codec which unregisters the component driver thereby also removing the topology.", per my understanding, this only remove the codec soc component(not the soc component we register here?), how does it removing the topology.
There was a problem hiding this comment.
@keyonjie take a look at __snd_soc_unregister_component() in soc-core.c. It removes the tplg for all components who's dev matches the input argument "dev". Now, in our case both the hdmic codec component device and the SOF component device are the same and thats why snd_hdac_ext_bus_device_remove() results in removing the topology
There was a problem hiding this comment.
yes, if we share the "dev" in both hdmi codec and SOF component, that explains, thanks.
f8d3fb9 to
c69380a
Compare
plbossart
left a comment
There was a problem hiding this comment.
Thanks @ranj063. I find your commit message verbose and convoluted, with multiple repeats, and leaves open questions.
Suggested edits:
Unregistering the machine driver after removing the DSP results in a kernel panic during the removal
of the pipeline widget in topology, which will attempt to power down the core on which it is scheduled by access DSP registers after snd_sof_remove() disables the DSP and unmaps the DSP BAR.
The suggested fix is to unregister the machine driver
first and then remove the DSP to avoid the situation described
above.
Note that the kernel panic only happens in the case of cases where the HDaudio link is not managed by the hdac library, e.g. no codec or cases where HDMI is not supported with an I2S codec.
When the hdac library is used, snd_sof_remove() calls
snd_hdac_ext_bus_device_remove() to remove the codec which
unregisters the component driver thereby also removing the
topology.
--->>> but in the latter case, unregistering the machine driver first will result in error message "Topology forgot to remove link %d", no??
c69380a to
0df8885
Compare
@plbossart fixed commit message now. Also, I have not seen the above error in either case. Could you please give me some more details on which platform you see this error on? |
snd_sof_remove() disables the DSP and unmaps the DSP BAR. Removing topology after disabling the DSP results in a kernel panic while unloading the pipeline widget. This is because pipeline widget unload attempts to power down the core it is scheduled on by accessing the DSP registers. So, the suggested fix here is to unregister the machine driver first to remove the topology and then disable the DSP to avoid the situation described above. Note that the kernel panic only happens in cases where the HDaudio link is not managed by the hdac library, e.g. no codec or when HDMI is not supported. When the hdac library is used, snd_sof_remove() calls snd_hdac_ext_bus_device_remove() to remove the codec which unregisters the component driver thereby also removing the topology before the DSP is disabled. Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
No need to call snd_soc_unregister_component in case of error because the component device is resource-managed. Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
0df8885 to
5a1a793
Compare
I haven't seen this error, just checking if we are abusing the topology code by doing a cleanup during the card release instead of at the component release. We've circled on this multiple times and I am not happy to change courses without knowing what I am really signing up for. |
@plbossart we're not abusing the topology code. The only difference between the 2 cases is when the topology is removed. In the codec case, it is removed when the hdmi component is unregistered and in the nocodec it is removed when the machine driver is unregistered. In terms of the flow, they are both exactly the same. |
Sorry, I don't see how removing an HDMI codec would result in the SOF component being removed as well. This is also convoluted in the commit message below: When the hdac library is used, snd_sof_remove() calls I am missing a link here between (a) and (b) |
@plbossart every single person in the comments has asked the same question. So let me clarify it again: If you look at the parent-child hierarchy in SOF, both the SOF component device and the HDMI codec component device are the same (which is the SOF device). And when the HDMI codec is removed, we unregister the component device which calls __snd_soc_unregister_component() in soc-core.c. This function removes the topology for all components whose device matches the input argument device. Since both the HDMI component and the SOF pcm component's have the same dev, the topology gets removed as well. Does that make sense? Now, I did submit a patch to alsa-dev to remove the topology removal code from unregister_component. But mark preferred to keep it to make the device removal robust. |
Ah, that almost makes sense then, but I don't recall how these two separate parts of the code end-up using the same device. Any pointers?
Yes, and that's fine. |
@plbossart this is a bit convoluted but what I see the the hdac_device's dev member points to the SOF device. ANd when the HDMI driver regsiters the component driver, this is the device that gets passed to it. |
care to add a pointer for the rest of us? Thanks! |
@plbossart yes sure, I'm looking. Will add it when I find it. |
|
@plbossart @keyonjie @bardliao i was so very wrong in my assumption on why the topology gets removed. But now I know what happens but it is still very convoluted. Let me try anyway. There are 2 reasons for why the topology gets removed when the HDMI codec is removed:
As for 2, I feel like this might be a mistake in the ASoC core. Deleting a component shouldnt unbind the card. This is OK if there is only ever one component associated with the card but in the case of multiple components, it shouldnt happen. |
Thanks @ranj063. it'd be useful to represent the sequence with a call graph, e.g. so that we can track the logic. It's very hard with sentences to follow the code. |
|
@plbossart here's the pseudo call trace for the 2 cases I explained in my previous message.
snd_hdac_ext_bus_device_remove(ehdaudio0D2) -> bus_remove_device(ehdaudio0D2) -> device_release_driver(ehdaudio0D2) -> device_release_driver(bxt-pcm512x) The last call to release the machine driver is what removes the topology. I am not sure what happens in the release() callback for the machine driver but this ends up removing the topology. @libinyang is looking into what happens and can add more details.
This is what I feel is likely a mistake. Unregistering the HDMI codec component, results in unbinding the card. This is fine if there is only ever one component for the card but shouldnt be the case when there are more than 1 like in our case where we have the SOF component and the HDMI codec component. |
|
@libinyang @plbossart I have a bit more information on why the device link removal causes the machine driver to be released as well. I think this is the expected behaviour with device links. The device link ensures that the consumer (m/c drv) cannot be probed before the supplier (hdmi codec). And if the supplier is removed, the consumer should be unbound and released. So this is fine and there are no changes needed. But this will mean that when the m/c driver is released, the topology will be removed as well. |
plbossart
left a comment
There was a problem hiding this comment.
I have a migraine reading all the comments. This looks like a mess but I will have to trust this fix is legit.
I need someone else to approve (only the 3rd PR this morning that is stuck for lack of reviewers).
bardliao
left a comment
There was a problem hiding this comment.
The fix looks good to me. In fact, I think the HDMI case is happen to work. So, we still need the fix.
…hen look_up_lock_class() got confused Printing this information will be helpful: ------------[ cut here ]------------ Looking for class "l2tp_sock" with key l2tp_socket_class, but found a different class "slock-AF_INET6" with the same key WARNING: CPU: 1 PID: 14195 at kernel/locking/lockdep.c:940 look_up_lock_class+0xcc/0x140 Modules linked in: CPU: 1 PID: 14195 Comm: a.out Not tainted 6.0.0-rc6-dirty thesofproject#863 Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 RIP: 0010:look_up_lock_class+0xcc/0x140 Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/bd99391e-f787-efe9-5ec6-3c6dc4c587b0@I-love.SAKURA.ne.jp
Unregistering the machine driver results in the removal of topology components. Some components like the pipeline component, when they are unloaded will attempt to power down the core that it is scheduled on.
snd_sof_remove() disables the DSP and unmaps the DSP BAR from the kernel space. Unnregistering the machine driver after removing the DSP results in a kernel panic during the removal of the pipeline widget in topology. This is because unloading the pipeline widget attempts to power down the core that the pipeline is scheduled on. While attempting the power down, the core tries to read the DSP status and this results in a kernel oops because the DSP BAR has been unmapped already.
So, the suggested fix here is to unregister the machine driver first and then remove the DSP to avoid the situation described above.
Also note that the above situation only happens in the case of nocodec topology. In the case of topologies with HDA pipelines(either iDisp or HDA codec), the above situation is avoided because the topology is unloaded before the DSP BAR is unmapped. This happens because snd_sof_remove() calls
snd_hdac_ext_bus_device_remove() to remove the codec which unregisters the component driver thereby also removing the topology.
Fixes #861