The video in YT and the music played in YTMusic are two different uploads, you can easily get one in YT by checking the YTM URL and getting the ID. So yeah, yt-dlp should get you only the song if you created a playlist with only songs instead of music videos.