In order to fix such an issue ... After starting up the Windows, see if the red x cross still appears on the volume icon. If it is gone away then good, else, continue to the next solution.

The Volume Control icon in the system tray is the quickest way to control the volume on your Windows 11/10 PC. However if after a recent update or a software installation, it no longer responds ...