Apple 4K remote Siri not hearing me

I have tried everything rebooting the Apple 4K charging the battery on the remote and holding the volume up and back button to have the remote recognize the Apple 4K but nothing has fixed Siri saying it cannot hear me or having trouble hearing me when I try to use it on the remote. When I use the Apple remote on my iPhone Siri works fine as it should with common request like pausing or playing or repeating. Has anyone had this issue and know what the solution is?

I had once some dust/dirt inside the small hole with the mic.