First off, let me give you a little bit of background. I have been using macOS for about a year straight as my primary OS. I have been uninstalling programs the way I have always heard online, like holding down “option” in launchpad to uninstall apps from the Mac App Store and simply dragging applications into the trash for third-party ones. I’ve recently discovered that when uninstalling apps by one of the two methods I just mentioned, the app leaves behind .conf and/or .plist files in folders such as “Application Support”. I looked into this to try and find a way to remove even these files after uninstalling an app, and I came across a handy little utility named “AppCleaner”. It has been great, but I was wondering how other people view this “issue” or other apps they might use to get the job done. I know that giving Full Disk Access to niche software can be risky, I just wanted to hear what some other power users thought! 
Hazel has a feature called “App Sweep” that locates those left-behind bits and pieces of uninstalled apps and offers to trash them. Since I’ve used Hazel from the first release, I trust it fully.
Katie
Most of the advice you’ll hear about this is not to worry about it. Those files don’t cause any harm, and if you change your mind and want to reinstall the app again later you’ll still have your preferences. Personally, I go through my Library and drag things I know I won’t need to the trash every few months. But, if you have any question at all about if you should be deleting something from your Library, the answer is probably to just leave it alone.
I like a clean system too, but most of the time I think 3rd party uninstallers do more harm than good, or at best are simply unnecessary.
I’ve used AppCleaner for years and I’ve never had a problem.
I don’t like to clear out those kinds of files, because usually if I for some reason want to reinstall the app again it becomes useful to have all my previous settings and such.
The exception to the above being if there is something specific that I know I won’t need that is also taking up a gargantuan amount of space (1GB+).
I’ve stuck with the AppDelete app uninstaller because it gives me the option to archive an app before I uninstall it. A few times I have changed my mind about deleting an app. If I have archived before deleting I can use AppDelete to unarchive the deleted app and I’m back in business! It is also handy if I am not sure about the new version of an app and want to be able to downgrade if needed.
I’ve used AppCleaner for years and really like it. It works great and it shows you what it’s going to delete so you can uncheck the items (if any) that you don’t want to delete.
I use CleanMyMac which can uninstall apps, or clean up afterword when you drag one to the trash.
As others have said, most of these files are small and innocuous.
I usually uninstall apps with Raycast or Forklift. Both of them will display a dialog asking you if you want to remove the additional stuff.