Yes - to some degree.
Settings - iCloud Drive - Options - Documents - deselect all Apps/folders you do not want to be there.
Have a look (I have no connection to the creator of this YouTube video):
The only issue is that you have to be quite vigilant not to enable the creation of folders (apps in iOS and what not).
In iOS, you can find it in Settings - iCloud. You can disable apps over there, too. If the app integrates well with the iOS Files app, you should be fine accessing iCloud Drive that way.
Be careful to check beforehand if you have any data inside of those “zombie” folders. I have not tried this myself, so you might first get yourself a backup of those folders.