Hello DekHog,
It depends on if you want all icons to never show thumbnails preview windows and only lists, or just have it set to show a list after so many windows are open instead.
If never, then you can use one of the options that you like best in the tutorial below to do this.
Taskbar Thumbnail Previews - Enable or Disable If only after so many windows are open to have the list, then the tutorial below can help show you how to.
Taskbar List or Thumbnail Previews Mode - Change
Hope this helps,
Shawn