What type of file structure are you browsing? In other words, a Win2k3/2k8 share or perhaps a different type of file server?
What are you using to browse? Is (slow) Explorer-like behaviour also evident if you say map a drive on the command prompt and attempt to browse the file structure that way? (NET USE * \\serverName\shareName
to map, DIR to list)
Are you using the in-built PPTP client or something 3rd-party?