since there's no local dns server they're connecting to, they won't know how to resolve the names to the IP's. If you type "ping (yourpcname)" in a commandline (start>search>cmd and click the first result) on one of the clients does it resolve to an IP or just say "ping request could not find host"?
For a solution either way, you can do this:
Go to C:\windows\system32\drivers
\etc and open the "hosts" file in notepad.
At the end of the text start a new line (don't include a star\asterisk in front this time) and type in the remote PC's IP, then a space and then the FQDN (which you can find by right-clicking Computer>properties and looking at "Full computer name"). Then save and exit and try RDP with the workstation name again.
You need to do this to each computer, but this is the best solution because you only have a few. There may be a full-on fix and some weird local-dns-emulation with homegroups but I'm far more familiar with domains, as "homegroups" are newer (usually temporary) implementations from Microsoft that seem to get phased out and completely redesigned with every new Windows.
Hopefully that helps.