What's the router model # ?
Might be able to give you more specifics if the info below doesn't resolve it.

There should be a reset button on the bottom or back of the router. Push the reset while powered on. Connect the Comcast ethernet cable into the WAN port and one of the computers Ethernet cable into one of the other ports.

Open a browser and type the routers IP address (D-link is typicaly 192.168.0.1) you should then be asked for the routers default password. ( http://www.answersthatwork.com/Downl..._addresses.pdf )
You should now be able to configure the router settings.
Setting Up Wireless Access Points - For Dummies