Custom Search Engine - Missing sites?

[Arron Z. Daniels]

Hello all,

I wanted to pose this question to the community to see if we could find a solution.  When creating a Custom Search Engine (CSE) you are required to first add a site to start with:

So exploring a little further, what about finding sites that don't have the typical generic top level domain names?   For instance the site About.me wouldn't be indexed in the search results because it doesn't have a standard web suffix (.com or .com or .org).  

Has anyone found a fix for this? 

[Dave Galley]

Arron,

There is a setting you can change, once you create the CSE. Go to the edit page, and it's on the Basic tab. Find the is a dropdown which you can switch between "search only the selected sites" and "search the web but emphasize the selected sites", and change it to the latter. So you could just do *.com and it would return all but emphasize those (higher in results).

If you want equal weighting across many domains, just add all the ones you're interested in. If you don't have particular ones you're interested in, add them all! From the same Basic tab, click "add" under the Sites to Search section, and there is an option to "add in bulk". You could just add the entire list of valid TLDs.

I have a file prepared for just this purpose (attached). You can just cut and past into the bulk add box. May require updating once every couple of years. :)

[Arron Z. Daniels]

That's fantastic!  I am trying to create a CSE that is a catch all.  I started making a spreadsheet last night, so I am glad you shared, thanks. When I get done I will send you the link to it  Give it a look.

[Dave Galley]

Sure thing. TLDs are set by a global standards body and the canonical list is always freely available. :)

Just be careful not to try to reinvent Google, you know? Too much of a catch-all is potentially not worth it.

[Arron Z. Daniels]

Good point. I am making a one that will x-ray my favorite places to find profiles.  I can see where getting too far will (as you said) "reinvent Google." 

[Zulfiqar Rizvi (aka Zulfi)]

We could add for example www.about.me/* in the sites we add.

Dave please correct me if I am wrong.

[Arron Z. Daniels]

Zulfiqar,

I believe what Dave is saying that the setting he was speaking of in the CSE will allow the engine to index all sites, but give priority to the ones I have added.  So about.me will be searched and indexed in the search results, but lower than my added sites.  

How are you fairing with the all mighty CSEs?  

Thanks!

Arron